{"id":5396,"date":"2018-02-18T13:00:05","date_gmt":"2018-02-18T13:00:05","guid":{"rendered":"http:\/\/writeasync.net\/?p=5396"},"modified":"2018-02-14T03:36:00","modified_gmt":"2018-02-14T03:36:00","slug":"a-real-async-getfiles","status":"publish","type":"post","link":"http:\/\/writeasync.net\/?p=5396","title":{"rendered":"A real async GetFiles?"},"content":{"rendered":"

I’ve lamented in the past that there is no real async GetFiles<\/a>. But that’s okay — we’re problem solvers! Perhaps if we could drop down to the core native API, we could fill in a gap like this. Let’s start by figuring out how GetFiles is implemented in .NET. Searching in the .NET Core source code, we eventually find a FileSystemEnumerator class for Win32<\/a> which calls NtQueryDirectoryFile<\/a>.<\/p>\n

Ah, but there are some problems. First, since this is an API from the NT native layer<\/a>, we are not working with the kinds of constructs that we are used to, such as OVERLAPPED structures<\/a> and the like. (From previous experience, we know that overlapped I\/O is actually quite doable in .NET<\/a>.) Instead, we would have to do some sort of complex APC magic<\/a> — not so straightforward in managed code! Second, it seems this might not even work if we tried, if the wisdom of Stackoverflow<\/a> is to be believed.<\/p>\n

Do we give up hope? Not necessarily! Checking the documentation for the structures returned from NtQueryDirectoryFile (say, FILE_FULL_DIR_INFORMATION<\/a>), we see that there is another way to get this information. Specifically, we could “create an IRP with major function code IRP_MJ_DIRECTORY_CONTROL and minor function code IRP_MN_QUERY_DIRECTORY.” I guess that clears it up, aside from three questions I would have about that sentence.<\/p>\n

An IRP (pronounced “urp”<\/a>) is an I\/O request packet<\/a>. These are mainly used by drivers to perform requests on the file system or network. IRP_MJ_DIRECTORY_CONTROL is major function code<\/a> which tells a driver what general type of function needs to be performed (think: “class”) — in this case it would refer to the set of directory handling routines. IRP_MN_QUERY_DIRECTORY is a minor function code which exactly describes the requested operation (think: “method”)<\/p>\n

This is all well and good, but how would we even send an IRP from user mode? Unfortunately, there aren’t any ways to send arbitrary requests. We have DeviceIoControl<\/a>, but that sends IRP_MJ_DEVICE_CONTROL<\/a> and does not support the directory query command we want. There are also some WMI functions to send IRPs (e.g. the command line \"wmic.exe sysdriver ... call ...\"<\/code>) but this is only for IRP_MJ_SYSTEM_CONTROL<\/a>.<\/p>\n

It seems that all signs are pointing to “figure out this APC thing.” Before attempting this in managed code, let’s take a look at a possible implementation in native C++. The basic idea will be to use PPL<\/a> to define a real-ish async “get files” operation that returns a task holding a list (vector<\/code>) of path names (wstring<\/code>). I say real-ish<\/em> because the use of APCs means that we will be forced to stay on the same thread to query the information and handle the async callback. Yes, I realize that means this is hardly worth it, but we’re doing this just to show that we can!<\/p>\n

First things first: we need to install the Windows Driver Kit<\/a>. It contains the necessary headers and libraries to call NT APIs without a lot of hassle. After that, we can create a simple static library project to help isolate all the NT junk from our main application — mixing Windows.h<\/code> and NT headers doesn’t really work out well. Note that the WDK adds several Visual Studio templates for creating driver projects, but we are not<\/em> going to use one here, as we are writing user mode code that just happens to use NT APIs. Because of this, we’ll have to add a few project settings to make the compiler happy:<\/p>\n