{"id":2621,"date":"2014-03-31T13:00:45","date_gmt":"2014-03-31T13:00:45","guid":{"rendered":"http:\/\/writeasync.net\/?p=2621"},"modified":"2014-03-30T08:44:43","modified_gmt":"2014-03-30T08:44:43","slug":"from-sync-to-async-file-io","status":"publish","type":"post","link":"http:\/\/writeasync.net\/?p=2621","title":{"rendered":"From sync to async: file I\/O"},"content":{"rendered":"

In the previous post<\/a>, I described the basics of switching to async for network calls. Today I will discuss the same for file operations.<\/p>\n

Reading and writing<\/h2>\n

To read and write files asynchronously, look no further than System.IO.FileStream<\/code><\/a>. You must<\/em> use the constructor with the useAsync<\/code> parameter<\/a> set to true<\/code>. Failing to do so will cause all async operations to be inefficiently serviced via the thread pool instead of using overlapped I\/O<\/a>. Here is a quick example of how to read the last N<\/em> bytes of one file and write them to a new file:<\/p>\n

\r\nprivate static async Task ReadAndWriteLastBytesAsync(string inputFile, string outputFile, int byteCount)\r\n{\r\n    \/\/ This is the default buffer size.\r\n    \/\/ (see <http:\/\/msdn.microsoft.com\/en-us\/library\/47ek66wy(v=vs.110).aspx>)\r\n    const int BufferSize = 4096;\r\n    using (FileStream inputStream = new FileStream(inputFile, FileMode.Open, FileAccess.Read, FileShare.Read, BufferSize, true))\r\n    using (FileStream outputStream = new FileStream(outputFile, FileMode.Create, FileAccess.Write, FileShare.Read, BufferSize, true))\r\n    {\r\n        byte[] bytes = new byte[byteCount];\r\n        inputStream.Seek(-bytes.Length, SeekOrigin.End);\r\n        int actualCount = await inputStream.ReadAsync(bytes, 0, bytes.Length);\r\n        await outputStream.WriteAsync(bytes, 0, actualCount);\r\n    }\r\n}\r\n<\/pre>\n
Unfortunately there aren’t many convenience methods for dealing with async file I\/O, but it’s easy enough to write your own. Here’s an example showing a possible asynchronous counterpart for File.ReadAllLines<\/code><\/a>:<\/p>\n
\r\nprivate static async Task<string[]> ReadAllLinesAsync(string path, Encoding encoding)\r\n{\r\n    const int BufferSize = 4096;\r\n    List<string> lines = new List<string>();\r\n    using (StreamReader reader = new StreamReader(new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, BufferSize, true), encoding))\r\n    {\r\n        while (!reader.EndOfStream)\r\n        {\r\n            string line = await reader.ReadLineAsync();\r\n            lines.Add(line);\r\n        }\r\n    }\r\n\r\n    return lines.ToArray();\r\n}\r\n<\/pre>\n
Enumerating files\/directories<\/h2>\n
Now for some bad news… there are no asynchronous APIs for traversing files and directories in .NET. It is theoretically<\/em> possible to make an asynchronous request for a file\/directory listing in Windows (e.g. see ZwQueryDirectoryFile<\/code><\/a>), but good luck trying to do it, let alone from managed code.<\/p>\n
But not to worry — as it turns out, Directory.EnumerateFiles<\/code><\/a> can do a good enough job here. It has some limited degree of deferred execution<\/a> and with a strategically placed Task.Yield<\/code><\/a> will work fine for most circumstances. Here is a sample method which walks a directory tree and executes an async method for each file:<\/p>\n
\r\nprivate static async Task ForEachFileAsync(string path, string searchPattern, SearchOption searchOption, Func<string, Task> doAsync)\r\n{\r\n    \/\/ Avoid blocking the caller for the initial enumerate call.\r\n    await Task.Yield();\r\n\r\n    foreach (string file in Directory.EnumerateFiles(path, searchPattern, searchOption))\r\n    {\r\n        await doAsync(file);\r\n    }\r\n}\r\n<\/pre>\n
And here is a sample use case which recursively searches for all text files in a directory tree and computes the Adler-32 checksum<\/a> for each file:<\/p>\n
\r\nprivate static Task SampleAsync()\r\n{\r\n    return ForEachFileAsync(\r\n        @"Some\\Path\\Here",\r\n        "*.txt",\r\n        SearchOption.AllDirectories,\r\n        f => ComputeAdler32Async(f));\r\n}\r\n\r\nprivate static async Task ComputeAdler32Async(string file)\r\n{\r\n    const int BufferSize = 4096;\r\n    const int ModAdler = 65521;\r\n    int a = 1;\r\n    int b = 0;\r\n    using (FileStream stream = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.Read, BufferSize, true))\r\n    {\r\n        byte[] buffer = new byte[BufferSize];\r\n        int bytesRead;\r\n        do\r\n        {\r\n            bytesRead = await stream.ReadAsync(buffer, 0, BufferSize);\r\n            for (int i = 0; i < bytesRead; ++i)\r\n            {\r\n                a = (a + buffer[i]) % ModAdler;\r\n                b = (b + a) % ModAdler;\r\n            }\r\n        }\r\n        while (bytesRead > 0);\r\n    }\r\n\r\n    int adler32 = (b << 16) | a;\r\n    Console.WriteLine("{0}: 0x{1:X}", file, adler32);\r\n}\r\n<\/pre>\n
Deleting files<\/h2>\n
More bad news… there is no asynchronous API to delete a file. However, this might do in a pinch — a method to truncate and flush a file asynchronously, after which it is deleted:<\/p>\n
\r\nprivate static async Task DeleteFileAsync(string file)\r\n{\r\n    using (FileStream stream = new FileStream(file, FileMode.Truncate, FileAccess.Write, FileShare.Delete, 4096, true))\r\n    {\r\n        await stream.FlushAsync();\r\n        File.Delete(file);\r\n    }\r\n}\r\n<\/pre>\n
Creating\/deleting directories<\/h2>\n
Alas, there are also no asynchronous APIs for creation and deletion of directories. You might consider pushing these operations after an already async step so that at worst you’re only blocking a thread pool thread. For example, consider this code which recursively deletes an entire directory tree; it takes advantage of the fact that the file truncate\/flush operations are async and does the delete as a final step:<\/p>\n
\r\npublic static class Recursive\r\n{\r\n    public static async Task DeleteAsync(string path)\r\n    {\r\n        \/\/ Don't block calling thread\r\n        await Task.Yield();\r\n        await DeleteInnerAsync(path);\r\n    }\r\n\r\n    private static async Task DeleteInnerAsync(string path)\r\n    {\r\n        foreach (string file in Directory.EnumerateFiles(path, "*", SearchOption.TopDirectoryOnly))\r\n        {\r\n            await DeleteFileAsync(file);\r\n        }\r\n\r\n        foreach (string directory in Directory.EnumerateDirectories(path, "*", SearchOption.TopDirectoryOnly))\r\n        {\r\n            await DeleteInnerAsync(directory);\r\n        }\r\n\r\n        Directory.Delete(path);\r\n    }\r\n\r\n    private static async Task DeleteFileAsync(string file)\r\n    {\r\n        using (FileStream stream = new FileStream(file, FileMode.Truncate, FileAccess.Write, FileShare.Delete, 4096, true))\r\n        {\r\n            await stream.FlushAsync();\r\n            File.Delete(file);\r\n        }\r\n    }\r\n}\r\n<\/pre>\n
Despite the limitation of relatively few truly asynchronous APIs, “good enough” async file I\/O is possible and relatively painless in .NET.<\/p>\n","protected":false},"excerpt":{"rendered":"
In the previous post, I described the basics of switching to async for network calls. Today I will discuss the same for file operations. Reading and writing To read and write files asynchronously, look no further than System.IO.FileStream. You must… <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21,91],"tags":[],"class_list":["post-2621","post","type-post","status-publish","format-standard","hentry","category-async","category-design"],"_links":{"self":[{"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts\/2621","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2621"}],"version-history":[{"count":0,"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts\/2621\/revisions"}],"wp:attachment":[{"href":"http:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2621"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2621"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2621"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}