{"id":5852,"date":"2023-04-04T07:00:26","date_gmt":"2023-04-04T14:00:26","guid":{"rendered":"http:\/\/writeasync.net\/?p=5852"},"modified":"2023-04-04T07:36:52","modified_gmt":"2023-04-04T14:36:52","slug":"no-time-no-problem","status":"publish","type":"post","link":"http:\/\/writeasync.net\/?p=5852","title":{"rendered":"No time, no problem"},"content":{"rendered":"<p>Unit tests are supposed to be <a href=\"https:\/\/martinfowler.com\/articles\/nonDeterminism.html\">deterministic<\/a>. If a <a href=\"https:\/\/leanylabs.com\/blog\/good-unit-tests\/\">test<\/a> is called out as being <a href=\"https:\/\/www.everydayunittesting.com\/2014\/08\/test-attribute-9-deterministic.html\">nondeterministic<\/a>, you can bet that it depends on <a href=\"https:\/\/stackoverflow.com\/questions\/44651266\/comparing-current-time-in-unit-test\">time<\/a> in <a href=\"https:\/\/stackoverflow.com\/questions\/4594652\/strategies-for-dealing-with-datetime-now-in-unit-tests\">some way<\/a>. Any app big enough and with sufficient unit tests likely has something akin to <a href=\"https:\/\/github.com\/dotnet\/aspnetcore\/blob\/main\/src\/SignalR\/common\/Shared\/ISystemClock.cs\">ISystemClock<\/a>, <a href=\"https:\/\/github.com\/microsoft\/reverse-proxy\/blob\/main\/src\/ReverseProxy\/Utilities\/ITimer.cs\">ITimer<\/a>, etc. <a href=\"https:\/\/github.com\/RobThree\/IClock\">Entire libraries exist<\/a> to remove direct dependencies on the system clock. Even here, I&#8217;ve written <a href=\"http:\/\/writeasync.net\/?p=4221\">code to cope with unpredictable periodic execution<\/a> to make testing fast and reliable. Time is a big deal!<\/p>\n<p>This is why it&#8217;s good to see .NET tackle this problem, once(?) and for all(??). Welcome, <a href=\"https:\/\/github.com\/dotnet\/runtime\/issues\/36617\">TimeProvider<\/a>! While the details are not finalized, the proposal has been floating around for a while now and is all but certain to land in some form in .NET 8. (The API has been <a href=\"https:\/\/www.youtube.com\/watch?v=KVDnUSH90qI&#038;t=0h34m27s\">reviewed and approved<\/a>!)<\/p>\n<p>There is even a plan to ship a <a href=\"https:\/\/github.com\/dotnet\/runtime\/issues\/36617#issuecomment-1488795826\">manual time provider<\/a> to allow deterministic testing. But some of us can&#8217;t wait that long. Our tests need to be fast and reliable now! As an exercise, I took the proposed API and made an interface (I don&#8217;t have the same restrictions as the .NET Runtime, so I&#8217;ll forgo the abstract class) with two implementations, FakeTimeProvider and RealTimeProvider. I then went ahead and TDD&#8217;d my way to a complete (enough) prototype. I know dozens of others must have done this already, but I tried to do so on my own without referring to other code.<\/p>\n<p>Follow my journey here, via Git commits:<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/cf9855aa8c3bc0abb855efa8120643e2f9064f43\">DateTime<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/e9917126c1982bae3314aa95efb2cfd9d834156d\">Timestamp<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/1fad3478f60935b0a26a90a71bc5166f10266caf\">TimerImmediateNonPeriodic<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/da83f418936c233e42686d39fd6cc4f14506f21a\">TimerDelayedNonPeriodic<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/29fe689fb4207ce675bb27e54fa5359efa16dc16\">TimerDelayedPeriodic<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/b59896e92b895c69fbb35f9801b846eb44049011\">TimerChangeFromPeriodicToDisabled<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/ea125cdf5eac4f81a6be9a90fee7f4109a2c6e03\">TimerChangeFromDisabledToPeriodic<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/53c25de29a5c5c6051cfac1bad8e6a92be63b49a\">Refactor FakeTimeProvider &#8211; use LinkedList<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/57af3c5f49bd09092b6c8d47b4c0ee23cc02b5e1\">TwoAlternatingTimers<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/7d7f1e195152fb13e721098d9aa152daa136cb18\">TwoTimersFireManyDuringInterval<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/beef9f636890fd4346a39da0d02498092760f560\">MultipleTimersDifferentOrder<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/8a56481b0cb7b6ce88395f745150b091d91725be\">WaitUntilCanceled<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/9a16d94ca709f0b586d08ba65015948c05c96de7\">WaitUntilDuration<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/a3fb9b1eaa23449be7d253f70eb9eddbd63bcb7b\">WaitRacesWithCanceled<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/453a93ff41209571536b647c58e3c689ff3d3a3b\">AsyncDispose<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/brian-dot-net\/writeasync2\/commit\/d411c902db7d60b821b4911113abcb9fe98278bd\">Fix test instability<\/a><\/li>\n<\/ul>\n<p>The first thing to note is that I intend to fulfill the same (single-threaded) unit tests for both the RealTimeProvider and FakeTimeProvider. Thus I use the <a href=\"http:\/\/xunitpatterns.com\/Testcase%20Superclass.html\">Testcase Superclass<\/a> (AKA Abstract Test Fixture) pattern. There is one abstract test class, TimeProviderTest with two abstract members:<\/p>\n<pre class=\"brush: csharp; title: ; notranslate\" title=\"\">\r\npublic abstract class TimeProviderTest\r\n{\r\n    protected abstract ITimeProvider Init();\r\n\r\n    protected abstract void Wait(ITimeProvider provider, TimeSpan duration);\r\n}\r\n<\/pre>\n<p>The <code>Init<\/code> method creates the specific implementation under test and the <code>Wait<\/code> method does the correct wait step for the implementation. For the real implementation, we can just use Thread.Sleep to wait, but this will not work with the fake (you could say <a href=\"https:\/\/interestingengineering.com\/science\/what-einstein-meant-by-time-is-an-illusion\">time is an illusion<\/a>). The tests must therefore be somewhat constrained and not break out of their simulation sandbox so to speak. For example, this is how we can test a non-periodic timer which fires after some delay:<\/p>\n<pre class=\"brush: csharp; title: ; notranslate\" title=\"\">\r\n    &#x5B;Fact]\r\n    public void TimerDelayedNonPeriodic()\r\n    {\r\n        var evt = new CountdownEvent(1);\r\n        ITimeProvider time = Init();\r\n\r\n        using ITimer timer = time.CreateTimer(o =&gt; evt.Signal(int.Parse(o?.ToString() ?? &quot;-1&quot;)), &quot;1&quot;, TimeSpan.FromMilliseconds(50), TimeSpan.Zero);\r\n\r\n        evt.Wait(TimeSpan.Zero).Should().BeFalse();\r\n\r\n        Wait(time, TimeSpan.FromMilliseconds(100));\r\n\r\n        evt.Wait(TimeSpan.Zero).Should().BeTrue();\r\n    }\r\n<\/pre>\n<p>The test must wait for 100 milliseconds (real or virtual) and then check that the event has been signaled. This brings us to our second note &#8212; these tests are <em>not<\/em> 100% deterministic. It is totally possible (and expected) to make a FakeTimeProvider which never varies in its behavior. But the fact is that the RealTimeProvider will never be so &#8212; the nuances of thread scheduling and background execution would always get in the way. To that end, I wanted tests that could be predictable enough to pass for both implementations yet still have a degree of freedom to account for some acceptable jitter. Take a look at this test for disabling a periodic timer:<\/p>\n<pre class=\"brush: csharp; title: ; notranslate\" title=\"\">\r\n    &#x5B;Fact]\r\n    public void TimerChangeFromPeriodicToDisabled()\r\n    {\r\n        var evt = new CountdownEvent(10);\r\n        ITimeProvider time = Init();\r\n\r\n        using ITimer timer = time.CreateTimer(o =&gt; evt.Signal(int.Parse(o?.ToString() ?? &quot;-1&quot;)), &quot;1&quot;, TimeSpan.Zero, TimeSpan.FromMilliseconds(40));\r\n\r\n        Wait(time, TimeSpan.FromMilliseconds(50));\r\n\r\n        evt.CurrentCount.Should().Be(8);\r\n\r\n        timer.Change(Timeout.InfiniteTimeSpan, Timeout.InfiniteTimeSpan).Should().BeTrue();\r\n\r\n        Wait(time, TimeSpan.FromMilliseconds(50));\r\n\r\n        evt.CurrentCount.Should().BeInRange(7, 8);\r\n\r\n        Wait(time, TimeSpan.FromMilliseconds(50));\r\n\r\n        evt.CurrentCount.Should().BeInRange(7, 8);\r\n    }\r\n<\/pre>\n<p>The timing is tight enough that we cannot guarantee we won&#8217;t see an extra event fire between the change and the next wait. So the tests compromise a bit and allow a range of values. These tests could be strengthened even further to ensure the count never goes backwards (technically a value <code>7<\/code> followed by <code>8<\/code> would be allowed here). But I called this good enough for a prototype.<\/p>\n<p>Given a TimeProvider API, calls to Task.Delay can be replaced with an extension method WaitAsync as follows:<\/p>\n<pre class=\"brush: csharp; title: ; notranslate\" title=\"\">\r\n    public static async Task WaitAsync(this ITimeProvider provider, TimeSpan duration, CancellationToken token = default)\r\n    {\r\n        \/\/ WARNING: THIS CODE HAS BUGS!\r\n        var tcs = new TaskCompletionSource();\r\n        using ITimer timer = provider.CreateTimer(o =&gt; ((TaskCompletionSource)o!).SetResult(), tcs, duration, TimeSpan.Zero);\r\n        using CancellationTokenRegistration reg = token.Register((o, t) =&gt; ((TaskCompletionSource)o!).SetCanceled(t), tcs);\r\n        await tcs.Task;\r\n    }\r\n<\/pre>\n<p>But wait, there&#8217;s a bug! Do you see it? Well, if you run this test hundreds of times you may eventually find the issue(s):<\/p>\n<pre class=\"brush: csharp; title: ; notranslate\" title=\"\">\r\n    &#x5B;Fact]\r\n    public void WaitRacesWithCanceled()\r\n    {\r\n        ITimeProvider time = Init();\r\n        using var cts = new CancellationTokenSource();\r\n\r\n        using ITimer timer = time.CreateTimer(o =&gt; ((CancellationTokenSource)o!).Cancel(), cts, TimeSpan.FromMilliseconds(70), TimeSpan.Zero);\r\n        Task task = time.WaitAsync(TimeSpan.FromMilliseconds(70), cts.Token);\r\n\r\n        task.IsCompleted.Should().BeFalse();\r\n\r\n        Wait(time, TimeSpan.FromMilliseconds(100));\r\n\r\n        task.IsFaulted.Should().BeFalse();\r\n        task.IsCompleted.Should().BeTrue();\r\n    }\r\n<\/pre>\n<p>Here is one possible exception:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\nSystem.InvalidOperationException : An attempt was made to transition a task to a final state when it had already completed.\r\n\r\nStack Trace:\u2009\r\n    TaskCompletionSource.SetResult()\r\n    &lt;&gt;c.&lt;WaitAsync&gt;b__3_0(Object o)\u2009line\u200920\r\n...\r\n<\/pre>\n<p>Yes, that&#8217;s right, we&#8217;re using the unconditional SetResult\/SetCanceled on <a href=\"https:\/\/learn.microsoft.com\/en-us\/dotnet\/api\/system.threading.tasks.taskcompletionsource\">TaskCompletionSource<\/a> instead of the <a href=\"https:\/\/learn.microsoft.com\/en-us\/dotnet\/api\/system.threading.tasks.taskcompletionsource.trysetresult\">Try&#8230; variants<\/a>.<\/p>\n<p>As a final step, I ran the tests in a loop hundreds of times using this PowerShell snippet:<\/p>\n<pre class=\"brush: powershell; title: ; notranslate\" title=\"\">\r\n# for Debug\r\ndo { dotnet test } while ($LASTEXITCODE -eq 0)\r\n# for Release\r\ndo { dotnet test -c Release } while ($LASTEXITCODE -eq 0)\r\n<\/pre>\n<p>That resulted in a few changes to loosen the assertions appropriately (e.g. using <code>Should().BeInRange(...)<\/code> instead of <code>Should().Be(...)<\/code>).<\/p>\n<p>This prototype should be sufficient to use in unit tests which operate in a single-threaded fashion. To put it another way, the FakeTimeProvider is not thread-safe. I think this is a fair restriction, since the whole point is to allow as much determinism as possible. Trying to make multi-threaded &#8220;unit&#8221; tests but insisting on a fake timer may just be defeating the purpose somewhat.<\/p>\n<p>We can&#8217;t be sure yet what shape this .NET proposal will take, so things may still shift before .NET 8 is finalized. Until then, the implementation above could be a good enough starting point. Only time will tell&#8230;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Unit tests are supposed to be deterministic. If a test is called out as being nondeterministic, you can bet that it depends on time in some way. Any app big enough and with sufficient unit tests likely has something akin&hellip; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[41,51],"tags":[],"class_list":["post-5852","post","type-post","status-publish","format-standard","hentry","category-tdd","category-testing"],"_links":{"self":[{"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts\/5852","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5852"}],"version-history":[{"count":4,"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts\/5852\/revisions"}],"predecessor-version":[{"id":5856,"href":"http:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts\/5852\/revisions\/5856"}],"wp:attachment":[{"href":"http:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5852"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5852"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5852"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}