Async exclusive lock: integration test

In the previous post, I introduced my ExclusiveLock. In the typical TDD style, I ended up with a correct single-threaded implementation. However, as soon as I added parallelism and exception tracking to the basic integration test, I began hitting exceptions indicative of state corruption:

System.InvalidOperationException: The token is not valid.
at LockSample.ExclusiveLock.Release(Token token) in writeasync\projects\LockSample\source\LockSample.Core\ExclusiveLock.cs:line 46
at LockSample.Program.d__c.MoveNext() in writeasync\projects\LockSample\source\LockSample.App\Program.cs:line 78
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LockSample.Program.d__8.MoveNext() in writeasync\projects\LockSample\source\LockSample.App\Program.cs:line 65


System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
at System.Collections.Generic.List`1.RemoveAt(Int32 index)
at LockSample.Program.d__12.MoveNext() in writeasync\projects\LockSample\source\LockSample.App\Program.cs:line 114
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LockSample.Program.d__c.MoveNext() in writeasync\projects\LockSample\source\LockSample.App\Program.cs:line 93
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LockSample.Program.d__8.MoveNext() in writeasync\projects\LockSample\source\LockSample.App\Program.cs:line 65


But first, let’s talk about the integration test itself. The basic idea is that the ExclusiveLock should help enforce exclusive access to a shared resource without corruption of itself or the resource it is protecting. The resource I chose for the test is a simple List<int>. This type is not safe for multiple writers nor does it allow writes during enumeration. The main test loop involved selecting one of four random operations on each iteration. The loop was executed in parallel by multiple actors. In the final refactoring it looked like this:

while (!token.IsCancellationRequested)
{
    ExclusiveLock.Token elt = await l.AcquireAsync();
    try
    {
        await Task.Yield();
        switch (random.Next(4))
        {
            case 0:
                await worker.EnumerateAsync();
                break;
            case 1:
                await worker.AppendAsync();
                break;
            case 2:
                await worker.RemoveAsync();
                break;
            case 3:
                await worker.RemoveAllAsync();
                break;
        }
    }
    finally
    {
        l.Release(elt);
    }
}

After getting the exceptions above, I added proper locking. The concurrency fixes were relatively simple this time. I chose the nextOwners queue as my sync root and basically locked almost all of the work in AcquireAsync and Release. (As usual, I left the SetResult code path out of the lock scope in the pending waiter case inside Release.)

After the fixes, my integration test app ran properly. The app only appended or removed from the end of the list and always added in increasing index order. The validation was thus that the list when enumerated should be a monotonically increasing sequence starting from 1. No state corruption was detected and the simple verification passed every time it ran.