In a contest between async proper, Task.Run, and dedicated threads, who will win? Let’s gather some data points for a specific scenario and find out!
Today’s benchmark will make use of a trivial single process WCF named pipes application using .NET 4.5. The complete source code is as follows:
namespace AsyncPerf
{
using System;
using System.Collections.Generic;
using System.Reflection;
using System.ServiceModel;
using System.ServiceModel.Description;
using System.Threading;
using System.Threading.Tasks;
internal sealed class Program
{
private static readonly Uri BaseAddress = new Uri("net.pipe://localhost/TestService");
private static void Main(string[] args)
{
if (args.Length != 4)
{
Console.WriteLine("Provide the task count, message count, operation delay (ms), and method name.");
return;
}
int senderCount = int.Parse(args[0]);
int messageCount = int.Parse(args[1]);
TimeSpan delay = TimeSpan.FromMilliseconds(int.Parse(args[2]));
MethodInfo method = typeof(AsyncMethods).GetMethod(args[3]);
Func<int, Task> func = (Func<int, Task>)method.CreateDelegate(typeof(Func<int, Task>));
ServiceHost host = new ServiceHost(new TestService(delay), BaseAddress);
ServiceThrottlingBehavior throttlingBehavior = new ServiceThrottlingBehavior
{
MaxConcurrentCalls = int.MaxValue,
MaxConcurrentInstances = int.MaxValue,
MaxConcurrentSessions = int.MaxValue,
};
host.Description.Behaviors.Add(throttlingBehavior);
host.AddServiceEndpoint(typeof(ITestService), GetBinding(), string.Empty);
host.Open();
List<Task> tasks = new List<Task>();
for (int i = 0; i < senderCount; ++i)
{
tasks.Add(func(messageCount));
}
Task.WaitAll(tasks.ToArray());
}
private static NetNamedPipeBinding GetBinding()
{
return new NetNamedPipeBinding(NetNamedPipeSecurityMode.None);
}
[ServiceContract(Name = "ITestService")]
private interface ITestServiceLegacyAsync
{
[OperationContract(AsyncPattern = true)]
IAsyncResult BeginIncrement(int input, AsyncCallback callback, object state);
int EndIncrement(IAsyncResult result);
}
[ServiceContract(Name = "ITestService")]
private interface ITestServiceTaskAsync
{
[OperationContract(AsyncPattern = true)]
Task<int> IncrementAsync(int input);
}
[ServiceContract]
private interface ITestService
{
[OperationContract]
int Increment(int input);
}
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]
private sealed class TestService : ITestService
{
private readonly TimeSpan delay;
public TestService(TimeSpan delay)
{
this.delay = delay;
}
public int Increment(int input)
{
Thread.Sleep(delay);
return input + 1;
}
}
private static class AsyncMethods
{
public static async Task SendTaskRunAsync(int count)
{
ITestService proxy = ChannelFactory<ITestService>.CreateChannel(GetBinding(), new EndpointAddress(BaseAddress));
int i = 0;
while (i < count)
{
i = await Task.Run(() => proxy.Increment(i));
}
}
public static Task SendThreadAsync(int count)
{
Action action = delegate
{
ITestService proxy = ChannelFactory<ITestService>.CreateChannel(GetBinding(), new EndpointAddress(BaseAddress));
int i = 0;
while (i < count)
{
i = proxy.Increment(i);
}
};
return Task.Factory.StartNew(action, TaskCreationOptions.LongRunning);
}
public static async Task SendLegacyTask2Async(int count)
{
ITestServiceLegacyAsync proxy = ChannelFactory<ITestServiceLegacyAsync>.CreateChannel(GetBinding(), new EndpointAddress(BaseAddress));
int i = 0;
while (i < count)
{
i = await Task.Factory.FromAsync(
(a, c, s) => ((ITestServiceLegacyAsync)s).BeginIncrement(a, c, s),
r => ((ITestServiceLegacyAsync)r.AsyncState).EndIncrement(r),
i,
proxy);
}
}
public static async Task SendLegacyTaskAsync(int count)
{
ITestServiceLegacyAsync proxy = ChannelFactory<ITestServiceLegacyAsync>.CreateChannel(GetBinding(), new EndpointAddress(BaseAddress));
int i = 0;
while (i < count)
{
i = await Task.Factory.FromAsync(
(c, s) => proxy.BeginIncrement(i, c, s),
r => proxy.EndIncrement(r),
null);
}
}
public static async Task SendRealTaskAsync(int count)
{
ITestServiceTaskAsync proxy = ChannelFactory<ITestServiceTaskAsync>.CreateChannel(GetBinding(), new EndpointAddress(BaseAddress));
int i = 0;
while (i < count)
{
i = await proxy.IncrementAsync(i);
}
}
}
}
}
(Note the service throttling settings, which will prevent us from hitting any artificial concurrency limits.)
We have five different algorithms in the runnings:
- SendRealTaskAsync: Uses a Task-based async contract to send messages.
- SendLegacyTaskAsync: Uses a legacy asynchronous programming model contract wrapped on the outside by Task.Factory.FromAsync.
- SendLegacyTask2Async: Same as SendLegacyTaskAsync except using a different overload of Task.Factory.FromAsync which avoids the use of a generated closure.
- SendThreadAsync: Uses a synchronous contract and offloads the entire send loop to a new thread.
- SendTaskRunAsync: Uses a synchronous contract wrapped on the outside with Task.Run.
All measurements were taken on a Surface Pro 3 i7 model (quad core, 1.7-3.3 GHz, 8 GB RAM).
Contest 1: 1000 concurrent senders, no server-side delay
In the first contest, we will compare performance without any server delay at all, i.e. the message responses arrive as fast as possible.
SendLegacyTask2Async
SendLegacyTaskAsync
SendRealTaskAsync
SendTaskRunAsync
SendThreadAsync
Clearly, dedicated threads are bad for such a high degree of concurrency. But what is this? Task.Run seems to outperform the genuine Task-based async contract. While this may at first be a shock, consider the fact that there is no server delay. Hence, the completion time of an operation here is going to be nearly as fast as a local method call. The amount of thread switching and async overhead does not pay off in this case.
Contest 2: 1000 concurrent senders, 100 ms server-side delay
In the next contest, we introduce 100 ms of delay on the message responses.
SendLegacyTask2Async
SendLegacyTaskAsync
SendRealTaskAsync
SendTaskRunAsync
SendThreadAsync
Dedicated threads lose, as expected. We didn’t even get to the point where all threads were even active given the various delays in creation of new thread pool threads and the total thread count limit. Real task-based async is on par with SendLegacyTask2Async and a bit better than SendLegacyTaskAsync as expected. For a more proper comparison, it might have been prudent to also implement server-side async (this is partially why the threads seem to be growing to such a high amount) — but the general trend is relatively clear nonetheless.
Conclusion? When the operations being performed are basically CPU-bound, the overhead of async might give you pause. However, in a more realistic situation where most I/O time is spent waiting around, tasks do the trick quite well.