Bill Blogs in C# -- async

Bill Blogs in C# -- async

Created: 5/3/2017 8:37:36 PM

Consider this two methods:

 

public Task DoWorkAsync()
{
    var arg1 = ComputeArg();
    var arg2 = ComputeArg();
    return AwaitableMethodAsync(arg1, arg2);
}

public async Task DoWork2Async()
{
    var arg1 = ComputeArg();
    var arg2 = ComputeArg();
    await AwaitableMethodAsync(arg1, arg2);
}

 

Do you notice the difference?

The first is a synchronous method that returns a Task. The Task may or may not have completed when the method returns. The second as an async method that returns the result of awaiting other work.

These two methods look almost the same, but the code generated by the compiler for them is very different. These two talks on InfoQ by Jon Skeet and I go into all the gory details about the differences:

In most cases, you should prefer writing the first version when possible. The method is much simpler, and is much easier to reason about. It’s a synchronous method that returns an object that represents work that may be ongoing.

The second is more complicated. It builds a state machine. It manages re-entrancy for code that should execute when the awaited task finishes. It returns. It resumes execution. It’s difficult to reason about.

You can write the first version for any task-returning method that could be a synchronous method. That’s the case when:

  • The method does not do any work after the only task-returning method is called.
  • The return of the only task returning method matches the signature of this method.

The curious case of IDisposable

Now, let’s look at a variation of the two methods above:

 

public Task DoWorkAsync()
{
    using (var service = new Service())
    {
        var arg1 = ComputeArg();
        var arg2 = ComputeArg();
        return service.AwaitableMethodAsync(arg1, arg2);
    }
}

public async Task DoWork2Async()
{
    using (var service = new Service())
    {
        var arg1 = ComputeArg();
        var arg2 = ComputeArg();
        await service.AwaitableMethodAsync(arg1, arg2);
    }
}

 

Can you spot the difference? Can you spot the bug? The introduction of a local variable the refers to an object the implements IDisposable means you must use the second version, where the compiler generates the state machine and a continuation.

I gave a hint as to the reason in the first description. The first method is synchronous. There are no continuations. The service object will be Disposed() as soon as AwaitableMethodAsync() returns. The object is disposed if the async work is completed. The object is disposed when the async work is not completd. The compiler generated finally clause will be executed before the method returns the (possibly still running) task. There is a high-probability that this idiom results in an ObjectDisposedException in some cases.

The asynchronous method generates the code so that the compiler generated finally clause executes only after the task returned from AwaitableMethodAsync() completes. The service will be Disposed only when it’s done doing all its work.

Note that my explanation of when you can write the synchronous version above is accurate: because of the compiler generated finally clause, there is code that must execute after the task completes. It’s just not easily visible in your source code.

Testing for this case

This condition can be hard to catch in automated unit tests. (In fact the error I introduced this week was not caught by unit tests in the library I was working on.) Often we write unit tests for asynchronous methods that always return synchronously, using Task.FromResult(). These tests are fine, and verify that the fast path works correctly.

You should also write tests that verify the slow path, where a Task has not completed synchronously. It doesn’t have to be measurably slow. Just sprinkle an ‘await Task.Yield()’ statement in your mock implementation and you will force the slow path.

Yes, that bug I introduced is fixed. It’s also now caught by a test.

Created: 5/18/2016 6:19:33 PM

The TL;DR; version is:

Sometimes.

The more important question is how you ensure that you generate the method call you want. Let’s start with a bit of background. Lambda expressions do not have types. However, they can be converted into any compatible delegate type. Take these two declarations as a starting point:

Action task = async () => await Task.Yield();
Func<Task> task2 = async () => await Task.Yield();

Notice that this lambda body can be assigned to either an Action or a Func<Task>. The lambda expression can represent either an async void method, or a Task returning async method.

Well, let’s suppose you call Task.Run with that lambda body:

Task.Run(async () => await Task.Yield());

(Ignore for a moment the obvious uselessness of calling Task.Run and telling it to yield.) Which of the following overloads does that Lambda resolve to:

public static Task Run(Action action);
public static Task Run(Func<Task> action);

They correspond to the two delegate declarations used in the first code sample above. This call compiles, so the compiler must find one of them to be a better method. Which one?

The compiler prefers the method call that expects a delegate that returns a Task. That’s good, because it’s the one you’d rather use. The compiler comes to this conclusion using the type inference rules about the return value of anonymous delegates. The “inferred return type” of any async anonymous function is assumed to be a Task. Knowing that the anonymous function represented by the lambda returns a Task, the overload of Task.Run() that has the Func<Task> argument is the better match.

The C# language overload rules, along with the rules for type inference for async lambda expressions ensures that the preferred overload generates a Task returning async method.

So, what does that mean for me?

Remember that async void methods are not recommended. They are fire-and-forget methods, and you can’t observe any errors that might occur in the async method. You want to avoid accidentally creating an async void lambda expression.

There are two recommendations that come from these rules in the language specification.

First, avoid using async lambdas as arguments to methods that expect Action and don’t provide an overload that expects a Func<Task>. If you do that, you’ll create an async void lambda. The compiler will happily assume that’s what you want.

Second, if you author methods that take delegates as arguments, consider whether programmers may wish to use an async lambda as that argument. If so, create an overload that uses Func<Task> as the argument in addition to Action. As a corollary, create an overload that takes Func<Task<T>> in addition to Task<T> for arguments that return a value.

The language team members worked hard on these overload rules to ensure that in most cases, the compiler prefers Task-returning anonymous functions when you write async lambdas. You have to make sure the right overloads are available.

Created: 2/24/2015 8:57:45 PM

During CodeMash, Stephen Cleary gave me a copy of his Concurrency in C# Cookbook.

It’s an essential reference for any C# developer using concurrency in their applications. These days, hopefully that means most C# developers.

The 13 chapters provide short recipes for many uses of concurrency. The recipes are short. Therefore the recipes provide minimal breadth and depth on how each different recipe works. That’s great if you have a good understanding of how these different features, and you’re looking to decide which recipe is the best one for your current challenge. However, if you are looking for a tutorial or introduction to concurrency this book will leave you with many questions.

Stephen’s material is clear, concise, and will help you follow the proper guidance in a variety of situations. It will give you more options for concurrent programming, and you will be better at using them. If you know some of the concurrent techniques available to modern .NET developers, you’ll quickly catch on to the style and you’ll be exposed to recipes you may not know. That will make you a better developer.

This book is not for developers that have no experience with concurrent programming. Stephen assumes you know the basics. His explanations assume a background in the tools used.

This book has earned a handy place on my shelf. In particular, the chapter on Data Flows helps me remember to use this handy library more often. I believe I’ll reach the point where I reference this book whenever I’m looking at how to build a concurrency related library or program.

Created: 2/19/2014 7:19:40 PM

I spoke last night at the new Chicago .NET Users group.  It’s a great, and growing group. I had a great time. (confession: I love Chicago, so I always look for reasons to go there.) Aaron has a great group started.  They are a very knowledgeable bunch. 

I spoke on async coding practices in C#, specifically the Task Asynchronous Programming (TAP) model. That can be a complicated topic, and everyone followed the discussion quite well.

Here are several of the resources I mentioned during the talk:

The Github repository for my samples. Each branch represents on step of the demo. The tip of the master branch is the finished sample.

Pearson recently published my LiveLesson series on Async programming fundamentals.  That goes into more depth on the async practices than I had time to cover last night.

Jon Skeet and I did a lengthy session at CodeMash 2012 where we discussed both how to use async, and what the compiler does to your code when you create async methods.  InfoQ has those sessions online.

I had a few great questions about using async methods at the server. Due to time, I gave brief answers.  For more depth, I recommended Stephen Cleary’s talk from CodeMash 2014.

Thanks again for a great group, and the folks at DevMynd for hosting. I had a great time, and I’m happy to come back again.

Created: 12/26/2013 3:11:44 PM

Is one of your goals to improve your knowledge and skill with C#’s async features? The async Fundamentals Live Lessons I created have just been released by Pearson.

It’s available on InformIT, and through a Safari subscription. I highly recommend the subscription. There’s a lot of content that will improve your skills in many areas.

I really like the format for this lesson. I first used it at an async precompiler during CodeMash 2013.  These lessons take you on a guided tour through many common mistakes developers make when using async and await in C#. In each lesson, you’ll learn ways to improve on these common mistakes.

For those developers that are new to C# async, I start by explaining the syntax and the features provided by async and await.

After that, you’ll be introduced to the Code Smell Pig:  Async Pig Sitting 01_Color

This character needs to learn better practices for Async programming in C#. He tries hard, but there are many subtleties to learn about async programming that he just doesn’t know. In the beginning, his async code is just a synchronous implementation with some async APIs used. He’s simply sprinkled some asynchronous APIs into fundamentally synchronous code. The result is not a good implementation: asyncomatic

Throughout the lessons, I’ll show you what he’s done wrong, and how to correct it. Throughout the lessons, you’ll learn what you should be doing instead of the common mistakes our little friend makes. I’ll discuss the problems associated with sync over async, async over sync. You’ll learn the dangers of async void methods. You’ll see how to properly handle and report errors in asynchronous methods. You’ll learn how to control synchronization contexts, and when you should use the default behavior, and when you should explicitly control the continuation contexts.

If some of those terms are unfamiliar, watch the live lesson. I define all of them as part of the lessons where those techniques are used.

Finally, in the last lesson, I discuss some async specific issues for unit tests. I show how to construct unit tests that control the order of task completion, and whether any task completes successfully or with errors.

In the end, you’ll have some squeaky clean async code:

cleanasync

 

I hope you check it out. It will help you learn more about async programming. I hope the lighthearted look at common mistakes provides a good way to learn and understand the subtle issues associated with async programming.

I think the language teams have created a great set of features for async programming. These techniques will help you use them wisely.

Special thanks to John Lucas for the artwork and the design of the code smell pig.

Created: 10/31/2013 12:11:02 PM

I finally had enough of a break that I can post the slides from my two talks at the Boston Code Camp.

My first talk was Practical LINQ, N tips and tricks for some number N. You can download the presentation materials here: PracticalLINQpptx.pdf

The demos are on GitHub: PracticalLINQ  Each of the labeled commits mirrors the changes I made during the presentation.

My second presentation was The Task Asynchronous Programming Model. The presentation materials are here: TAPExplained.pdf

Those demos are also available on GitHub: AsyncVoid. Again, the labeled commits match the changes I made during the presentation.

If you have any questions or comments, please make them below, and I’ll update the repositories, or write a new blog post about what’s changed.

Created: 10/14/2013 1:15:33 PM

This coming Saturday, October 19th, I’ll be at Boston Code Camp 20. I’ve got two talks:  Practical LINQ in the morning, and The Task Asynchronous Programming Model in the late afternoon.

This will be the first time I’ve spoken at the Boston Code Camp, and I’m looking forward to meeting developers in the New England community. I’m spending more time on the East Coast now and I’d like to become more involved with that community.

Between my sessions, I’ll be splitting time between web sessions and Windows 8 sessions.  There’s a great set of content on both topics, and it will be a good chance for me to pick up some knowledge on both of these areas.

I hope to see you there.

Created: 8/1/2013 5:32:29 PM
That Conference

ThatConference is just around the corner. There are two parts of the event where I'll have a big presence.

First, The ThatConference GiveCamp. I'm one of the project leads, for the volunteer checkin system. Visit the link above and you'll get a great overview on what we are building. The GiveCamp / Hackathon takes place on Saturday and Sunday. Come join us, and build some code that can have a real impact on people. We can make humanitarian response more efficient, and literally save lives. The other project being developed at the ThatConference GiveCamp is the Mobile Based Emergency Training project that Humanitarian Toolbox started at DevTeach. Phil Japikse will be at ThatConference and leading that project again.

Also, a big thanks to Telerik for providing Humanitarian Toolbox with copies of their tools for the projects we are creating.

I'm also excited to be speaking there this year. My session will discuss the reasons to minimize the use of async void methods. Come join me on Monday afternoon and get your async on.

Finally, I'm going to be doing more work for Humanitarian Toolbox between other sessions. The main focus of Humanitarian Toolbox is to own the projects over time. We want them ready to deploy when disaster strikes. I'll be hanging out, adding more code to the crisis checkin tool. Please join me. I'll post on twitter where I'm coding.

If you're coming to ThatConference, join us.

Created: 3/5/2013 4:49:13 PM

Hat tip to Stephen Toub for discussing this with me and helping to describe the solution.

At my CodeMash precompiler, I mentioned how the C# compiler ensures that methods marked with the 'async' keyword that contain 'await' expressions never throw synchronous exceptions. Instead, those methods will return a Task (or Task<T>) that will be placed in the faulted state if the method throws an exception. The compiler does the work to add the appropriate try/catch clauses to your methods, and translates any exceptions thrown during your method's execution into returned faulted task.

Jon Skeet was concerned by this strategy. He felt that API designers would prefer throwing synchronous exceptions for obvious programmer errors (things like ArgumentNullException and so on). I mentioned that the language specification defines that async methods will returned faulted tasks; they will not throw synchronous exceptions.

Of course, there is a way around this. You have to separate your public async APIs into two pieces: A public synchronous API that does parameter validation and state validation. This synchronous method then calls an internal async method that does the asynchronous work.

As an example, consider this (rather contrived) async method:

 

      
        
          
            
public
          
        
         async Task<string> FizzBuzzAsync(int val)
{
    if (val <= 0)
        throw new ArgumentException("We can't fizzbuzz negative numbers, or 0");

    await Task.Delay(250);
    var rVal = string.Empty;
    if (val % 3 == 0)
        rVal += "Fizz";
    if (val % 5 == 0)
        rVal += "Buzz";
    if (string.IsNullOrWhiteSpace(rVal))
        rVal = val.ToString();
    return rVal;
}

 

Calling this method with a negative number is a programming error. We'd like to have that condition throw a synchronous exception. We can achieve this by separating the method into two parts. The first part is a synchronous method that performs the parameter validation and state validation. The second part is an internal method that performs the asynchronous work.  The first method will throw exceptions synchronously. The second will report errors using a faulted task.

 

      
        
          
            public
          
        
         Task<string> FizzBuzzAsync(int val)
{
    if (val <= 0)
        throw new ArgumentException("We can't fizzbuzz negative numbers, or 0");
 
    return FizzBuzzAsyncImpl(val);
}
 
private static async Task<string> FizzBuzzAsyncImpl(int val)
{
    await Task.Delay(250);
    var rVal = string.Empty;
    if (val % 3 == 0)
        rVal += "Fizz";
    if (val % 5 == 0)
        rVal += "Buzz";
    if (string.IsNullOrWhiteSpace(rVal))
        rVal = val.ToString();
    return rVal;
}

That ensures that your public async methods conform to both important rules.

First, TAP (Task Asynchronous Pattern) methods do not allow synchronous runtime exceptions. They must return errors by returning a faulted task.

Second, to make it easier for callers to detect and correct programming errors, simple programming errors will throw synchronous exceptions.

Created: 1/30/2013 10:16:06 PM
UPDATE:  Jake Ginnivan pointed out an inefficiency in my code below.  I was awaiting the async method under test twice.   As I mentioned a couple blog posts ago, I want to write tests that provide reasonable examples for users of my libraries. I expect developers to copy my tests as example code and modify it for production code. That can be tough for async tests when the async method should throw an exception. Well, as I was busy laying the background, Phil Haack leaped ahead to the punchline in his recent blog post. My own solution was a bit different: public static async Task ThrowsExceptionAsync<TException>(Func<Task> func)     where TException : Exception {     try     {         await func().ConfigureAwait(false); } catch (TException) {         return; }     Assert.Fail("Delegate did not throw " + typeof(TException).Name); } And you'd use it like this: await ThrowsExceptionAsync<ArgumentException>(() => worker.GetMessage("   ")); Note that the worker.GetMessage() does not need to be awaited here, because it is awaited in the ThrowsExceptionAsync method. Most of the differences between my version and Phil’s are due to the fact that I didn't use any xUnit specific features. However, one part I will discuss in a bit more detail. Note the ConfigureAwait(false) extension method when calling the async method under test: That instructs the framework to avoid the extra work of marshaling the continuation (the code after the await) onto the calling thread. This is a good practice for library code that does not need to have its continuations execute on the UI thread. I'll discuss this in more detail in a future blog post. There's also a few big picture concepts to get across on this post. The first key practice is that that async methods should return Task, not be void methods. If your async methods are void returning, you can't determine if your async methods worked correctly, or failed. By returning Task, you can query that Task (or let the compiler generate that code) to determine if it succeeded, failed, or was cancelled. The second key is that you should await for the result of any async method that returns a Task or Task<T> (even in a lambda). That causes the compiler to generate all the code to check that returned Task and ensure that it completed successfully. If it completed in a faulted state, the compiler generates code to throw the exception during the continuation. Finally, don't Wait() for Tasks to finish. await tasks. The compiler generates much better code to handle continuations when you await. And, that code is asyncrhonous, not synchronous.
Created: 1/21/2013 7:38:43 PM
Last week I promised that I'd write a blog post on using Assert.ThrowsException() to test async methods. Before I get to that, let's go over some of the other issues that come up with testing async methods. First, let's look at a test to verify that an async method works correctly:   public async Task<string> GetMessage(string user) {      return await Task.FromResult(string.Format("Hello {0}", user)); }   One test that works could be written like this:   [TestMethod] public void SimplePathTest() {     var worker = new Worker();     var answer = worker.GetMessage("unit tests").Result;     Assert.AreEqual("Hello, unit tests", answer); }   I don’t like writing tests like that.  Calling “.Result” on a task is a code smell.  The test method blocks until the result is available. That’s probably OK in a test method (more on that in future blog posts), but I’m still concerned. Developers often copy code from unit tests into production code. (I’ve done it myself when I’m learning how a library works.)  For that reason, I want my unit tests for follow the practices I would use in production code. That means I want to ‘await’ the result, not block for it. You might try reworking the test like this: [TestMethod] public async void SimplePathTest() {      var worker = new Worker();      var answer = await worker.GetMessage("unit tests");      Assert.AreEqual("Hello, unit tests", answer); }   You’ll see one of two results from this change:
  • You may see that test disappear from your list of unit tests.
  • You may run the test and find that results are not reported for any code that follows the first ‘await’.
I actually prefer the first, because you’ll quickly figure out that there’s a problem. The problem is that you have an async method that has the void return type. There’s no Task returned for the test runner to examine for any exceptions that were thrown. If your test fails and the Assert expressions throw exceptions, no code is there to receive the exception and report the test failure. If you are using MSTest, or XUnit, you can fix this problem by declaring your async tests with the Task return type: [TestMethod] public async Task SimplePathTest() { var worker = new Worker();      var answer = await worker.GetMessage("unit tests");      Assert.AreEqual("Hello, unit tests", answer); } That’s it.  Now, your async tests are using the idioms that you’d want people to use in production code. Above I mentioned that this idiom is supported for XUnit and MSTest. NUnit 2.6.2 adds this support as well.  I haven’t had the chance to work with the latest version yet. I’ll write more blog entries about NUnit support for async tests as I work with them. Next time, I’ll finally answer that comment about how to test async methods that throw exceptions with a ThrowsException<> style expression.
Created: 1/14/2013 4:45:43 PM
The Windows Store version of the Microsoft Test Framework does not include an ExpectedExceptionAttribute class. I’ve already received questions on how people should go about creating tests where the code under test should generate exceptions. First, a note on the seriousness (or lack of) for this change:  The existing .NET libraries for testing do not display this problem.  Your existing unit test libraries are fine. But moving forward, how are you supposed to test failures? Instead of adding the ExpectedExceptionAttribute, you should use the new method: public static void ThrowsException<T>(Action code) where T : Exception There are reasons for this change. ExpectedException is a course-grained check that can cause tests to pass that really should fail. Consider this test:   [TestMethod, ExpectedException(ArgumentException)] public void ExceptionPath() { setupTestEnvironment(); var underTest = new Worker(); setupTestObject(underTest); var systemParms = GenerateCollaborator(); underTest.DoSomeWork(systemParms); }   The ExcpectedException attribute means that this test passes if *any* code in this test method throws the expected exception. It may not where you expect. It may be that some of your setup fails, and the particulars of the failure mean that the expected exception is what's thrown. That could mask problems in your production code.  If the setup method setupTestObject() throws an ArgumentException, this test passes, even though it shouldn’t. Because of the nature of this issue, it can crop up anytime. You could be updating test code and you could accidentally introduce this error. It would masked because the test passed when you started, and the test passed when you finished. Assert.ThrowsException fixes this issue.  Here’s the new version of the same test: [TestMethod] public void ExceptionPath() { setupTestEnvironment(); var underTest = new Worker(); setupTestObject(underTest); var systemParms = GenerateCollaborator(); underTest.DoSomeWork(systemParms); Assert.ThrowsException<ArgumentException>(() => underTest.DoSomeWork(default(string))); } This different version of the test passes if and only if the DoSomeWork() method throws the ArgumentException. If the ExceptionPath() test method throws an exception anywhere else, this test fails. As it should. If you update tests as you add features, or change defined behavior, I recommend changing your failure path tests to make use of this technique. I wouldn't retrofit every test you have, though. That feels like a whole lot of busy work, with only small benefit. Next, we'll look at how to write tests for async methods that throw exceptions. There are many nuances there that can mask issues in your tests, and in your production code.
Created: 11/10/2012 2:51:24 PM

I’m finishing up my calendar and planning for next week. It’s a big week for developers here in Southeast Michigan. There are three big events you should attend:

Wed Nov 14th:  Windows 8 / VS 2012 Launch and InstallFest

This coming Wednesday, our local .NET developer group, AADND, is hosting an install fest / launch event for Windows 8 and Visual Studio 2012.  You can come and install Windows 8 and Visual Studio 2012 on your box. A number of people that have already worked with Windows 8 and VS2012 will be on hand to help, and to provide guidance. I’ll be there to help and to discuss the Open Source environment for Windows developers.

Thursday Nov 15th:  Doing Privacy Right. A workshop for app developers.

SRT Solutions is teaming up with the Association for Competitive Technology and the Mobile Technology Association of Michigan to raise awareness of privacy issues as it relates to mobile and app development.  We’ve got development experts, FTC officials, and legal and policy experts to help navigate what can be a complicated landscape. It is important information for developers to have at their disposal. I’ll be saying some opening remarks, and helping with Q and A on Windows 8 development.

Saturday Nov 17th: 1DevDayDetroit.

And the week ends with a full day of development goodness at Cobo Hall in Detroit. Dave McKinnon and Dave Giard have put together a strong program (well, I’m speaking too), and it promises to be a great day.  I am speaking on async / await features in C# 5 and what that means for .NET developers.

The first two events are free, and a great way to learn and grow your development skills.  1DevDayDetroit is $99.00. Check them out. All three events do require pre-registration. 

Created: 10/18/2012 12:26:09 AM

I’m thrilled to be enjoying a new experience at CodeMash 2013: I’ll be hosting a precompiler workshop on C# 5.0 async programming techniques.

I’ve spoken at every previous CodeMash, but this is the first time I’m running a half day workshop. I like the challenge of preparing a half day of async content, and taking participants on a larger journey. That’s what this is about: My goal, if you give me four hours, is to teach you to see async, await, Task<T> and related types the same way you see for, if, and foreach: tools you use every day. There’s quite a bit to cover, and I’m looking forward to every minute of it.  I hope you’ll join me.

You can see all the sessions here:  http://www.codemash.org/sessions They are a menus of awesome that’s better than a bacon bar.

Created: 7/18/2012 8:41:23 PM

I was honored to be invited to the speak at the Portland, ME user group last week. It was a great group of developers, and we had a lively discussion about the new async and await features in C# 5.  We went through some of the current thinking on how to leverage async in your applications.

And, I explained how async and await relate to Dr. Who.

Slides can be downloaded here.

Demos can be downloaded here.

Current Projects

I create content for .NET Core. My work appears in the .NET Core documentation site. I'm primarily responsible for the section that will help you learn C#.

All of these projects are Open Source (using the Creative Commons license for content, and the MIT license for code). If you would like to contribute, visit our GitHub Repository. Or, if you have questions, comments, or ideas for improvement, please create an issue for us.

I'm also the president of Humanitarian Toolbox. We build Open Source software that supports Humanitarian Disaster Relief efforts. We'd appreciate any help you can give to our projects. Look at our GitHub home page to see a list of our current projects. See what interests you, and dive in.

Or, if you have a group of volunteers, talk to us about hosting a codeathon event.