I’ve been writing a new project using a microservices-based architecture; and during the development of the latest service; I realized that it needs to communicate with no less than 7 other microservices over HTTP (it also listens on two particular queues and publishes to two message queues).
During one iteration, it could potentially talk to all seven microservices depending on the logic path taken. As such, there is a lot of time spent talking over the network, and in a normal synchronous .NET Core application, there’s a lot of time spent blocking while this communication is happening. To resolve the blocking slowing down its responsiveness to the rest of the system, I ported it from being a synchronous to an asynchronous Microservice. This was a feat and took the better part of a day to do (for a Microservice, a day feels like a really long time). Along the way, I ran into several places where I have questions, but no answers (or at least no firm understanding as to whether or not my answer is right), and so I’ll post those questions here. You’ll find no answers here, only questions:
How far do I need to go down the asnyc rabbit hole?
If you’re writing a .NET Core Microservice, chances are you’re doing JSON serialization/deserialization. Since JSON.NET doesn’t have Async, our options are to leave it synchronous in any call, or use Task.Run() to make it async:
sync:
{
"owner": {
"reputation": 41,
"user_id": 6223870,
"user_type": "registered",
"profile_image":
"https://www.gravatar.com/avatar/e1d1beda042e5faa2177c415da848307?s=128&d=identicon&r=PG&f=1",
"display_name": "Harshal Zope",
"link": "http://stackoverflow.com/users/6223870/harshal-zope"
},
"is_accepted": false,
"score": 0,
"last_activity_date": 1493298802,
"creation_date": 1493298802,
"answer_id": 43658947,
"question_id": 39165805
}
var owner = JsonConvert.DeserializeObject(jsonstring);
async:
await Task.Run(() => JsonConvert.DeserializeObject(jsonstring));
Since Microsoft recommends CPU-bound work be put in a task, what is the point where that should occur? Are small serializations/deserializations like the above CPU-bound? Are big ones? what is that threshold? How do you test for it?
If you don’t put code inside an async method in a Task.Run; what happens? If it depends on previous code; it’ll run in order; but what if it doesn’t? Does it run immediately? Besides the nano-seconds of blocking, is there any other reason to care whether everything inside an async method is awaitable?
How do you deal with synchronous libraries in asynchronous code?
RabbitMQ’s .NET client famously does not support async/await; (as an aside, have we not seen pressure to convert to async because no one is using it or because no one is using RabbitMQ in .NET?) and you’ll even get errors in some places if you try to make the code async, and they put it in their user-guide:
Symptoms of incorrect serialisation of IModel operations include, but are not limited to,
- invalid frame sequences being sent on the wire (which occurs, for example, if more than one BasicPublish operation is run simultaneously), and/or
- NotSupportedExceptions being thrown from a method in class RpcContinuationQueue complaining about “Pipelining of requests forbidden” (which occurs in situations where more than one AMQP RPC, such as ExchangeDeclare, is run simultaneously).
And Stack Overflow’s advice isn’t helpful; the answer to “How do I mix async and non-async code?” is: “Don’t do that“. In another Stack Overflow post, the answer is, “Yea, you can do it with this code.”
What’s the answer? Don’t do it? Keep your entire service synchronous because the message queueing system you use doesn’t support async? Or do you convert but implement this work-around for the pieces of code that need it?
Why is it after 5 years, the adoption of async seems to be negligible? Unlike some languages, where you have no choice but to embrace async; C# as a culture still seems to treat async as a second-class citizen; and the vast majority of blogposts I’ve read on the subject go into very topical and contrived uses; without digging deeper into the real pitfalls you’ll hit when you use async in an application.
SynchronizationContext: When do I need to care about it? When do I not? Do I only care about it when it’s being used inside an object with mutable state? Do I care about it if I’m working with a pure method? What is the trigger that I can use when learning whether I need to worry about it?
It’s my experience (and partially assumption) that awaitable code that relies on other awaitable code will automatically know to wait to execute until it has the value it needs from the other awaitable code; is this true across the board? What happens when I intermix synchronous and asynchronous code?
Is it truly a problem if I have blocking code if it’s not a costly method? Will there be logic problems? Flow control issues?
Is it OK if I use a TaskCancelledException to catch client HttpClient.*Async() timeouts? Should I refactor code to use cancellation tokens, even if no user-input is ever taken in? (the service itself doesn’t accept user input; it just processes logic).
I’m not at all sure if I’m alone in having these questions and everyone else gets it; or if it’s not more widely addressed because async isn’t widely adopted. I do know that in every .NET Codebase I’ve seen since async was released, I haven’t seen anyone write new code in using async (this is a terrible metric; don’t take it as some sort of scientific assertion, it’s just what I’ve seen).
True sync code (i.e. CPU bound code) is fine to mix with async code. However, if you are on a UI thread in a desktop application e.g. an event handler, you’d want to put that cpu bound code on a background thread or you will lock up the UI.
What’s often referred to as “sync” code is actually “blocking” code and has no CPU work going on.
For a server app, blocking code reacts very badly with async and the threadpool and server apps in general; as it just chews away at available threadpool threads forcing them into a stalled state doing no useful work. So you have to wait for the threadpool to spawn another thread to make progress; which then the blocking code will then stall.
Each new thread will aslo consume 1MB of stack space; so you can see this quickily runs into scaling problems. This is commonly referred to as the C10K problem where a number of clients (by today’s standards) can bring down a server due to resource exhaustion; and no client makes progress.
It is far more efficent and scalable to not have blocking calls and for all code to be async or actually doing something (CPU bound sync). Using the async model means you end up with a very low thread count; as work progresses until it can’t and then rather than blocking; it switches to other work it can make progress on.
Using Task.Run to execute blocking work will not help and can make the problem worse. Its good for UI work, but then you only have 1 client so its a different scalablity situation; where you are trying to avoid locking a UI. However you can still easily eat all the threadpool threads with blocking calls if you aren’t careful, so using proper async calls is better even in this situation.
You don’t have to make a method async if it doesn’t make any async/blocking calls; just because everything is “async”.
After all these years, all the codebases I’ve worked on with asynchronous code have either pre-dated the async/await keywords, so have kept on using the old-style Begin/End and explicit off-loading for consistency; or used a different language altogether.
> How far do I need to go down the asnyc rabbit hole?
You should use async if that method needs to use await. What you *don’t* want to do is force async in places where it doesn’t belong. Like CPU-bound methods on the server…
> JSON.NET doesn’t have Async
> Are small serializations/deserializations like the above CPU-bound? Are big ones? what is that threshold? How do you test for it?
Async/await is primarily for I/O. JSON serialization/deserialization is purely a CPU-bound activity (unless you’re doing I/O at the same time – i.e., deserializing *off a stream*). So the example `JsonConvert.DeserializeObject(jsonstring)` – taking an in-memory string and converting it to in-memory JSON – is fully CPU-bound and is not something that *should* be asynchronous.
> Since Microsoft recommends CPU-bound work be put in a task, what is the point where that should occur?
As a general rule, CPU-bound work should be wrapped in Task.Run only if it affects the UI thread. The specifics for this I’ve collected here ( https://blog.stephencleary.com/2013/04/ui-guidelines-for-async.html ). As there is no UI thread on the server side, the general rule is that you should *not* use Task.Run. https://msdn.microsoft.com/en-us/magazine/dn802603.aspx
> If you don’t put code inside an async method in a Task.Run; what happens? If it depends on previous code; it’ll run in order; but what if it doesn’t? Does it run immediately?
It starts running synchronously. https://blog.stephencleary.com/2012/02/async-and-await.html
Note that the point of `await` is to enable asynchronous code that is sequential (i.e., without needing explicit callbacks).
> Besides the nano-seconds of blocking, is there any other reason to care whether everything inside an async method is awaitable?
It *shouldn’t* be. There’s nothing wrong with an async method that calls synchronous code. Like, for example, `JsonConvert.DeserializeObject(jsonstring)`. You would want to avoid an async method that *blocks*, like `WebClient.DownloadString` and stuff like that, but calling (nonblocking) *synchronous* code from *asynchronous* code is perfectly fine and normal.
> RabbitMQ’s .NET client famously does not support async/await
The linked issue states that they’re building a new async client from scratch, so they’ve heard people requesting it and are responding.
> and you’ll even get errors in some places if you try to make the code async
I can’t find where the user guide states this. It says if you try to do multiple messages at the same time, you’ll see errors. This would be true whether you use async or not.
> How do you deal with synchronous libraries in asynchronous code?
On the server side, the best you can do is just call them synchronously. Yes, this means you end up blocking – potentially blocking within an async method – but if that’s the best you can do, then that’s the best you can do. Your scalability will be limited due to the synchronous blocking, but there isn’t anything you can do about it.
On the client side (where you have a UI thread), then you can wrap them in Task.Run and *pretend* they’re asynchronous (but in reality, you’re blocking a thread pool thread). I call this approach “fake asynchrony”.
> And Stack Overflow’s advice isn’t helpful
It’s important to note here that your question was about consuming a synchronous API inside your own code that you want to be asynchronous… but the SO questions you link to are about consuming asynchronous APIs inside your own code that is synchronous. They’re opposite scenarios.
So, my advice above is for async-over-sync. The SO questions and my next advice are for sync-over-async.
> What’s the answer? Don’t do it?
I’m one of the ones on SO who always say “don’t do sync-over-async”, because that’s the *ideal* answer. Ideally, all I/O should be asynchronous, and your code is never forced to be synchronous. But sometimes that’s not realistic, so I wrote an aritcle describing various hacks and their pros and cons: https://msdn.microsoft.com/en-us/magazine/mt238404.aspx
In your particular scenario – where you are doing async-over-sync – none of these hacks are needed.
> In another Stack Overflow post, the answer is, “Yea, you can do it with this code.”
That code is extremely dangerous. Based on the number of upvotes, I cringe at how many times it’s been copy/pasted into applications without developers understanding how it works or the dangers it poses.
> Keep your entire service synchronous because the message queueing system you use doesn’t support async? Or do you convert but implement this work-around for the pieces of code that need it?
In your scenario, just call the synchronous code directly. You’ll end up blocking, but it’s not possible to *force* code to be asynchronous when it wasn’t designed that way.
Under no circumstances should you use `AsyncHelpers`. It’s solving the wrong problem (sync-over-async, when you’re dealing with async-over-sync), and it’s dangerous to boot.
> Why is it after 5 years, the adoption of async seems to be negligible?
That’s not been my experience. There are some places (namely IoC/DI containers) that are lagging, but every other major library either already supports it or has plans to do so.
As cloud and mobile continue to accelerate, async will become more and more important.
What’s interesting to me is how many other *languages* have adopted async/await. Microsoft was not known as an innovator in language design until async came along.
> Unlike some languages, where you have no choice but to embrace async
I assume you’re referring to JavaScript, which (almost) has been async-only from the very beginning. This is simply due to its nature of being client UI code.
> the vast majority of blogposts I’ve read on the subject go into very topical and contrived uses; without digging deeper into the real pitfalls you’ll hit when you use async in an application.
Blog posts need reasonable length limitations, and the approaches you take for async in one situation are often very different than the same situation in another environment.
For example, earlier in this comment I’ve recommended calling RabbitMQ’s synchronous APIs directly. But that only makes sense *given your environment* – that is, in server-side code. If someone was in the same boat but needed to call it from a UI thread, I would recommend wrapping the call in Task.Run, to keep the UI thread free (blocking the thread pool thread instead). But that’s exactly what you *don’t* want to do on the server side, because you’d just end up performing a thread switch and throwing off your server’s thread pool heuristics for no benefit at all.
Perhaps we haven’t found the right way to talk about async yet, or are missing some key technological development, but at the moment the “best” answer (or even whether or not an answer is “right”) is very dependent on your environment.
> SynchronizationContext: When do I need to care about it? When do I not? What is the trigger that I can use when learning whether I need to worry about it?
Don’t care about your SyncCtx directly. Instead, care about what architectural level your methods are at.
As I note in my article on async best practices ( https://msdn.microsoft.com/en-us/magazine/jj991977.aspx ), use ConfigureAwait(false) on every await in a method that does not need its context. The methods that need the context are closest to the application level: they’re updating UI elements (client side), or using HttpContext.Current (server side). The methods that do not need the context are in a more abstract level: they’re handling business logic, etc., without any knowledge of the environment they’re running in.
> Do I only care about it when it’s being used inside an object with mutable state? Do I care about it if I’m working with a pure method?
Whether a method should use ConfigureAwait(false) is entirely determined by whether it needs its context, which is determined by what architectural level it’s in. It doesn’t have anything to do with mutable state. A pure method would not depend on its context, but lots of non-pure methods also do not depend on their context.
> It’s my experience (and partially assumption) that awaitable code that relies on other awaitable code will automatically know to wait to execute until it has the value it needs from the other awaitable code; is this true across the board?
Yes. See my async intro: https://blog.stephencleary.com/2012/02/async-and-await.html
> What happens when I intermix synchronous and asynchronous code?
Calling synchronous code from asynchronous code causes nothing unusual. `int.Parse` works just fine, and so does `JsonConvert.DeserializeObject`. If you call a blocking synchronous method from asynchronous code, then your code will not be as efficient as it could ideally be, but it would still work.
Calling asynchronous code from synchronous code is much more problematic. In the first place, you’re giving up all the benefits of asynchronous code as soon as you block on it (the *entire point* of asynchronous code is to free up threads). In addition to this, there’s no way to do it cleanly that works in every scenario. You have to resort to hacks: https://msdn.microsoft.com/en-us/magazine/mt238404.aspx
> Is it truly a problem if I have blocking code if it’s not a costly method? Will there be logic problems? Flow control issues?
No, no, and no. The worst impact – in your specific scenario – is that you will be blocking server threads on I/O. Blocked threads on servers means more resources used (and thus higher costs), and also means your server won’t scale up and down as quickly as it could with purely asynchronous code. But it would *work*, as in, provide the correct result.
> Is it OK if I use a TaskCancelledException to catch client HttpClient.*Async() timeouts?
I would catch `OperationCanceledException`. As a general rule, you should catch OCE because some APIs cause TCE and others cause OCE. Since OCE is the base type of TCE, it’s easier to just catch OCE rather than looking it up each time.
This isn’t related to async, but it is a little cancellation “gotcha”. I’m planning to do a video series on cancellation soon.
> Should I refactor code to use cancellation tokens, even if no user-input is ever taken in? (the service itself doesn’t accept user input; it just processes logic).
The first part to consider is “can I get a token that makes sense?” ASP.NET can provide a token that is cancelled when the client drops its connection or if the request cancelled due to an asynchronous timeout. You can also create your own token for manual timeouts, if you wish.
The second part to consider is “can I use it?” The vast majority of the time, your code will use cancellation tokens by merely passing them down to the APIs it calls. So, if the APIs you call take a token, then you can use it.
If you can get a token that makes sense for your service, and if your service can use it, then yes, you should refactor the code to use cancellation tokens.
> I do know that in every .NET Codebase I’ve seen since async was released, I haven’t seen anyone write new code in using async
Depends. I’ve seen a lot of new async codebases (I mean ones I didn’t write!).
On the other hand, I spent last weekend working on a URI parsing/formatting library. It’s purely synchronous – not an await to be seen in the whole thing. Async isn’t “the better way to do everything” – it’s just a tool, and if you’re doing synchronous work, then write it with synchronous code.
> I’m not at all sure if I’m alone in having these questions and everyone else gets it; or if it’s not more widely addressed because async isn’t widely adopted.
I’m working on some video instruction that I hope will be pretty exhaustive. Feel free to reach out if you have more questions!
This is one of the most useful comments I’ve seen in a long time, thank you very much for taking the time to write it! It’s ~four years later but this is still relevant and has helped me understand a few async concepts that I struggled with. Thanks to George as well for asking all these questions in the first place, too!
Part of the issue with async/await is documentation is scattered across MSDN and instead of finding the proper up to date references people use SO questions that are 5 years old….
Stephen Cleary’s response exhibits this issue as his blog posts are easier to find then the actual documentation.
I also feel that using async/await is “harder” in .Net core because the libraries around it are so young. Mainly, Web API and Entity framework. In a .Net standard app WebAPI model binding and EF async support solves most of the issues you experienced in your services.
Using async/await, unfortunately requires that you understand a bit about threading and how the framework and OS handle threads. Unless you already have a pretty good understanding of these things, you won’t be able to spend 20 minutes reading an article about async/await and be ready to start writing production code. It’s a bit of a long course, but I suggest watching the MVA video series by Wintellect “Advanced .NET Threading” available for free.