weixin_39886238
weixin_39886238
2021-01-01 13:36

Is it time to say goodbye to russian dolls?

and I did some spike investigations during the performance checks for V6.

The two biggest concerns which drove us to do the spiking were: - The stack trace in case of exceptions in the pipeline is massive - The performance of the pipeline tends to decrease up to 300% when an exception is raised inside a behavior.

Before you read on a few quick facts: - We always had the stack trace problem. That is the nature of the russian doll model. - The stack trace became larger when we introduced async/await - Introducing async/await changed how the code behaves under exceptions

With this issue I want to raise awareness and start a discussion.

Performance

The performance decrease is caused by the async statemachine and how exception propagation works with async. Here is a short snippet which is part of the TaskAwaiter infrastructure:


    private static void ThrowForNonSuccess(Task task)
    {
      switch (task.Status)
      {
        case TaskStatus.Canceled:
          ExceptionDispatchInfo exceptionDispatchInfo = task.GetCancellationExceptionDispatchInfo();
          if (exceptionDispatchInfo != null)
            exceptionDispatchInfo.Throw();
          throw new TaskCanceledException(task);
        case TaskStatus.Faulted:
          ReadOnlyCollection<exceptiondispatchinfo> exceptionDispatchInfos = task.GetExceptionDispatchInfos();
          if (exceptionDispatchInfos.Count <= 0)
            throw task.Exception;
          exceptionDispatchInfos[0].Throw();
          break;
      }
    }
</exceptiondispatchinfo>

Translating this to natural language:

When a behavior throws an exception that exception gets captured in an ExceptionDispatchInfo and rethrown and catched on every await in the call stack.

When an exception happens the implications of that are massive when it comes to heap allocations and execution speed:

5 threads and 100 iterations with a 10ms delay in many behaviours execution time: 16sec .net is using 4MB of 13MB total private bytes 10.7KB Gen1 140.2KB Gen2

Here is a simulation which shows the effect.

https://github.com/danielmarbach/AsyncCageMatch/blob/master/AsyncOnly/Program.cs#L69

Note: Memory pressure needs to be measured with a memory profiler

The deeper down in the callstack the exception happens, the more heap allocations and performance impacts can be observed.

Callstack

The russian doll model in combination with async/await essentially adds the following to the call stack: - MoveNext for each await statement, as you can see below the PassthroughBehavior is not visible because it doesn't await. - HandleNonSuccessAndDebuggerNotification and - ThrowForNonSuccess


System.InvalidOperationException: ThrowExceptionBehavior
   at AsyncOnly.ThrowExceptionBehavior.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 274
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayBehavior8.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 186
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayBehavior7.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 176
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayBehavior6.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 166
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayBehavior5.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 156
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayInUsingBehavior2.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 213
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayTwiceBehavior4.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 262
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayTwiceBehavior3.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 250
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayTwiceBehavior2.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 238
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayTwiceBehavior1.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 226
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayInUsingBehavior1.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 198
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayBehavior4.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 146
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayBehavior3.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 136
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayBehavior2.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 126
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.DelayBehavior1.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 116
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncOnly.Program.Main(String[] args) in C:\particular\AsyncCageMatch\AsyncOnly\Program.cs:line 17
</invoke></invoke></invoke></invoke></invoke></invoke></invoke></invoke></invoke></invoke></invoke></invoke></invoke></invoke></invoke>

What did we try to solve in the spike?

  • Stacktrace
  • Performance

What approaches did we see?

  • Using a monadic model instead of async/await
  • Implemenent trampolining

Monadic approach

Can be seen

https://github.com/danielmarbach/AsyncCageMatch/blob/master/AsyncMagic/Program.cs

Basically we changed to pipeline to return a BehaviorContinuation similar to the reactive model.

This approach has the following impacts, again under exceptions

Performance

5 threads and 100 iterations with a 10ms delay in many behaviours execution time 5sec .net is using 1MB of 10MB total private bytes 144.9KB Gen1 0 bytes Gen2

So essentially a third of the execution time, a third of the private bytes used on the heap. Less pressure on the gargabe collector. No Gen2 objects.

Callstack


System.InvalidOperationException: ThrowExceptionBehavior
   at AsyncMagic.ThrowExceptionBehavior.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncMagic\Program.cs:line 352
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncMagic.BehaviorChain.<invoke>d__2.MoveNext() in C:\particular\AsyncCageMatch\AsyncMagic\Program.cs:line 93
</invoke></invoke>

Downsides

The problem of this approach is that we need to wrapped everything important into a closure and dealing with things that are disposable becomes really complex. For example


    public class DelayInUsingBehavior1 : IBehavior
    {
        public async Task<behaviorcontinuation> Invoke(BehaviorContext context)
        {
            var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled);

            await SomethingThatThrows();
</behaviorcontinuation>
  • If SomethingThatThrows throws the scope would never be disposed. We would need an infrastructure which allows to put IDisposables on the context and disposes them properly.
  • The behavior chain would need to implement parts of the logic which is done by the compiler when we use async/await.
  • It is also a much more unnatural programming model compared to async/await only.
  • And in the success case it is essentially slightly slower (maybe 2 %).

Trampolining

First, I tried to implement Trampolining with Russian Dolls. It is not possible because we don't use tail calls. If we used trampolining without russian dolls we could get rid of recursion and use the trampolin to forward the pipelline.

https://github.com/danielmarbach/AsyncCageMatch/blob/master/AsyncTrampolining/Program.cs

Without dolls

https://github.com/danielmarbach/AsyncCageMatch/blob/master/AsyncTrampoliningWithoutDolls/Program.cs

Performance

Haven't measured yet

Callstack


System.InvalidOperationException: ThrowExceptionBehavior
   at AsyncTrampolining.ThrowExceptionBehavior.<invoke>d__0.MoveNext() in C:\particular\AsyncCageMatch\AsyncTrampoliningWithoutDolls\Program.cs:line 351
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncTrampolining.BehaviorChain.<>c__DisplayClass3_0.<<invokenext>b__0>d.MoveNext() in C:\particular\AsyncCageMatch\AsyncTrampoliningWithoutDolls\Program.cs:line 188
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncTrampolining.Trampoline.<>c__DisplayClass0_0`2.<<maketrampoline>b__0>d.MoveNext() in C:\particular\AsyncCageMatch\AsyncTrampoliningWithoutDolls\Program.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AsyncTrampolining.Program.Main(String[] args) in C:\particular\AsyncCageMatch\AsyncTrampoliningWithoutDolls\Program.cs:line 20
</maketrampoline></invokenext></invoke>

Downsides

  • Not possible to implement trampolining with the current russian doll model
  • Not proven to be faster
  • Less efficient than the standard async/await model in the success case.

Partial dolls

I couldn't come up with a better name, sorry.

By looking at our behaviors I realized that we essentially have three types of behaviors:


class BeforeBehavior : Behavior<tcontext>
{
   Task Invoke(TContext context, Func<task> next) {
       await DoSomethingBeforeNext();
       await next();
   }
}

class AfterBehavior : Behavior<tcontext>
{
   Task Invoke(TContext context, Func<task> next) {
       await next();
       await DoSomethingAfterNext();
   }
}

class SurroundBehavior : Behavior<tcontext>
{
   Task Invoke(TContext context, Func<task> next) {
       await DoSomethingBeforeNext();
       await next();
       await DoSomethingAfterNext();
   }
}
</task></tcontext></task></tcontext></task></tcontext>

Behaviors of type Before and After could essentially be changed to this:


class BeforeBehavior : Behavior<tcontext>
{
   Task Invoke(TContext context) {
       await DoSomethingBeforeNext();
   }
}

class AfterBehavior : Behavior<tcontext>
{
   Task Invoke(TContext context) {
       await DoSomethingAfterNext();
   }
}
</tcontext></tcontext>

and the pipeline model builder could make sure that behaviors of type After would be moved at the end of a pipeline and be executed in reverse order. With that trickery you could turn the nested doll model from


Behavior
   Behavior
     Behavior
        Behavior
           Behavior
              Behavior
                 Behavior
                   Behavior
                      Behavior
                         Behavior
                            Behavior
                               Behavior
                                 Behavior

to something like


Surround1
   Surround2
     Before1 
     Before2
     Surround3
       Before3
       Before4
       Before5
       After5
       After4
     After3
     After2
     After1

essentially bringing down the depth of the pipeline from N number of Behaviors to M number of Surrounding Behaviors.

Downsides

I did a quick spike on the approach. It has a few problems - When implementing a behavior you suddenly have to decide whether you want it to be a before, after or surrounding behavior - When implementing a stage connector you have to decide wether you need a stage connector that just creates one or many contexts or the stage connectors itself is a surrounding stage connector

Note: We would need to introduce something like


    public interface IStageConnector<tfrom tto>
        where TFrom : IBehaviorContext
        where TTo : IBehaviorContext
    {
        Task<ienumerable>> Invoke(TFrom context);
    }
</ienumerable></tfrom>

and


    public interface IStageConnector<tfrom tto>
        where TFrom : IBehaviorContext
        where TTo : IBehaviorContext
    {
        Task Invoke(TFrom context, Func<tto task> next);
    }
</tto></tfrom>
  • The sorting algorithm is not trivial
  • The actual behavior chain execution becomes more complex because you no longer have a uniform interface you can invoke
  • The currently implemented StageBehaviors, Behaviors... in core show that we'd still end up having many surrounding behaviors like FLR, SLR, MoveFault, TxScopeWrapper, UoW, DeserializeConnector, TransportReceiveConnector, LoadHandlersConnector...

Conclusion

The most pragmatic solution would be the following: - When an exception happens these operations are expensive anyway. Not much we need to improve. Let's hope that future compiler and .NET framework version will make this story better. - Plug in a stack trace cleaner (which can be overridden or disabled) which filters the stack trace to only show the offending behavior.

Questions

  • Should we put more thought into a better model which tackles both the callstack and the performance issue under exceptions? or
  • Should we go with the pramatic solution described above?

该提问来源于开源项目:Particular/NServiceBus

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

12条回答

  • weixin_39886238 weixin_39886238 4月前

    Another approach would be to use IBehavior and essentially decorate them into each other when necessary based on the required execution plan. Similar to something I did for Caliburn Micro years ago

    https://github.com/danielmarbach/StockTicker/blob/master/source/StockTicker/StockTickerViewModel.cs#L57 https://github.com/danielmarbach/StockTicker/blob/master/source/StockTicker/Actions/ActionBuilder.cs https://github.com/danielmarbach/StockTicker/blob/master/source/StockTicker/Actions/AsyncResultDecorator.cs

    So essentially we could add FLR, SLR behaviors as decorators to all behaviors which really need it. So instead of a deeply nested callstack we'd end up with a call stack which is only nested where it needs to be nested and everything else is executed in a while loop. So essentially something like

    
    NeedsToWrapAll
     -- NeedsToWrapPart
         -- Behavior
         -- Behavior
         -- BehaviorWithUsingThatNeedsToWrapInner
            -- Behavior
            -- Behavior
     -- Behavior
     -- Behavior
    
    

    Since we have strongly typed stages could annotate the stages it needs to wrap and then sort them accordingly in the pipeline model builder.

    点赞 评论 复制链接分享
  • weixin_39713219 weixin_39713219 4月前

    that "monadic" approach somehow feels like reimplementing async/await with sticks and stones at a huge cost for the happy cases & developer experience.

    would something like that be simpler?

    
    class MyBehavior : Behavior<mycontext>{
        public override OnNext(message, ctx){ ... }
        public override OnComplete(ctx) { ... }
        public override OnError(ctx) { ... }
        public override Finally(ctx) { ... }
    }
    </mycontext>
    点赞 评论 复制链接分享
  • weixin_39587407 weixin_39587407 4月前

    i like that api. Would be interesting to see how complex it makes a real behaviour

    点赞 评论 复制链接分享
  • weixin_39886238 weixin_39886238 4月前

    I've given some thought at that proposal and the complexity is still the same because you have to use the context to track changes. Furthermore if we do that the behaviors can no longer be reused if you make the mistake of sharing data over method level.

    点赞 评论 复制链接分享
  • weixin_39886238 weixin_39886238 4月前

    see my statement above about the monadic approach ;)

    The behavior chain would need to implement parts of the logic which is done by the compiler when we use async/await.

    点赞 评论 复制链接分享
  • weixin_39682697 weixin_39682697 4月前

    Are the async infrastructure methods the only thing we don't like about the stacktrace?

    点赞 评论 复制链接分享
  • weixin_39587407 weixin_39587407 4月前

    three things - the messy stack trace - async causes slower perf during exceptions. since at ever level of the pipeline the exception is re-thrown instead of bubbled - since exceptions are rethown if u have "break on all exceptions" turned on VS will break as many times as there are levels in the pipeline

    点赞 评论 复制链接分享
  • weixin_39886238 weixin_39886238 4月前

    Updated the issue description with partial dolls

    点赞 评论 复制链接分享
  • weixin_39886238 weixin_39886238 4月前

    I came to the conclusion (confirmed with and ) that partial dolls doesn't give enough benefits to justify the added complexity. The async dolls as it stands today is still the most elegant and convenient solution we have. I'm closing this issue.

    点赞 评论 复制链接分享
  • weixin_39886238 weixin_39886238 4月前

    By the way. ActionFilters from Asp.net vNext suffer from the same stack trace problem. See here

    https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNetCore.Mvc.Core/Internal/FilterActionInvoker.cs#L267-L294

    点赞 评论 复制链接分享
  • weixin_39886238 weixin_39886238 4月前

    ping

    点赞 评论 复制链接分享
  • weixin_39682697 weixin_39682697 4月前

    Monadic approach looks promising but as far as I understand it has two major issues: - Error-prone resource management - Is a non-idiomatic C#

    The latter we probably cannot change. However first problem could potentially be mitigated with some fluent api and explicit closure modeling e.g.:

    
    class BContext : BehaviorContext {
        Transaction Transaction {get; set;}
    }
    
    void Define(Pipeline pipline){
        Pipeline<bcontext>.Define()
            .Invoke((c) =>{
                c.Transaction = new Transaction()
            })
            .After((c) => {
                c.Transaction.Complete()
            })
            .Failure((c, e) =>{
                c.Transaction.Rollback();
            })
            .Finally((c) => {
                c.Transaction.Dispose()
            })
    } 
    </bcontext>
    点赞 评论 复制链接分享

相关推荐