weixin_39766867
weixin_39766867
2021-01-12 03:23

Fatal error. Internal CLR error. (0x80131506) - Unknown reason!

Description

I have created an AVX2 (256-bit) version of ChaCha20 symmetric encryption algorithm, however after using the following trick in order to force compiler to generate better performance code (link), everything failes on runtime error. This happens when I'm trying to access 'CachedMasks' static property of type ReadOnlySpan as it should take data from '.text' section of executable file. I dont understant if JIT generates a bad sequence of instructions, but it does not seem to be a GC problem with reference. I have no clue how to solve this or why this happens on multiple machines. I can provide a whole sample aplication if nessesary.

c#
        private static ReadOnlySpan<uint> CachedMasks => new uint[]
        {
           1, 2, 3, 0, 5, 6, 7, 4,   // <<< 32, >>> 96
           2, 3, 0, 1, 6, 7, 4, 5,   // <<< 64, >>> 64
           3, 0, 1, 2, 7, 4, 5, 6,   // <<< 96, >>> 32
        };
</uint>

Configuration

Note: This problem happends on my home PC and in work too! It is not machine specific.

.NET Core SDK (reflecting any global.json): Version: 3.1.101 Commit: b377529961

Runtime Environment: OS Name: Windows OS Version: 10.0.17763 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\3.1.101\

Host (useful for support): Version: 3.1.1 Commit: a1388f194c

.NET Core SDKs installed: 1.0.2 [C:\Program Files\dotnet\sdk] 2.1.202 [C:\Program Files\dotnet\sdk] 2.1.502 [C:\Program Files\dotnet\sdk] 2.1.503 [C:\Program Files\dotnet\sdk] 2.1.504 [C:\Program Files\dotnet\sdk] 2.1.505 [C:\Program Files\dotnet\sdk] 2.1.507 [C:\Program Files\dotnet\sdk] 2.1.600 [C:\Program Files\dotnet\sdk] 2.1.602 [C:\Program Files\dotnet\sdk] 3.0.101 [C:\Program Files\dotnet\sdk] 3.1.101 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed: Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 1.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 1.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.WindowsDesktop.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other information

c#
Fatal error. Internal CLR error. (0x80131506)
   at ExCrypto.ChaCha20.get_CachedMasks()
   at ExCrypto.ChaCha20.DB_QUATERROUND(System.Runtime.Intrinsics.Vector256`1<uint32> ByRef, System.Runtime.Intrinsics.Vector256`1<uint32> ByRef, System.Runtime.Intrinsics.Vector256`1<uint32> ByRef, System.Runtime.Intrinsics.Vector256`1<uint32> ByRef)
   at ExCrypto.ChaCha20.chacha20_block(ExCrypto.chacha20_ctx, System.Span`1<byte>)
   at ExCrypto.ChaCha20.chacha20_encrypt(ExCrypto.chacha20_ctx, System.ReadOnlySpan`1<byte>, System.Span`1<byte>)
   at ChaChaAVX2_Testing.Program.ChaCha20RfcBenchmark()
   at ChaChaAVX2_Testing.Program.Main(System.String[])
</byte></byte></byte></uint32></uint32></uint32></uint32>

Everything is caused by this line of code:

c#
uint* masks = (uint*)Unsafe.AsPointer(ref MemoryMarshal.GetReference(CachedMasks));

if changed to this line, no runtime error will appear:

c#
uint* masks = stackalloc uint[] { 1, 2, 3, 0, 5, 6, 7, 4, 2, 3, 0, 1, 6, 7, 4, 5, 3, 0, 1, 2, 7, 4, 5, 6 };

Note: I cannot use second line because my code is working with AVX2 instructions and that would lead to a bad performance.

该提问来源于开源项目:dotnet/runtime

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

7条回答

  • weixin_39737233 weixin_39737233 4月前

    I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

    点赞 评论 复制链接分享
  • weixin_39582480 weixin_39582480 4月前

    Aside ReadOnlySpan<uint> CachedMasks => new uint[] only works for ReadOnlySpan<byte> for other types it will allocate an array each time (due to unknown endianness) so CachedMasks will not be a fixed location for uint, which will mean the (uint*)Unsafe.AsPointer call will create an untracked pointer and GC hole.

    点赞 评论 复制链接分享
  • weixin_39766867 weixin_39766867 4月前

    Good to know, I'll try an uint array instead. But when I use fixed statement it will crash too? why?

    c#
    fixed (uint* masks = &MemoryMarshal.GetReference(CachedMasks))
    { /*TODO*/ }
    
    点赞 评论 复制链接分享
  • weixin_39982568 weixin_39982568 4月前
    cs
    uint* masks = (uint*)Unsafe.AsPointer(ref MemoryMarshal.GetReference(CachedMasks));
    

    You create a GCHole because only byte data is forced into the RO data section. Either allocate it with GC.AllocateArray(len, pinned: true), pin it, or, given you are using this on AVX2 and can assume little endian, you could just do

    cs
    private static ReadOnlySpan<byte> CachedMasks => new byte[]
    {
        1, 0, 0, 0,        2, 0, 0, 0,        3, 0, 0, 0,        0, 0, 0, 0,    5, 0, 0, 0,     6, 0, 0, 0,     7, 0, 0, 0,     4, 0, 0, 0,
        2, 0, 0, 0,        3, 0, 0, 0,        0, 0, 0, 0,        1, 0, 0, 0,    6, 0, 0, 0,     7, 0, 0, 0,     4, 0, 0, 0,     5, 0, 0, 0,
        3, 0, 0, 0,        0, 0, 0, 0,        1, 0, 0, 0,        2, 0, 0, 0,    7, 0, 0, 0,     4, 0, 0, 0,     5, 0, 0, 0,     6, 0, 0, 0
    }
    </byte>

    (pretty sure that is the right way to do it for little endian)

    点赞 评论 复制链接分享
  • weixin_39766867 weixin_39766867 4月前

    Gonna to try it. But I think that you did not answered my question about 'fixed' statement, if 'ReadOnlySpan' allocates a new array each time because it doesnt know endianess in advance, then if I get a managed pointer on it and use it inside of 'fixed' block, GC should deallocate that pointer once that block is finished?! (I'm using sequential code, no async or parallel stuff)

    点赞 评论 复制链接分享
  • weixin_39766867 weixin_39766867 4月前

    FIXED! Thanks a lot guys! I'm going to close this issue!

    There should be some warning section in docs that if you are using ReadOnlySpan as static property or variable and you will use any other parameter than byte than behaviour will change drastically! Normal .NET devs dont know about this.

    Can someone add this into documentation as a warning section - "When using ReadOnlySpan as static variable/property"?

    Thanks.

    点赞 评论 复制链接分享
  • weixin_39994806 weixin_39994806 4月前

    An analyzer could perhaps warn about static ReadOnlySpan<T> property => new T[] { … }; where T is not byte and the array elements are constants, and suggest replacing with static ReadOnlySpan<T> property { get; } = new T[] { … };. That would fix the repetitive construction of arrays but not the crash caused by not pinning the array.

    点赞 评论 复制链接分享

相关推荐