weixin_39575937
weixin_39575937
2021-01-04 01:02

[Application Insights] Configure reporting of diagnostic scopes from the Event Hubs client library

Query/Question When I receive batch from Event Hub, multiple EventData entries are grouped under one, single RequestTelemetry. This looks like this: image On the picture above, I have just one request telemetry but there can be more. All of those are aggregated using the: https://github.com/Azure/azure-sdk-for-net/blob/a43218f7f062a929bea20989094fc6566a5980b4/sdk/eventhub/Azure.Messaging.EventHubs/src/Primitives/EventProcessor%7BTPartition%7D.cs#L520 Here, we see, that all eventData entries are grouped under one scope.

I was digging into the App Insights and EventHub source code and was searching for string "EventProcessor.Process" which ointed me to this file: https://github.com/Azure/azure-sdk-for-net/blob/1e0ca217af52f045c54c1b063072e68d8ed785e9/sdk/eventhub/Azure.Messaging.EventHubs.Shared/src/Diagnostics/DiagnosticProperty.cs#L61

I'm adding App Insights to my console app (dockerized) as follows:

 csharp
return hostBuilder.ConfigureServices((context, collection) => collection.AddLogging(builder =>
{
    builder.Services.AddApplicationInsightsTelemetryWorkerService(options =>
    {
        options.InstrumentationKey = "key goes here";
        options.EnableAdaptiveSampling = false;

        options.DeveloperMode = false;
        options.EnableHeartbeat = false;
        options.EnableEventCounterCollectionModule = false;
        options.EnableDiagnosticsTelemetryModule = false;
        options.EnableDependencyTrackingTelemetryModule = true;
    });
}));

Question is: how do I disable this?

Environment: - Name and version of the Library package used:


<packagereference include="Microsoft.ApplicationInsights.DependencyCollector" version="2.15.0"></packagereference>
<packagereference include="Microsoft.ApplicationInsights.WorkerService" version="2.15.0"></packagereference>
<packagereference include="Azure.Messaging.EventHubs" version="5.2.0"></packagereference>
<packagereference include="Azure.Messaging.EventHubs.Processor" version="5.2.0"></packagereference>
  • Hosting platform or OS and .NET runtime version

.NET SDK (reflecting any global.json):
 Version:   5.0.100
 Commit:    5044b93829

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17763
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.100\

Host (useful for support):
  Version: 5.0.0
  Commit:  cf258a14b7

.NET SDKs installed:
  5.0.100 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  • IDE and version :

Visual Studio 19.4

该提问来源于开源项目:Azure/azure-sdk-for-net

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

5条回答

  • weixin_39611275 weixin_39611275 4月前

    Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc .

    Issue Details **Query/Question** When I receive batch from Event Hub, multiple EventData entries are grouped under one, single RequestTelemetry. This looks like this: ![image](https://user-images.githubusercontent.com/20299573/100746229-ba868100-33e0-11eb-8ea4-81720acc4e60.png) On the picture above, I have just one request telemetry but there can be more. All of those are aggregated using the: https://github.com/Azure/azure-sdk-for-net/blob/a43218f7f062a929bea20989094fc6566a5980b4/sdk/eventhub/Azure.Messaging.EventHubs/src/Primitives/EventProcessor%7BTPartition%7D.cs#L520 Here, we see, that all eventData entries are grouped under one scope. I was digging into the App Insights and EventHub source code and was searching for string "EventProcessor.Process" which ointed me to this file: https://github.com/Azure/azure-sdk-for-net/blob/1e0ca217af52f045c54c1b063072e68d8ed785e9/sdk/eventhub/Azure.Messaging.EventHubs.Shared/src/Diagnostics/DiagnosticProperty.cs#L61 I'm adding App Insights to my console app (dockerized) as follows:
     csharp
    return hostBuilder.ConfigureServices((context, collection) => collection.AddLogging(builder =>
    {
        builder.Services.AddApplicationInsightsTelemetryWorkerService(options =>
        {
            options.InstrumentationKey = "key goes here";
            options.EnableAdaptiveSampling = false;
    
            options.DeveloperMode = false;
            options.EnableHeartbeat = false;
            options.EnableEventCounterCollectionModule = false;
            options.EnableDiagnosticsTelemetryModule = false;
            options.EnableDependencyTrackingTelemetryModule = true;
        });
    }));
    
    Question is: how do I disable this? **Environment:** - Name and version of the Library package used:
    
    <packagereference include="Microsoft.ApplicationInsights.DependencyCollector" version="2.15.0"></packagereference>
    <packagereference include="Microsoft.ApplicationInsights.WorkerService" version="2.15.0"></packagereference>
    <packagereference include="Azure.Messaging.EventHubs" version="5.2.0"></packagereference>
    <packagereference include="Azure.Messaging.EventHubs.Processor" version="5.2.0"></packagereference>
    
    - Hosting platform or OS and .NET runtime version
    
    .NET SDK (reflecting any global.json):
     Version:   5.0.100
     Commit:    5044b93829
    
    Runtime Environment:
     OS Name:     Windows
     OS Version:  10.0.17763
     OS Platform: Windows
     RID:         win10-x64
     Base Path:   C:\Program Files\dotnet\sdk\5.0.100\
    
    Host (useful for support):
      Version: 5.0.0
      Commit:  cf258a14b7
    
    .NET SDKs installed:
      5.0.100 [C:\Program Files\dotnet\sdk]
    
    .NET runtimes installed:
      Microsoft.AspNetCore.All 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
      Microsoft.AspNetCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
      Microsoft.AspNetCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
      Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
      Microsoft.NETCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      Microsoft.NETCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      Microsoft.WindowsDesktop.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
      Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
    
    - IDE and version :
    
    Visual Studio 19.4
    
    Author:xagonAssignees:jsquireLabels: `Client`, `Monitor - ApplicationInsights`, `Service Attention`, `customer-reported`, `needs-team-attention`, `question` Milestone:-
    点赞 评论 复制链接分享
  • weixin_39634443 weixin_39634443 4月前

    Hi . If you'd be so kind, your thoughts on potential configuration options for displaying Event Hubs data within Application Insights would be greatly appreciated.

    点赞 评论 复制链接分享
  • weixin_39575937 weixin_39575937 4月前

    Oh god, I am so sorry I should have linked the example program from the beginning :)

    Example repo showing the issue: https://github.com/xagon/eventHub-group-of-appInsights-requests (HostBuilder + app insights worker registration + example implementation of Event Hub client)

    Now, run the app, then send 100 messages to Event Hub (partitioned). What will happen in App Insights, you will get something like this: image

    As you can see, we have a batch received of 4 messages, and they are all grouped under a single RequestTelemetry called "EventProcessor.Process". Said that, and the post above, we should have more than enough info I guess?

    Question: How do I disable this grouping when a batch of Event Hub Messages is received for processing? On the example we have 4 entries... but in production, it generates sometimes thousands of log entries which are "grouped" under one RequestTelemetry, but logically they are not related.

    点赞 评论 复制链接分享
  • weixin_39634443 weixin_39634443 4月前

    No need to apologies and no worries at all. I'm just trying to drill in so that I understand enough context to help. :smile:

    With respect to Event Hubs, what you're seeing is that a single request was made to the Event Hubs service to read a batch of events, which the processor then dispatched 1-by-1 to the event handler to process. The grouping is intended to correlate the batch that was read with the processing of each event in that batch.

    There is no way from the Event Hubs library there's no way to override the behavior for how spans are collected or reported. The design was based on a specification that the Azure Monitoring team wrote to try and align data across the Azure SDKs with the OpenTelemetry initiative.

    I'm not sure from the Application Insights side if there are options for filtering or tuning, so I'm going to loop the Monitor folks in to see if they have any guidance to offer.

    点赞 评论 复制链接分享
  • weixin_39634443 weixin_39634443 4月前

    Hi . Thank you for your feedback. Forgive me, but after reading the issue over a couple of times, I'm not clear on what you're asking. Can you help me understand what you're looking to disable?

    点赞 评论 复制链接分享

相关推荐