6 Tools to Detect Your .NET Performance Issues

In previous post, we have written an article about how to fix high CPU usage in server. In this post, we will give you tools to detect your .net performance issues, hopefully it will be more clearly and detail.

Performance issues can be very difficult if you don’t have the right tools. If you have the optics into the core of the matter, then half of the battle is already won. And the best way to get that kind of visibility is by knowing which tools to use. But there’s the problem of you don’t know what you don’t know. And if you aren’t even aware of what kind of tools exist and what they can show, you can find yourself fumbling in a dark room, looking for a needle in a haystack (did I get that right?). If you know that there are tools that can see metrics from production, see method execution times, and find slow database queries, you will have made your problems much easier. Luckily, there are plenty of excellent tools in the space of .NET. These can help you detect, monitor, optimize, and fix performance issues. Both locally and in the cloud.

It’s a big field and there are probably hundreds of tools out there, but they can be grouped into a small number of categories. Let’s see some of those categories as well as some specific tools:

1. Performance Counters and Performance Monitors

The operating system and the CLR provide a ton of information about a .NET process in the form of Performance Counters. These are a built-in mechanism in Windows that allows you to follow a whole lot of useful metrics. Performance counters are easy to use, come free, and perhaps not utilized as much as they deserve.

So what kind of information are we talking about here? Well, some examples are: CPU Usage, memory usage, exceptions thrown rate, ASP.NET request rate, ASP.NET request response time, and many more. There are hundreds of different counters you can monitor and they come as specific as possible. For example, if you want to find out about Memory Usage of a process, there are counters for Private Bytes, Virtual Bytes, Working Set, Working Set – Private, Gen X Collections, % Time in GC, Large Object Heap Size, and many more.

When working locally in Windows, you can use PerfMon to observe performance counters.

2. Performance Profilers

Another super important tool to detect and fix performance issues is a performance profiler. These profilers are able to record your .NET process for some period of time and then give visibility about what went on inside. This information is all you can dream of in performance analysis: it shows all the methods that were executed, the amount of time spent in each method, and the number of times each method was called (in certain modes). In one easy run of a performance profiler, you’ll be able to pinpoint the perf issue in the application.

In the above image, for example, (taken from dotTrace) you can see the method GetInput took 22 seconds (1), the code of that method (2), and that it used 36% system code, 28% reflection, and 21% WinForms code (3). And there’s much more information to get from the snapshot other than this view.

Performance profilers are usually executed locally, in your dev environment. But you can attach a profiler to a process in production with some caveats. For one thing, the profiler itself has some performance overhead when attached, and you probably won’t want to degrade your perf in production. For another, attaching to a process in production is pretty tricky. You’ll either have to attach remotely, which rarely works well in my experience, or copy a profiler agent to the production machine, attach from there, and then copy the performance snapshot back to a dev machine for analysis. Besides being a cumbersome process, it can be hard to analyze a production machine when there are a bunch of random user requests going on in parallel. You might be hard-pressed to pinpoint a specific issue. Another way to go is to get profile snapshots using an APM (coming up next), which is going to save you a lot of trouble, or with PerfView, which is going to save you a lot of performance overhead.

3. PerfView

If you are familiar with PerfView, you might say it’s another performance profiler, same as dotTrace or ANTS performance profiler, but I think it can be categorized a bit differently. So let’s first talk about what exactly is PerfView. It is an open-source analysis tool by Microsoft that relies mainly on ETW events. It can perform performance profiling, memory profiling, and a whole lot of various diagnostics reports on your .NET process.

The reason I’m placing this in a different category than performance profilers is that I think a regular performance profiler doesn’t replace PerfView and PerfView doesn’t replace a regular profiler. I’ll tell you why in a minute, but first let’s talk about what’s so special in PerfView. For one thing, it’s extremely customizable. So much so that you can extensively configure PerfView to how much or how little information you want it to capture. For example, when sampling performance you can change the sample interval from 1/8th of a millisecond up to 10 milliseconds or more. You can change whether to capture garbage collection information, thread change information (to be able to see async call stacks), whether to limit file capture size, whether to stop capture on CPU spike, on memory spike, on performance counter value or after some time passed. Why is all this important? Because it allows profiling with a very low performance overhead, as low as 3%. Combine that with a light-weight program that you can copy to a production machine, and a powerful command-line interface, and you can effectively profile production environments. So in this way, PerfView is different from a classic performance profiler.

The other thing about PerfView is that’s it’s very not intuitive. It has a ton of different settings that make it very hard to find your way around it and to get any sort of insights from it. You can easily deduct wrong conclusions because of not being familiar with one of its many subtleties. That’s the other side of the coin and why I think knowing PerfView doesn’t replace knowing a regular (and much easier) performance profiler.

Just to mention some more things that PerfView can do better than other performance profilers: profile both managed and native code, see the timeline view of when each method was called in the snapshot recording (also available in dotTrace Timeline Profiling), profile memory allocations, find memory leaks by comparing memory snapshots, see garbage collection reports, and more. In a way, PerfView is also a powerful memory profiler.

I mentioned before that PerfView is suitable for profiling in production. And while it’s very powerful, bear in mind that’s it’s very difficult to set up. That’s why you might be better off with APMs. Some APMS actually use PerfView under the hood, but save you the trouble of all the configuration and show the results more intuitively.

4. Application Performance Monitoring tools (APMs)

Application Performance Monitoring tools give an inside-the-hood view of your application’s performance and health in production. They will be able to show things like request counts and duration, CPU and memory on each machine, database query durations, traffic on each service, crashes, performance counters, exceptions, and more. Some of them will even be able to show code level performance profiling information, similar to what you see in a regular performance profiler when running locally. Other than that, APMs usually offer a logging storage target: a centralized database where you can send all your logs to.

As you can imagine, APMs are extremely useful for detecting performance problems in production. You can find the slowest requests and investigate them. You can see CPU and memory issues. You can go over database queries and find the slowest ones. You can even profile and see flame graphs of requests that you want to investigate (not all APMs support this).

APMs work by installing lean agents on your production machines. Those agents will either collect ETW logs of your .NET process (same as PerfView does) or attach to your .NET process as a profiler, which gives them access to .NET Profiling API. Or both.

5. BenchmarkDotNet

BenchmarkDotNet is a .NET library that allows you to measure execution times and create benchmarks. This is useful when you want to optimize some method locally and you’re trying to figure out what implementation works best. When you have, for example, two different implementations for something that executes in a hot spot in your code.

Comparing execution times can be tricky. You have to run the execution a bunch of times and calculate mean time, consider things like CPU warm-up and CPU cache, memory allocations, different CLR implementations, code optimization, and the list goes on. There are many pitfalls when doing a good benchmark. Luckily, there’s no reason for you to think of any of those things because BenchmarkDotNet takes care of everything.

public class Md5VsSha256
{
        private const int N = 10000;
        private readonly byte[] data;

        private readonly SHA256 sha256 = SHA256.Create();
        private readonly MD5 md5 = MD5.Create();

        public Md5VsSha256()
        {
                data = new byte[N];
                new Random(42).NextBytes(data);
        }

        [Benchmark]
        public byte[] Sha256() => sha256.ComputeHash(data);

        [Benchmark]
        public byte[] Md5() => md5.ComputeHash(data);
}

public class Program
{
        public static void Main(string[] args)
        {
                var summary = BenchmarkRunner.Run(typeof(Program).Assembly);
        }
}

Another usage of BenchmarkDotNet is to integrate it with your continuous integration and tests. When you have algorithms that are the core of your business, you’d want to be completely safe from any kind of bug degrading their performance. And if you want these measurements done reliably, you can count on BenchmarkDotNet.

6. Logs

You might be wondering why I placed plain old logs next to all the other powerful tools in the list. The reason is that logs are a crucial part of an application’s performance management. Logs are still the absolute best source of information from production. They are infinitely more reliable and granular than any kind of tool (like an APM).

To get reliable information from production, like how much time some method runs on average, or how long a database query took, you can use simple stopwatches and log the elapsed time. Here’s a simple example:

public void MyMethod()
{
        Stopwatch myMethodTime = Stopwatch.StartNew();
        try
        {
                // ...
        }
        finally
        {
        myMethodTime.Stop();
                _logger.log($"MyMethod took {myMethodTime.ElapsedMilliseconds} milliseconds");
        }
}

Or you can use some kind of decorator, base method, wrapper or attribute pattern.

The usual practice is to place such stopwatches in performance-critical areas of your code or in places where you already detected a performance problem. Then, you can start optimizing the perf issue and see reliable production data to find out if your changes improved or regressed performance.

When analyzing execution time, make sure to look at percentiles and not the average. Averages can paint a very misleading picture. For example, suppose that a method runs for 1ms in 90% of cases and 100ms in 10% of the cases. The average will be 11ms. Does it represent the actual situation? Not really, because in reality the method is working fine in 90% of cases and there’s some bug or issue that happens in the other 10%. The average would just tell you that the method is working 10 times slower than it should be, which is misleading.

Here’s an example query in KQL that goes over logs and show percentiles:

Logs
| parse Message With "MyMethod took " duration: int "milliseconds"
| summarize avg(duration), percentiles(duration, 50,75,90, 95)

The result will be in our case:

average_duration percentile_duration_50 percentile_Duration_75 percentile_Duration_90 percentile_Duration_95
11 1 1 100 100

Conclusion

We saw 6 different tools, or rather categories of tools, that help to deal with performance issues. Some of those run locally and measure benchmarks or detect problems. Other tools can run in your staging or production environments and gather real-usage data from there. Some tools are able to get general information like CPU usage, memory levels, and network traffic volume. While others can get more specific and show method execution times, database query time durations, and memory allocations. Some tools, like APMs, do a whole lot with minimal effort. Other technologies, like logs, require much more work by the developer, but in many ways provide better information.

Are you looking for an affordable and reliable Asp.net core hosting solution?

Get your ASP.NET hosting as low as $1.00/month with ASPHostPortal. Our fully featured hosting already includes

  • Easy setup
  • 24/7/365 technical support
  • Top level speed and security
  • Super cache server performance to increase your website speed
  • Top 9 data centers across the world that you can choose.

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *