Content uploaded by Nagib Sabbag Filho
Author content
All content in this area was uploaded by Nagib Sabbag Filho on Sep 23, 2024
Content may be subject to copyright.
Performance analysis in Csharp with
BenchmarkDotNet: Report and Evaluation
Nagib Sabbag Filho
FIAP (Faculty of Informatics and Administration Paulista) Avenida Paulista, 1106 - 7º andar - Bela Vista, São Paulo, Brazil.
e-mail: profnagib.filho@fiap.com.br
PermaLink: https://leaders.tec.br/article/9e2ce1
set 23 2024
Abstract:
This article provides an introduction to Benchmarking in C# using the BenchmarkDotNet library, essential
for performance analysis in software development. It covers everything from installation and configuration
to creating benchmarks to measure the efficiency of algorithms and data structures. The text details the
interpretation of results, report generation, and best practices to ensure accurate measurements.
Key words:
Performance analysis, C#, BenchmarkDotNet, detailed results, performance evaluation, performance
metrics, code optimization, benchmark testing, algorithm comparison, profiling, execution time, memory
usage, result accuracy, statistical analysis, benchmarking tools, software development, code efficiency.
Introduction to Benchmarking in C#
Performance analysis is a critical part of software development, especially in applications where efficiency and speed
are essential. BenchmarkDotNet is a popular library in C# that allows developers to measure the performance of their
applications accurately and reliably. With the increasing complexity of applications, it becomes vital to understand how
different pieces of code behave under varied conditions. In this article, we will explore how to use BenchmarkDotNet
to perform performance measurements, interpret the results, and apply the necessary improvements.
Benchmarking is essential to ensure that the code not only functions correctly but also executes its functions
efficiently. Measuring performance can reveal bottlenecks, enabling developers to make necessary adjustments and
improvements. Furthermore, good benchmarking can be a deciding factor in choosing algorithms and data structures,
directly impacting the end-user experience.
Installation and Configuration of BenchmarkDotNet
To start using BenchmarkDotNet, you need to install it in your project. Installation can be easily done through NuGet.
In the Visual Studio Package Manager Console, run the following command:
Install-Package BenchmarkDotNet
After installation, you can start creating your benchmarks. A simple example of a benchmark can be seen below:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
public class MyBenchmarks
{
[Benchmark]
public int Sum()
{
Leaders.Tec.Br Volume 1, Number 12
1
int result = 0;
for (int i = 0; i < 1000; i++)
{
result += i;
}
return result;
}
}
public class Program
{
public static void Main(string[] args)
{
BenchmarkRunner.Run<MyBenchmarks>();
}
}
In this example, we created a simple benchmark that measures the execution time of a summation operation. The
Sum method is annotated with the [Benchmark] attribute, indicating that it should be measured. When executing this
benchmark, BenchmarkDotNet will handle the necessary configuration and performance data collection.
It is important to remember that simple benchmarks may not show the full potential of BenchmarkDotNet. To truly take
advantage of the library, you should consider more complex scenarios involving different algorithms and data
structures. This not only helps understand performance under different conditions but also allows for more meaningful
comparisons.
Interpreting the Results
After executing the benchmark, BenchmarkDotNet generates a detailed report with the results. The main data
presented includes:
Mean: The average execution time of the benchmark.
Standard Deviation: The variability of the measurements.
Median: The median value of the measurements.
This data helps analyze not only the average performance but also the consistency of the measurements. Let's
consider a more complex example that involves comparing different sorting algorithms.
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System;
using System.Linq;
public class SortingBenchmarks
{
private int[] data;
[GlobalSetup]
public void Setup()
{
Random rand = new Random();
data = Enumerable.Range(1, 10000).OrderBy(x => rand.Next()).ToArray();
}
Leaders.Tec.Br Volume 1, Number 12
2
[Benchmark]
public int[] QuickSort()
{
return data.OrderBy(x => x).ToArray();
}
[Benchmark]
public int[] BubbleSort()
{
int[] arr = (int[])data.Clone();
for (int i = 0; i < arr.Length - 1; i++)
{
for (int j = 0; j < arr.Length - i - 1; j++)
{
if (arr[j] > arr[j + 1])
{
int temp = arr[j];
arr[j] = arr[j + 1];
arr[j + 1] = temp;
}
}
}
return arr;
}
}
public class Program
{
public static void Main(string[] args)
{
BenchmarkRunner.Run<SortingBenchmarks>();
}
}
In the code above, we compare the performance of the quicksort algorithm and the bubble sort algorithm. The global
setup generates a random dataset of 10,000 elements, which is used for both benchmarks. Analyzing the results will
allow us to identify which algorithm is more efficient in terms of execution time.
Interpreting the results can reveal not only which algorithm is faster but also the efficiency in different scenarios. For
example, on already sorted datasets, the bubble sort algorithm may perform adequately, while on random data,
QuickSort will likely excel. It is here that careful analysis of the results becomes crucial.
Reports and Result Visualization
BenchmarkDotNet has rich reporting functionality, allowing results to be exported in different formats, including
Markdown, HTML, and CSV. To generate HTML reports, you can use the following command in your terminal after
executing the benchmark:
dotnet benchmark --exporters Html
This functionality is extremely useful for sharing results with your team or for documenting the performance of your
code over time. An example of a generated report may include graphs and tables that show the performance of
Leaders.Tec.Br Volume 1, Number 12
3
different implementations side by side.
Reports can be viewed directly in a browser, making it easy to analyze and present the results. Additionally, you can
include comments and annotations that help contextualize the presented data, making reports a powerful tool for
communication among team members and stakeholders.
Best Practices for Benchmarking
When conducting benchmarks, it is important to follow some best practices to ensure the accuracy of the results:
Isolate tests: Ensure that measurements are not affected by other running processes. This can be done by running
benchmarks in a controlled environment where only the application under test is running.
Run multiple iterations: Run your benchmarks multiple times to obtain a reliable average. BenchmarkDotNet
does this automatically, but it is important to understand this practice to ensure stable results.
Avoid JIT optimizations: Use the [GlobalSetup] attribute to prepare the test state before measurements. This
helps minimize the impact of Just-In-Time compilation on performance measurements.
Document tests: Keep a detailed record of the tests performed, including the conditions under which they were
executed and the results obtained. This documentation can be valuable for future performance analyses.
Integration with CI/CD
Integrating benchmarks into CI/CD pipelines can be an effective way to monitor performance over time.
BenchmarkDotNet can be easily configured to run benchmarks automatically on each commit or pull request. This
ensures that any performance degradation is detected quickly. To integrate it, you can add a step in your pipeline to
execute the benchmarks and generate reports.
This practice not only helps maintain code quality but also provides quick feedback for developers, allowing fixes to be
made before the code is merged. In addition, integrating benchmarks can help establish a performance culture within
the development team, where code efficiency is prioritized.
Advanced Use Cases and Final Considerations
BenchmarkDotNet also supports more advanced use cases, such as:
Memory testing, where you can measure the amount of memory allocated by your operations. This is especially useful
in applications that deal with large volumes of data or need to operate in resource-constrained environments.
Customizing benchmark environments, allowing you to simulate different execution conditions. You can, for
example, adjust system load or simulate different types of hardware.
Comparing different versions of your code, helping to identify which changes had a positive or negative impact
on performance.
By using BenchmarkDotNet effectively, you can not only optimize your code but also ensure that improvements are
sustainable over time. Performance analysis should be an ongoing part of the development lifecycle, and tools like
BenchmarkDotNet are essential in this process.
References
BENCHMARKDOTNET. BenchmarkDotNet. Available at: https://benchmarkdotnet.org/. Accessed in: 2024.
MICROSOFT. Attributes. Available at: https://docs.microsoft.com/dotnet/csharp/programming-guide/inside-a-
program/attributes. Accessed in: 2024.
MICROSOFT. Generic Collections. Available at: https://docs.microsoft.com/en-us/dotnet/standard/collections/generic/.
Accessed in: 2024.
Nagib is a University Professor and Tech Manager. He has a track record of achievements in technical and agile certifications,
including MCSD, MCSA, and PSM1. He holds a postgraduate degree in IT Management from SENAC and an MBA in Software
Leaders.Tec.Br Volume 1, Number 12
4