Pro .NET Benchmarking: The Art of Performance Measurement



Use this in-depth guide to correctly design benchmarks, measure key performance metrics of .NET applications, and analyze results. This book presents dozens of case studies to help you understand complicated benchmarking topics. You will avoid common pitfalls, control the accuracy of your measurements, and improve performance of your software. Author Andrey Akinshin has maintained BenchmarkDotNet (the most popular .NET library for benchmarking) for five years and covers common mistakes that developers usually make in their benchmarks. This book includes not only .NET-specific content but also essential knowledge about performance measurements which can be applied to any language or platform (common benchmarking methodology, statistics, and low-level features of modern hardware). What You'll Learn: • Be aware of the best practices for writing benchmarks and performance tests • Avoid the common benchmarking pitfalls • Know the hardware and software factors that affect application performance • Analyze performance measurements This book is for .NET developers concerned with the performance of their applications. Andrey Akinshin, PhD is a .NET MVP, ACM ICPC medalist, project lead of BenchmarkDotNet, program director of the DotNext conference, and software developer at JetBrains where he works on Rider.
... However, the benchmarks are not safe from external factors like environmental temperature, mechanical vibrations, or noise from other OS processes. According to Akinshin the natural noise can make up 5%-20% of the measured test duration [47]. Executing the benchmarks multiple times helps generating a more stable result. ...
... 11 API application programming interface. 44,47 CCU concurrent users. 8,9,16,26,43 CPU central processing unit. ...
The broad coverage of 4G networks and the introduction of 5G networks result in new opportunities for multiplayer mobile games. Unity, the most-used game engine for mobile applications, offers multiple solutions to develop a server application inside the engine. However, this thesis evaluates the benefits of developing a separate .NET server application for a Unity client. Three different server targets are used to compare operating systems, networking libraries, and serialization libraries. In addition, a game was developed alongside the benchmarks to test security aspects and run the libraries in a real-world environment. The combination of a .NET server with the mobile network infrastructure of Europe allows for round-trip times of less than 100 milliseconds while using an inexpensive server solution. Manual serialization yielded the best performance results while keeping packet size at a minimum and was especially used for packets that are frequently sent. For messages that can change or are not sent as often, libraries like MessagePack-CSharp or Protocol Buffers are interesting alternatives. In the networking benchmarks, ENet performed best and had the most promising metrics for weaker hardware setups. For all tests, the User Datagram Protocol (UDP) socket performance was better on Linux than on Windows.
Full-text available
Performance regressions can have a drastic impact on the usability of a software application. The crucial task of localizing such regressions can be achieved using bisection, which attempts to find the bug-introducing commit using binary search. This approach is used extensively by many development teams, but it is an inherently heuristical approach when applied to performance regressions, and therefore, does not have correctness guarantees. Unfortunately, bisection is also time-consuming, which implies the need to assess its effectiveness prior to running it. To this end, the goal of this study is to analyze the effectiveness of bisection for performance regressions. This goal is achieved by first formulating a metric that quantifies the probability of a successful bisection, and extracting a list of input parameters – the contributing properties – that potentially impact its value; a sensitivity analysis is then conducted on these properties to understand the extent of their impact. Furthermore, an empirical study of 310 bug reports describing performance regressions in 17 real-world applications is conducted, to better understand what these contributing properties look like in practice. The results show that while bisection can be highly effective in localizing real-world performance regressions, this effectiveness is sensitive to the contributing properties, especially the choice of baseline and the distributions at each commit. The results also reveal that most bug reports do not provide sufficient information to help developers properly choose values and metrics that can maximize the effectiveness, which implies the need for measures to fill this information gap.
Full-text available
In this paper we improve the efficiency of the simple matrix-multiplication algorithm using parallelism and hardware instrinsics with C# and .Net Task Parallel Library. We demonstrate how additional processing can result the overall shorter execution times. We present an algorithm in C# that achieves up to five times performance improvement on midrange hardware, using single-instruction-multiple-data instructions and unsafe code foo direct access to memory where other techniques are not possible in the .Net managed environment.
ResearchGate has not been able to resolve any references for this publication.