Pro .NET Benchmarking: The Art of Performance Measurement
Abstract
Use this in-depth guide to correctly design benchmarks, measure key performance metrics of .NET applications, and analyze results. This book presents dozens of case studies to help you understand complicated benchmarking topics. You will avoid common pitfalls, control the accuracy of your measurements, and improve performance of your software.
Author Andrey Akinshin has maintained BenchmarkDotNet (the most popular .NET library for benchmarking) for five years and covers common mistakes that developers usually make in their benchmarks. This book includes not only .NET-specific content but also essential knowledge about performance measurements which can be applied to any language or platform (common benchmarking methodology, statistics, and low-level features of modern hardware).
What You'll Learn:
• Be aware of the best practices for writing benchmarks and performance tests
• Avoid the common benchmarking pitfalls
• Know the hardware and software factors that affect application performance
• Analyze performance measurements
This book is for .NET developers concerned with the performance of their applications.
Andrey Akinshin, PhD is a .NET MVP, ACM ICPC medalist, project lead of BenchmarkDotNet, program director of the DotNext conference, and software developer at JetBrains where he works on Rider.
... However, the benchmarks are not safe from external factors like environmental temperature, mechanical vibrations, or noise from other OS processes. According to Akinshin the natural noise can make up 5%-20% of the measured test duration [47]. Executing the benchmarks multiple times helps generating a more stable result. ...
... 11 API application programming interface. 44,47 CCU concurrent users. 8,9,16,26,43 CPU central processing unit. ...
The broad coverage of 4G networks and the introduction of 5G networks result in
new opportunities for multiplayer mobile games. Unity, the most-used game engine
for mobile applications, offers multiple solutions to develop a server application inside
the engine. However, this thesis evaluates the benefits of developing a separate .NET
server application for a Unity client. Three different server targets are used to compare
operating systems, networking libraries, and serialization libraries. In addition,
a game was developed alongside the benchmarks to test security aspects and run the
libraries in a real-world environment.
The combination of a .NET server with the mobile network infrastructure of Europe
allows for round-trip times of less than 100 milliseconds while using an inexpensive
server solution. Manual serialization yielded the best performance results while
keeping packet size at a minimum and was especially used for packets that are frequently
sent. For messages that can change or are not sent as often, libraries like
MessagePack-CSharp or Protocol Buffers are interesting alternatives. In the networking
benchmarks, ENet performed best and had the most promising metrics for
weaker hardware setups. For all tests, the User Datagram Protocol (UDP) socket
performance was better on Linux than on Windows.
Performance regressions can have a drastic impact on the usability of a
software application. The crucial task of localizing such regressions can be achieved
using bisection, which attempts to find the bug-introducing commit using binary
search. This approach is used extensively by many development teams, but it is
an inherently heuristical approach when applied to performance regressions, and
therefore, does not have correctness guarantees. Unfortunately, bisection is also
time-consuming, which implies the need to assess its effectiveness prior to running
it. To this end, the goal of this study is to analyze the effectiveness of bisection
for performance regressions. This goal is achieved by first formulating a metric
that quantifies the probability of a successful bisection, and extracting a list of
input parameters – the contributing properties – that potentially impact its value; a
sensitivity analysis is then conducted on these properties to understand the extent
of their impact. Furthermore, an empirical study of 310 bug reports describing
performance regressions in 17 real-world applications is conducted, to better understand what these contributing properties look like in practice. The results show
that while bisection can be highly effective in localizing real-world performance
regressions, this effectiveness is sensitive to the contributing properties, especially
the choice of baseline and the distributions at each commit. The results also reveal that most bug reports do not provide sufficient information to help developers
properly choose values and metrics that can maximize the effectiveness, which implies the need for measures to fill this information gap.
In this paper we improve the efficiency of the simple matrix-multiplication algorithm using parallelism and hardware instrinsics with C# and .Net Task Parallel Library. We demonstrate how additional processing can result the overall shorter execution times. We present an algorithm in C# that achieves up to five times performance improvement on midrange hardware, using single-instruction-multiple-data instructions and unsafe code foo direct access to memory where other techniques are not possible in the .Net managed environment.
ResearchGate has not been able to resolve any references for this publication.