Martin Arlitt’s research while affiliated with University of Calgary and other places

The cost of errors related to machine learning classifiers, namely, false positives and false negatives, are not equal and are application dependent. For example, in cybersecurity applications, the cost of not detecting an attack is very different from marking a benign activity as an attack. Various design choices during machine learning model building, such as hyperparameter tuning and model selection, allow a data scientist to trade-off between these two errors. However, most of the commonly used metrics to evaluate model quality, such as $F_1$ score, which is defined in terms of model precision and recall, treat both these errors equally, making it difficult for users to optimize for the actual cost of these errors. In this paper, we propose a new cost-aware metric, $C_{score}$ based on precision and recall that can replace $F_1$ score for model evaluation and selection. It includes a cost ratio that takes into account the differing costs of handling false positives and false negatives. We derive and characterize the new cost metric, and compare it to $F_1$ score. Further, we use this metric for model thresholding for five cybersecurity related datasets for multiple cost ratios. The results show an average cost savings of 49%.

Every time we use the web, we place our trust in X.509 certificates binding public keys to domain identities. However, for these certificates to be trustworthy, proper issuance, management, and timely revocations (in cases of compromise or misuse) are required. While great efforts have been placed on ensuring trustworthiness in the issuance of new certificates, there has been a scarcity of empirical studies on revocation management. This study offers the first comprehensive analysis of certificate replacements (CRs) of revoked certificates. It provides a head-to-head comparison of the CRs where the replaced certificate was revoked versus not revoked. Leveraging two existing datasets with overlapping timelines, we create a combined dataset containing 1.5 million CRs that we use to unveil valuable insights into the effect of revocations on certificate management. Two key questions guide our research: (1) the influence of revocations on certificate replacement behavior and (2) the effectiveness of revocations in fulfilling their intended purpose. Our statistical analysis reveals significant variations in revocation rates, retention rates, and post-revocation usage, shedding light on differences in Certificate Authorities’ (CAs) practices and subscribers’ decisions. Notably, a substantial percentage of revoked certificates were either observed or estimated to be used after revocation, raising concerns about key-compromise instances. Finally, our findings highlight shortcomings in existing revocation protocols and practices, emphasizing the need for improvements. We discuss ongoing efforts and potential solutions to address these issues, offering valuable guidance for enhancing the security and integrity of web communications.

The decentralized nature of Bitcoin allows for pseudonymous money exchange beyond authorities’ control, contributing to its popularity for diverse illegal activities such as scams, ransomware attacks, money laundering, and black markets. In this paper, we characterize this landscape, providing insights into similarities and differences in the use of Bitcoin for such activities. Our analysis and the derived insights contribute to the understanding of Bitcoin transactions associated with illegal activities through three main aspects. First, our study offers a comprehensive characterization of money flows to and from Bitcoin addresses linked to different abuse categories, revealing variations in flow patterns and success rates. Second, our temporal analysis captures long-term trends and weekly patterns across categories. Finally, our analysis of outflow from reported addresses uncovers differences in graph properties and flow patterns among illicit addresses and between abuse categories. These findings provide valuable insights into the distribution, temporal dynamics, and interconnections within various categories of Bitcoin transactions related to illicit activities. The increased understanding of this landscape and the insights gained from this study offer important empirical guidance for informed decision-making and policy development in the ongoing effort to address the challenges presented by illicit activities within the cryptocurrency space.

On April 1, 2023 we stopped monitoring the traffic on our campus Internet link, nearly 20 years to the day since we first started doing so. During these two decades, we faced a vast array of issues that affected the collection, storage, analysis and backup of our monitoring data. In this paper we share some of our experiences, so that future networking researchers have an opportunity to learn from our successes as well as our many mistakes and misfortunes.

The containerized microservices architecture is being increasingly used to build complex applications. To minimize operating costs, service providers typically rely on an auto-scaler to "right size" their infrastructure amid fluctuating workloads. The agile nature of microservice development and deployment requires an auto-scaler that does not require significant effort to derive resource allocation decisions. In this paper, we investigate reducing auto-scaler development effort along a number of dimensions. First, we focus on a technique that does not require an expert to develop a model, e.g., a queuing model or machine learning model, of the system and tweak the model as the underlying microservice application changes. Second, we explore ways to limit the number of workload patterns that need to be considered. Third, we study techniques to reduce the number of resource allocation scenarios that one has to explore before deploying the auto-scaler. To address these goals, we first analyze the workload of 24,000 real microservice applications and find that a small number of workload patterns dominate for any given application. These results suggest that auto-scaler design can be driven by this small subset of popular workload patterns thereby limiting effort. To limit the number of resource allocation scenarios explored, we develop a novel heuristic optimization technique called MOAT, which outperforms Bayesian Optimization often used for such exercises. We combine insights obtained from real microservice workloads and MOAT to realize an auto-scaler called TRIM that requires no system modeling. For each popular workload pattern identified for an application, TRIM uses MOAT to pre-compute a near minimal resource allocation that satisfies end user response time targets. These resource allocations are then used at runtime when appropriate. We validate our approach using a variety of analytical, on-premise, and public cloud systems. From our results, TRIM in consort with MOAT significantly improves the performance of the industry-standard HPA auto-scaler by achieving up to 92% fewer response time violations and up to 34% lower costs compared to using HPA in isolation.