Arjun Nitin Bhagoji's research while affiliated with University of Chicago and other places
What is this page?
This page lists the scientific contributions of an author, who either does not have a ResearchGate profile, or has not yet added these contributions to their profile.
It was automatically created by ResearchGate to create a record of this author's body of work. We create such pages to advance our goal of creating and maintaining the most comprehensive scientific repository possible. In doing so, we process publicly available (personal) data relating to the author as a member of the scientific community.
If you're a ResearchGate member, you can follow this page to keep up with this author's work.
If you are this author, and you don't want us to display this page anymore, please let us know.
It was automatically created by ResearchGate to create a record of this author's body of work. We create such pages to advance our goal of creating and maintaining the most comprehensive scientific repository possible. In doing so, we process publicly available (personal) data relating to the author as a member of the scientific community.
If you're a ResearchGate member, you can follow this page to keep up with this author's work.
If you are this author, and you don't want us to display this page anymore, please let us know.
Publications (28)
Finding classifiers robust to adversarial examples is critical for their safe deployment. Determining the robustness of the best possible classifier under a given threat model for a given data distribution and comparing it to that achieved by state-of-the-art training methods is thus an important diagnostic tool. In this paper, we find achievable i...
The proliferation of global censorship has led to the development of a plethora of measurement platforms to monitor and expose it. Censorship of the domain name system (DNS) is a key mechanism used across different countries. It is currently detected by applying heuristics to samples of DNS queries and responses (probes) for specific destinations....
Extensive literature on backdoor poison attacks has studied attacks and defenses for backdoors using "digital trigger patterns." In contrast, "physical backdoors" use physical objects as triggers, have only recently been identified, and are qualitatively different enough to resist all defenses targeting digital trigger backdoors. Research on physic...
Representation learning, i.e. the generation of representations useful for downstream applications, is a task of fundamental importance that underlies much of the success of deep neural networks (DNNs). Recently, robustness to adversarial examples has emerged as a desirable property for DNNs, spurring the development of robust training methods that...
Backdoors are powerful attacks against deep neural networks (DNNs). By poisoning training data, attackers can inject hidden rules (backdoors) into DNNs, which only activate on inputs containing attack-specific triggers. While existing work has studied backdoor attacks on a variety of DNN models, they only consider static models, which remain unchan...
Federated learning is inherently vulnerable to model poisoning attacks because its decentralized nature allows attackers to participate with compromised devices. In model poisoning attacks, the attacker reduces the model's performance on targeted sub-tasks (e.g. classifying planes as birds) by uploading "poisoned" updates. In this report we introdu...
In adversarial machine learning, new defenses against attacks on deep learning systems are routinely broken soon after their release by more powerful attacks. In this context, forensic tools can offer a valuable complement to existing defenses, by tracing back a successful attack to its root cause, and offering a path forward for mitigation to prev...
Understanding the fundamental limits of robust supervised learning has emerged as a problem of immense interest, from both practical and theoretical standpoints. In particular, it is critical to determine classifier-agnostic bounds on the training loss to establish when learning is possible. In this paper, we determine optimal lower bounds on the c...
Anonymity systems like Tor are vulnerable to Website Fingerprinting (WF) attacks, where a local passive eavesdropper infers the victim's activity. Current WF attacks based on deep learning classifiers have successfully overcome numerous proposed defenses. While recent defenses leveraging adversarial examples offer promise, these adversarial example...
The term Federated Learning was coined as recently as 2016 to describe a machine learning setting where multiple entities collaborate in solving a machine learning problem, under the coordination of a central server or service provider. Each client’s raw data is stored locally and not exchanged or transferred; instead, focused updates intended for...
Open-world machine learning (ML) combines closed-world models trained on in-distribution data with out-of-distribution (OOD) detectors, which aim to detect and reject OOD inputs. Previous works on open-world ML systems usually fail to test their reliability under diverse, and possibly adversarial conditions. Therefore, in this paper, we seek to und...
Localized adversarial patches aim to induce misclassification in machine learning models by arbitrarily modifying pixels within a restricted region of an image. Such attacks can be realized in the physical world by attaching the adversarial patch to the object to be misclassified. In this paper, we propose a general defense framework that can achie...
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g. service provider), while keeping the training data decentralized. FL embodies the principles of focused data collection and minimization, and can mitiga...
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g. service provider), while keeping the training data decentralized. FL embodies the principles of focused data collection and minimization, and can mitiga...
When deploying machine learning models in real-world applications, an open-world learning framework is needed to deal with both normal in-distribution inputs and undesired out-of-distribution (OOD) inputs. Open-world learning frameworks include OOD detectors that aim to discard input examples which are not from the same distribution as the training...
While progress has been made in understanding the robustness of machine learning classifiers to test-time adversaries (evasion attacks), fundamental questions remain unresolved. In this paper, we use optimal transport to characterize the minimum possible loss in an adversarial classification scenario. In this setting, an adversary receives a random...
A large body of recent work has investigated the phenomenon of evasion attacks using adversarial examples for deep learning systems, where the addition of norm-bounded perturbations to the test inputs leads to incorrect output classification. Previous work has investigated this phenomenon in closed-world systems where training and test inputs follo...
Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server. In this work, we explore the threat of model poisoning attacks on federated learning initiated by a single, non-colluding...
Deep neural networks (DNNs) have enabled success in learning tasks such as image classification, semantic image segmentation and steering angle prediction which can be key components of the computer vision pipeline of safety-critical systems such as autonomous vehicles. However, previous work has demonstrated the feasibility of using physical adver...
Existing black-box attacks on deep neural networks (DNNs) have largely focused on transferability, where an adversarial instance generated for a locally trained model can “transfer” to attack other learning models. In this paper, we propose novel Gradient Estimation black-box attacks for adversaries with query access to the target model’s class pro...
The existence of evasion attacks during the test phase of machine learning algorithms represents a significant challenge to both their deployment and understanding. These attacks can be carried out by adding imperceptible perturbations to inputs to generate adversarial examples and finding effective defenses and detectors has proven to be difficult...
Sign recognition is an integral part of autonomous cars. Any misclassification of traffic signs can potentially lead to a multitude of disastrous consequences, ranging from a life-threatening accident to a large-scale interruption of transportation services relying on autonomous cars. In this paper, we propose and examine realistic security attacks...
We propose a new real-world attack against the computer vision based systems of autonomous vehicles (AVs). Our novel Sign Embedding attack exploits the concept of adversarial examples to modify innocuous signs and advertisements in the environment such that they are classified as the adversary's desired traffic sign with high confidence. Our attack...
Existing black-box attacks on deep neural networks (DNNs) so far have largely focused on transferability, where an adversarial instance generated for a locally trained model can "transfer" to attack other learning models. In this paper, we propose novel Gradient Estimation black-box attacks for adversaries with query access to the target model's cl...
We propose the use of dimensionality reduction as a defense against evasion attacks on ML classifiers. We present and investigate a strategy for incorporating dimensionality reduction via Principal Component Analysis to enhance the resilience of machine learning, targeting both the classification and the training phase. We empirically evaluate and...
Citations
... For instance, [92] shows how to obtain parameters from remote classifiers with partial knowledge of models. Several studies on adversarial ML have been conducted in other domains, such as network systems and website fingerprinting [95], [96], [97], [98], [99]. Nonetheless, we show that the hyperparameters of MTDs and details about their behavior can be stolen in a black-box setting. ...
... We are currently working towards disseminating EITR to a larger user-base and using it to classify additional sensitive and nonsensitive types of content, including categories defined by the users themselves, for various purposes, not necessarily related to sensitive content classification. In the above context we are also considering additional attacks, such as physical backdoor attack [54], sub-population data poisoning attack [28] and category inference attack [20] in FL.Overall, we believe that our subjective-logic based reputation scheme maybe beneficial to other types of machine learning applications going beyond classification but this is something to be demonstrated by future work. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/1101147196665858-1639545358768_Q64/Josephine-Passananti.jpg)
... Data distribution across individual sites can impact the training convergence and final accuracy of the federated model [14,23]. Thus, we explore the influence of IID data, or lack thereof, on FL training. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/278716732198940-1443462655916_Q64/Peter-Kairouz-2.jpg)
... In this paper, we use P2P FL with secure average computation (SAC) [10] to create a distributed anomaly detection mechanism that can be applied to O-RAN architecture. P2P FL eliminates the single point of failure, and the parameters are not required to be transmitted to a centralized cloud [11]. In addition to that, we consider two variations of the P2P FL, one being a clustered P2P FL model where each cluster maintains a separate FL model, which is desired when the data is localized as in RAN. ...
... Therefore, they fail to provide verified guarantees for their effectiveness. In fact, many OOD detection systems are susceptible to adversarial attacks Sehwag et al. [2019], Chen et al. [2022]. Meinke et al. [2021] show how to verify robustness to adversarial examples around given input samples. ...
Reference: Provably Bounding Neural Network Preimages
![[object Object]](https://i1.rgstatic.net/ii/profile.image/438404140802052-1481535100718_Q64/Vikash-Sehwag.jpg)
![[object Object]](https://i1.rgstatic.net/ii/profile.image/648690381381633-1531671247204_Q64/Chawin-Sitawarin.jpg)
... Localized patch attacks against object detection [36], [37] and semantic segmentation models [39] as well as trainingtime poisoning attacks using localized triggers [40], [41] have been proposed. Our threat model in this paper focuses on attacks against classification models at test time, but we expect our defense can be generalized to the above settings as well. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/503868430536704-1497143003939_Q64/Arsalan-Mosenia.jpg)
... Since natural images have the greatest variance in low frequencies, the distance of any data point to a class boundary are longest along low frequency dimensions, and thus a model that mainly uses these low frequencies should be the more robust. Previous work also found that this kind of PCA data preprocessing increases robustness to adversarial attacks [25]. ...
... Similarly, self-driving cars employ ML to navigate traffic without the need for human involvement. A mistaken decision for the autonomous vehicle based on a adversarial attack could result in a tragic accident [7,8]. Hence, defending against malicious attacks and boosting the robustness of ML models without sacrificing clean accuracy is critical. ...
... In [13], researchers were successful in manipulating Tesla's autopilot system by using split-second light projections on roads. Sensor jamming, blinding, spoofing, distributed denial-of-service (DDoS) attacks, manipulation of communication devices, and adversarial ML techniques on AI-driven algorithms are some of these attack methods demonstrated by researchers in this area [14]- [19]. These kinds of cyberphysical attacks are specifically targeted, posing potential threats to AVs and objects in ODD. ...
... Apart from these are other common attacks are, Integrity attack-A efficient resource assault that is identified as normal traffic due to false negatives [134]. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/964239309144073-1606903974531_Q64/Bo-Li-318.jpg)
