Vitaly Shmatikov's research while affiliated with Cornell University and other places

Publications (142)

Preprint
Full-text available
Today, creators of data-hungry deep neural networks (DNNs) scour the Internet for training fodder, leaving users with little control over or knowledge of when their data is appropriated for model training. To empower users to counteract unwanted data use, we design, implement and evaluate a practical system that enables users to detect if their dat...
Preprint
We investigate a new threat to neural sequence-to-sequence (seq2seq) models: training-time attacks that cause models to "spin" their outputs so as to support an adversary-chosen sentiment or point of view, but only when the input contains adversary-chosen trigger words. For example, a spinned summarization model would output positive summaries of a...
Preprint
We investigate a new threat to neural sequence-to-sequence (seq2seq) models: training-time attacks that cause models to "spin" their output and support a certain sentiment when the input contains adversary-chosen trigger words. For example, a summarization model will output positive summaries of any text that mentions the name of some individual or...
Preprint
Full-text available
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and demonstrate that state-of-the-art models for many tasks which rely on analyzing the meaning and similarity of texts-- including paraphrase identification, document retri...
Preprint
Code autocompletion is an integral feature of modern code editors and IDEs. The latest generation of autocompleters uses neural language models, trained on public open-source code repositories, to suggest likely (not just statically feasible) completions given the current context. We demonstrate that neural code autocompleters are vulnerable to dat...
Preprint
Components of machine learning systems are not (yet) perceived as security hotspots. Secure coding practices, such as ensuring that no execution paths depend on confidential inputs, have not yet been adopted by ML developers. We initiate the study of code security of ML systems by investigating how nucleus sampling---a popular approach for generati...
Preprint
We investigate a new method for injecting backdoors into machine learning models, based on poisoning the loss computation in the model-training code. Our attack is blind: the attacker cannot modify the training data, nor observe the execution of his code, nor access the resulting model. We develop a new technique for blind backdoor training using m...
Preprint
Federated learning (FL) is a heavily promoted approach for training ML models on sensitive data, e.g., text typed by users on their smartphones. FL is expressly designed for training on data that are unbalanced and non-iid across the participants. To ensure privacy and integrity of the federated model, latest FL approaches use differential privacy...
Preprint
Word embeddings, i.e., low-dimensional vector representations such as GloVe and SGNS, encode word "meaning" in the sense that distances between words' vectors correspond to their semantic proximity. This enables transfer learning of semantics for a variety of natural language processing tasks. Word embeddings are typically trained on large public c...
Conference Paper
To help enforce data-protection regulations such as GDPR and detect unauthorized uses of personal data, we develop a new model auditing technique that helps users check if their data was used to train a machine learning model. We focus on auditing deep-learning models that generate natural-language text, including word prediction and dialog generat...
Preprint
Differential privacy (DP) is a popular mechanism for training machine learning models with bounded leakage about the presence of specific points in the training data. The cost of differential privacy is a reduction in the model's accuracy. We demonstrate that this cost is not borne equally: accuracy of DP models drops much more for the underreprese...
Preprint
Overlearning' means that a model trained for a seemingly simple objective implicitly learns to recognize attributes that are (1) statistically uncorrelated with the objective, and (2) sensitive from a privacy or bias perspective. For example, a binary gender classifier of facial images also learns to recognize races\textemdash even races that are n...
Conference Paper
Collaborative machine learning and related techniques such as federated learning allow multiple participants, each with his own training dataset, to build a joint model by training locally and periodically exchanging model updates. We demonstrate that these updates leak unintended information about participants’ training data and develop passive an...
Preprint
To help enforce data-protection regulations such as GDPR and detect unauthorized uses of personal data, we propose a new \emph{model auditing} technique that enables users to check if their data was used to train a machine learning model. We focus on auditing deep-learning models that generate natural-language text, including word prediction and di...
Conference Paper
Access control in the Internet of Things (IoT) often depends on a situation --- for example, "the user is at home'' --- that can only be tracked using multiple devices. In contrast to the (well-studied) smartphone frameworks, enforcement of situational constraints in the IoT poses new challenges because access control is fundamentally decentralized...
Preprint
Federated learning enables multiple participants to jointly construct a deep learning model without sharing their private training data with each other. For example, multiple smartphones can jointly train a predictive keyboard model without revealing what individual users type into their phones. We demonstrate that any participant in federated lear...
Article
To protect database confidentiality even in the face of full compromise while supporting standard functionality, recent academic proposals and commercial products rely on a mix of encryption schemes. The recommendation is to apply strong, semantically secure encryption to the "sensitive" columns and protect other columns with property-revealing enc...
Preprint
Full-text available
Collaborative machine learning and related techniques such as distributed and federated learning allow multiple participants, each with his own training dataset, to build a joint model. Participants train local models and periodically exchange model parameters or gradient updates computed during the training. We demonstrate that the training data u...
Article
Major cloud operators offer machine learning (ML) as a service, enabling customers who have the data but not ML expertise or infrastructure to train predictive models on this data. Existing ML-as-a-service platforms require users to reveal all training data to the service operator. We design, implement, and evaluate Chiron, a system for privacy-pre...
Article
We demonstrate that state-of-the-art optical character recognition (OCR) based on deep learning is vulnerable to adversarial images. Minor modifications to images of printed text, which do not change the meaning of the text to a human reader, cause the OCR system to "recognize" a different text where certain words chosen by the adversary are replac...
Conference Paper
Machine learning (ML) is becoming a commodity. Numerous ML frameworks and services are available to data holders who are not ML experts but want to train predictive models on their data. It is important that ML models trained on sensitive inputs (e.g., personal images or documents) not leak too much information about the training data. We consider...
Article
Machine learning (ML) is becoming a commodity. Numerous ML frameworks and services are available to data holders who are not ML experts but want to train predictive models on their data. It is important that ML models trained on sensitive inputs (e.g., personal images or documents) not leak too much information about the training data. We consider...
Conference Paper
Full-text available
We quantitatively investigate how machine learning models leak information about the individual data records on which they were trained. We focus on the basic membership inference attack: given a data record and black-box access to a model, determine if the record was in the model’s training dataset. To perform membership inference against a target...
Conference Paper
Encrypted databases, a popular approach to protecting data from compromised database management systems (DBMS's), use abstract threat models that capture neither realistic databases, nor realistic attack scenarios. In particular, the "snapshot attacker" model used to support the security claims for many encrypted databases does not reflect the info...
Conference Paper
In a sensor spoofing attack, an adversary modifies the physical environment in a certain way so as to force an embedded system into unwanted or unintended behaviors. This usually requires a thorough understanding of the system's control logic. The conventional methods for discovering this logic are manual code inspection and experimentation. In thi...
Article
Modern mobile apps communicate and exchange data with other apps almost as much as they communicate and exchange data with the operating system. Many popular apps now occupy essential places in the app "ecosystem" and provide other apps with services, such as storage, that have traditionally been the responsibility of the OS. For example, an app ma...
Conference Paper
We develop a systematic approach for analyzing client-server applications that aim to hide sensitive user data from untrusted servers. We then apply it to Mylar, a framework that uses multi-key searchable encryption (MKSE) to build Web applications on top of encrypted data. We demonstrate that (1) the Popa-Zeldovich model for MKSE does not imply se...
Article
Full-text available
We demonstrate that modern image recognition methods based on artificial neural networks can recover hidden information from images protected by various forms of obfuscation. The obfuscation techniques considered in this paper are mosaicing (also known as pixelation), blurring (as used by YouTube), and P3, a recently proposed system for privacy-pre...
Article
Full-text available
We design, implement, and evaluate CovertCast, a censorship circumvention system that broadcasts the content of popular websites in real-time, encrypted video streams on common live-streaming services such as YouTube. CovertCast does not require any modifications to the streaming service and employs the same protocols, servers, and streaming softwa...
Article
Modern cloud services are designed to encourage and support collaboration. To help users share links to online documents, maps, etc., several services, including cloud storage providers such as Microsoft OneDrive and mapping services such as Google Maps, directly integrate URL shorteners that convert long, unwieldy URLs into short URLs, consisting...
Conference Paper
Full-text available
Modern mobile apps need to store and share structured data, but the coarse-grained access-control mechanisms in existing mobile operating systems are inadequate to help apps express and enforce their protection requirements. We design, implement, and evaluate a prototype of Earp, a new mobile platform that uses the relational model as the unified O...
Article
After decades of study, automatic face detection and recognition systems are now accurate and widespread. Naturally, this means users who wish to avoid automatic recognition are becoming less able to do so. Where do we stand in this cat-and-mouse race? We currently live in a society where everyone carries a camera in their pocket. Many people willf...
Conference Paper
Full-text available
Deep learning based on artificial neural networks is a very popular approach to modeling, classifying, and recognizing complex data such as images, speech, and text. The unprecedented accuracy of deep learning methods has turned them into the foundation of new AI-based services on the Internet. Commercial companies that collect user data on a large...
Conference Paper
Deep learning based on artificial neural networks is a very popular approach to modeling, classifying, and recognizing complex data such as images, speech, and text. The unprecedented accuracy of deep learning methods has turned them into the foundation of new AI-based services on the Internet. Commercial companies that col- lect user data on a lar...
Conference Paper
Many modern desktop and mobile platforms, including Ubuntu, Google Chrome, Windows, and Firefox OS, support so called Web-based system applications that run outside the Web browser and enjoy direct access to native objects such as files, camera, and geolocation. We show that the access-control models of these platforms are (a) incompatible and (b)...
Conference Paper
Augmented reality (AR) browsers are an emerging category of mobile applications that add interactive virtual objects to the user's view of the physical world. This paper gives the first system-level evaluation of their security and privacy properties. We start by analyzing the functional requirements that AR browsers must support in order to presen...
Article
Modern network security rests on the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protocols. Distributed systems, mobile and desktop applications, embedded devices, and all of secure Web rely on SSL/TLS for protection against network attacks. This protection critically depends on whether SSL/TLS clients correctly validate X.509 cer...
Conference Paper
Censorship circumvention systems such as Tor are highly vulnerable to network-level filtering. Because the traffic generated by these systems is disjoint from normal network traffic, it is easy to recognize and block, and once the censors identify network servers (e.g., Tor bridges) assisting in circumvention, they can locate all of their users. Cl...
Conference Paper
Hybrid mobile applications (apps) combine the features of Web applications and "native" mobile apps. Like Web applications, they are implemented in portable, platform-independent languages such as HTML and JavaScript. Like native apps, they have direct access to local device resources-file system, location, camera, contacts, etc. Hybrid apps are ty...
Conference Paper
Code injection attacks continue to plague applications that incorporate user input into executable programs. For example, SQL injection vulnerabilities rank fourth among all bugs reported in CVE, yet all previously proposed methods for detecting SQL injection attacks suffer from false positives and false negatives. This paper describes the design a...
Conference Paper
Genome-wide association studies (GWAS) have become a popular method for analyzing sets of DNA sequences in order to discover the genetic basis of disease. Unfortunately, statistics published as the result of GWAS can be used to identify individuals participating in the study. To prevent privacy breaches, even previously published results have been...
Conference Paper
In response to the growing popularity of Tor and other censorship circumvention systems, censors in non-democratic countries have increased their technical capabilities and can now recognize and block network traffic generated by these systems on a nationwide scale. New censorship-resistant communication systems such as Skype Morph, Stego Torus, an...
Conference Paper
Perceptual, "context-aware" applications that observe their environment and interact with users via cameras and other sensors are becoming ubiquitous on personal computers, mobile phones, gaming platforms, household robots, and augmented-reality devices. This raises new privacy risks. We describe the design and implementation of DARKLY, a practical...
Conference Paper
We present πBox, a new application platform that prevents apps from misusing information about their users. To strike a useful balance between users' privacy and apps' functional needs, πBox shifts much of the responsibility for protecting privacy from the app and its users to the platform itself. To achieve this, πBox deploys (1) a sandbox that sp...
Conference Paper
SSL (Secure Sockets Layer) is the de facto standard for secure Internet communications. Security of SSL connections against an active network attacker depends on correctly validating public-key certificates presented when the connection is established. We demonstrate that SSL certificate validation is completely broken in many security-critical app...
Article
Even experienced developers struggle to implement security policies correctly. For example, despite 15 years of development, standard Java libraries still suffer from missing and incorrectly applied permission checks, which enable untrusted applications to execute native calls or modify private class variables without authorization. Previous techni...
Article
We describe a new side-channel attack. By tracking changes in the application's memory footprint, a concurrent process belonging to a different user can learn its secrets. Using Web browsers as the target, we show how an unprivileged, local attack process -- for example, a malicious Android app -- can infer which page the user is browsing, as well...
Article
We systematically describe two classes of evasion exploits against automated malware detectors. Chameleon attacks confuse the detectors' file-type inference heuristics, while werewolf attacks exploit discrepancies in format-specific file parsing between the detectors and actual operating systems and applications. These attacks do not rely on obfusc...
Article
Modern systems keep long memories. As we show in this paper, an adversary who gains access to a Linux system, even one that implements secure deallocation, can recover the contents of applications' windows, audio buffers, and data remaining in device drivers-long after the applications have terminated. We design and implement Lacuna, a system that...
Conference Paper
Web applications written in languages such as PHP and JSP are notoriously vulnerable to accidentally omitted authorization checks and other security bugs. Existing techniques that find missing security checks in library and system code assume that (1) security checks can be recognized syntactically and (2) the same pattern of checks applies univers...
Conference Paper
Web applications written in languages such as PHP and JSP are notoriously vulnerable to accidentally omitted authorization checks and other security bugs. Existing techniques that find missing security checks in library and system code assume that (1) security checks can be recognized syntactically and (2) the same pattern of checks applies univers...
Conference Paper
Inter-domain routing in today's Internet is plagued by security and reliability issues (e.g., prefix hijacking), which are often caused by malicious or Byzantine misbehavior. We argue that route selection policies must move beyond static preferences that select routes on the basis of static attributes such as route length or which neighboring AS is...
Conference Paper
Full-text available
TxBox is a new system for sand boxing untrusted applications. It speculatively executes the application in a system transaction, allowing security checks to be parallelized and yielding significant performance gains for techniques such as on-access anti-virus scanning. TxBox is not vulnerable to TOCTTOU attacks and incorrect mirroring of kernel sta...
Conference Paper
Many commercial websites use recommender systems to help customers locate products and content. Modern recommenders are based on collaborative filtering: they use patterns learned from users' behavior to make recommendations, usually in the form of related-items lists. The scale and complexity of these systems, along with the fact that their output...
Conference Paper
We present a new approach to verifying that a completely untrusted, platform-as-a-service cloud is correctly executing an outsourced Web application.
Article
Web applications are vulnerable to semantic attacks such as denial of service due to infinite loops caused by malicious inputs and unauthorized database operations due to missing security checks. Unlike "conventional" threats such as SQL injection and cross-site scripting, these attacks exploit bugs in the logic of the vulnerable application and ca...
Article
Even experienced developers struggle to implement security policies correctly. For example, despite 15 years of development, standard Java libraries still suffer from missing and incorrectly applied permission checks, which enable untrusted applications to execute native calls or modify private class variables without authorization. Previous techni...
Conference Paper
Even experienced developers struggle to implement security policies correctly. For example, despite 15 years of development, standard Java libraries still suffer from missing and incorrectly applied permission checks, which enable untrusted applications to execute native calls or modify private class variables without authorization. Previous techni...
Article
The Communications Web site, http://cacm.acm.org, features more than a dozen bloggers in the BLOG@CACM community. In each issue of Communications, we'll publish selected posts or excerpts.twitterFollow us on Twitter ...
Article
This report describes the IARPA sponsored private information retrieval (PIR) project. The approach is based on the keyword-oblivious transfer cryptographic primitive, which allows a client and server to negotiate an exchange of data based on a keyword not learned by the server. Although no protocols exist that allow this primitive to scale to the...
Conference Paper
DNS cache poisoning is a serious threat to today’s Internet. We develop a formal model of the semantics of DNS caches, including the bailiwick rule and trust-level logic, and use it to systematically investigate different types of cache poisoning and to generate templates for attack payloads. We explain the impact of the attacks on DNS resolvers su...
Conference Paper
Full-text available
We present Airavat, a MapReduce-based system which provides strong security and privacy guarantees for distributed computations on sensitive data. Airavat is a novel integration of mandatory access control and differential privacy. Data providers control the security policy for their sensitive data, including a mathematical bound on potential priva...
Article
Software developers are increasingly choosing memory-safe lan-guages. As a result, semantic vulnerabilities—omitted security checks, misconfigured security policies, and other software design errors—are supplanting memory-corruption exploits as the primary cause of security violations. Semantic attacks are difficult to de-tect because they violate...
Article
Developing effective privacy protection technologies is a critical challenge for security and privacy research as the amount and variety of data collected about individuals increase exponentially.
Conference Paper
Full-text available
As networked systems grow in complexity, they are increasingly vulnerable to denial-of-service (DoS) attacks involving resource exhaustion. A single malicious "input of coma" can trigger high-complexity behavior such as deep recursion in a carelessly implemented server, exhausting CPU time or stack space and making the server unavailable to legitim...
Article
We consider the so called "cryptographic proto-cols" whose aim is to ensure some security properties when communication channels are not reliable. Such protocols usu-ally rely on cryptographic primitives. Even if it is assumed that the cryptographic primitives are perfect, the security goals may not be achieved: the protocol itself may have weak-ne...
Article
Operators of online social networks are increasingly sharing potentially sensitive information about users and their relationships with advertisers, application developers, and data-mining researchers. Privacy is typically protected by anonymization, i.e., removing names, addresses, etc. We present a framework for analyzing privacy and anonymity in...
Conference Paper
We present an efficient protocol for the privacy-preserving, distributed learning of decision-tree classifiers. Our protocol allows a user to construct a classifier on a database held by a remote server without learning any additional information about the records held in the database. The server does not learn anything about the constructed classi...
Conference Paper
Re-identification is a major privacy threat to public datasets containing individual records. Many privacy protection algorithms rely on generalization and suppression of "quasi-identifier" attributes such as ZIP code and birthdate. Their objective is usually syntactic sanitization: for example, k-anonymity requires that each "quasi-identifier" tup...
Conference Paper
We present a new class of statistical de- anonymization attacks against high-dimensional micro-data, such as individual preferences, recommendations, transaction records and so on. Our techniques are robust to perturbation in the data and tolerate some mistakes in the adversary's background knowledge. We apply our de-anonymization methodology to th...
Conference Paper
Full-text available
Many basic tasks in computational biology involve operations on individual DNA and protein sequences. These sequences, even when anonymized, are vulnerable to re-identification attacks and may reveal highly sensitive information about individuals. We present a relatively efficient, privacy-preserving implementation of fundamental genomic computatio...
Chapter
Obfuscation, when used as a technical term, refers to hiding information “in plain sight” inside computer code or digital data. The history of obfuscation in modern computing can be traced to two events that took place in 1976. The first was the publication of Diffie and Hellman’s seminal paper on public-key cryptography [DH76]. This paper is famou...
Conference Paper
We investigate the problem of verifying location claims of mobile devices, and propose a new property called simultaneous distance modification (SDM). In localization protocols satisfying the SDM property, a malicious device can lie about its distance from the verifiers, but all distanc es can only be altered by the same amount. We demonstrate that...
Conference Paper
Full-text available
We present an efficient protocol for privacy-preserving evaluation of diagnostic programs, represented as binary decision trees or branching programs. The protocol applies a branching diagnos- tic program with classification labels in the leaves to the user's attribute vector. The user learns only the label assigned by the program to his vector; th...
Article
Probe-response attacks are a new threat for collaborative intrusion detection systems. A probe is an attack which is deliberately crafted so that its target detects and reports it with a recognizable "fingerprint" in the report. The attacker then uses the collaborative infrastructure to learn the detector's location and defensive capabilities from...
Conference Paper
We design and evaluate a lightweight route verification mech- anism that enables a router to discover route failures and inconsistencies between advertised Internet routes and ac- tual paths taken by the data packets. Our mechanism is accurate, incrementally deployable, and secure against mali- cious intermediary routers. By carefully avoiding any...
Conference Paper
Denial of service (DoS) attacks are a growing threat to the availability of Internet services. We present dFence, a novel network-based defense system for mitigating DoS attacks. The main thesis of dFence is complete trans- parency to the existing Internet infrastructure with no software modifications at either routers, or the end hosts. dFence dyn...
Article
Full-text available
We consider the probabilistic contract signing protocol of Ben-Or, Goldreich, Micali, and Rivest as a case study in formal veri - cation of probabilistic security protocols. Using the probabilistic model checker PRISM, we analyse the probabilistic fairness guarantees the protocol is intended to provide. Our study demonstrates the diculty of combini...
Article
We present a new class of statistical de-anonymization attacks against high-dimensional micro-data, such as individual preferences, recommendations, transaction records and so on. Our techniques are robust to perturbation in the data and tolerate some mistakes in the adversary's background knowledge. We apply our de-anonymization methodology to the...
Conference Paper
Many applications of mix networks such as anonymous Web brows- ing require relationship anonymity: it should be hard for the at- tacker to determine who is communicating with whom. Conven- tional methods for measuring anonymity, however, focus on sender anonymity instead. Sender anonymity guarantees that it is difficult for the attacker to determin...
Conference Paper
Abstract Over the last several years, there has been an emerging interest in the development of widearea data collection and analysis centers to help identify, track, and formulate responses to the ever-growing number,of coordinated attacks and malware infections that plague computer,networks worldwide. As large-scale network threats continue to ev...
Conference Paper
The output of a data mining algorithm is only as good as its inputs, and individuals are often unwilling to provide accu- rate data about sensitive topics such as medical history and personal finance. Individuals may be willing to share their data, but only if they are assured that it will be used in an aggregate study and that it cannot be linked...
Article
largest online movie rental service—publicly released a dataset containing movie ratings of 500,000 Netflix subscribers. The dataset is intended to be anonymous, and all personally identifying information has been removed. We demonstrate that an attacker who knows only a little bit about an individual subscriber can easily identify this subscriber’...
Article
The transmission of voice communications as datagram packets over IP networks, commonly known as Voice-over- IP (VoIP) telephony, is rapidly gaining wide acceptance. With private phone conversations being conducted on insecure public networks, security of VoIP communications is increasingly important. We present a structured security analysis of th...
Article
We study the problem of circuit obfuscation, i.e., transforming the circuit in a way that hides everything except its input-output behavior. Barak et al. showed that a universal obfuscator that obfuscates every circuit class cannot exist, leaving open the possibility of special-purpose obfuscators. Known positive results for obfuscation are limited...
Conference Paper
Mix networks are a popular mechanism for anonymous In- ternet communications. By routing IP traffic through an overlay chain of mixes, they aim to hide the relationship between its origin and destina- tion. Using a realistic model of interactive Internet traffic, we study the problem of defending low-latency mix networks against attacks based on co...
Article
Over the last several years, there has been an emerging inter- est in the development of wide-area data collection and anal- ysis centers to help identify, track, and formulate responses to the ever-growing number of coordinated attacks and mal- ware infections that plague computer networks worldwide. As large-scale network threats continue to evol...
Conference Paper
We consider scenarios in which two parties, each in possession of a graph, wish to compute some algorithm on their joint graph in a privacy-preserving manner, that is, without leaking any information about their inputs except that revealed by the algorithm’s output. Working in the standard secure multi-party computation paradigm, we present new alg...
Conference Paper
We present a cryptographically sound formal method for proving correctness of key exchange protocols. Our main tool is a fragment of a symbolic protocol logic. We demonstrate that proofs of key agreement and key secrecy in this logic imply simulatability in Shoup's secure multi-party framework for key exchange. As part of the logic, we present cryp...
Conference Paper
We investigate whether it is possible to encrypt a database and then give it away in such a form that users can still access it, but only in a restricted way. In contrast to conventional privacy mechanisms that aim to prevent any access to individual records, we aim to restrict the set of queries that can be feasibly evaluated on the encrypted data...
Conference Paper
Human-memorable passwords are a mainstay of computer security. To decrease vulnerability of passwords to brute-force dictionary attacks, many organizations enforce complicated password-creation rules and require that passwords include numerals and special characters. We demonstrate that as long as passwords remain human-memorable, they are vulnerab...

Citations

... Although the backdoor attack has been extensively researched in multiple applications, such as computer vision (CV) [11][12][13][14][15][16][17] and natural language processing (NLP) [18][19][20][21][22][23][24][25][26][27][28][29], there is no research in the field of DGA detection. Due to the particularity of DGA , existing backdoor attacks can not be directly applied to the DGA detection approach based on deep learning. ...
... However, neural ranking models may inherit the adversarial vulnerabilities of neural networks [49], i.e, a small deliberate perturbation (e.g., some pixel variations on an image) could trigger dramatic change in the learning result [65]. Such vulnerability has raised dedicated concerns of the robustness and reliability of text ranking systems integrated with neural networks [46]. A trustworthy ranking system should be well aware of the malicious attack, in which deliberate but imperceptible content variations may cause the catastrophic ranking disorder. ...
... For example, the ability of machine learning algorithms, as tools that can assist people in making decisions for large projects, to defend against various attacks determines the future of machine learning algorithms. In addition to model attacks that are effective on specific models [20], attackers can cause poisoning attacks by manipulating or maliciously injecting anomalous data during training [21]. Video recognition [22], communication networks [23], malware detection [24], edge computing [25], and domain name resolution [16] have demonstrated actual poisoning attacks. ...
... completely corrupt the FL models. Focusing on breaching the confidentiality of FL models, inference attacks [39,40] and reconstruction attacks [60] could significantly increase the privacy risk of FL models. To eliminate the above threats, many defense methods [6,23,44] have been proposed and achieved outstanding defense performance against these attacks. ...
... The works in [24,57] study developing MIAs specifically for machine translation models. However, neither of the two existing works assume full confidence-score observability because they both aim to design practical MIAs with minimal side-information assumptions. ...
... It enables a more refined expression of situations where access can be granted. Our technique leverages lessons from Schuster et al. [24], which implemented a context server called Environmental Situational Oracle (ESO). An ESO encapsulates the implementation of how a situation is sensed, inferred, or actuated [24]. ...
... Therefore, the cryptography community proposed a relaxed notation called indistinguishability under ordered chosen-plaintext attack (IND-OCPA) [8]. However, it was shown that [5] effective attacks can be launched on IND-OCPA security caused by the access patterns. The root cause of this issue lies in the deterministic ciphertext in early-stage order-preserving encryption schemes. ...
... Therefore, we are naturally encouraged to instantiate PoUL with the TEEs-backed offerings, and thereby enable native implementation and practical deployment on the server side. Also, we admit that recent TEEs-empowered ML works [75], [48], [38], [74], [54], [41], [42], [86], [88], [53], [39], [11] may help mitigate a certain issue within our PoUL, but we need new and holistic designs tailored for proving end-to-end RTBF compliance in MLaaS. Our unlearning setting. ...