Olga Ohrimenko

NeurIPS Conference 2025 Conference Paper

AdaptDel: Adaptable Deletion Rate Randomized Smoothing for Certified Robustness

• Zhuoqun Huang
• Neil Marchant
• Olga Ohrimenko
• Benjamin Rubinstein

We consider the problem of certified robustness for sequence classification against edit distance perturbations. Naturally occurring inputs of varying lengths (e. g. , sentences in natural language processing tasks) present a challenge to current methods that employ fixed-rate deletion mechanisms and lead to suboptimal performance. To this end, we introduce AdaptDel methods with adaptable deletion rates that dynamically adjust based on input properties. We extend the theoretical framework of randomized smoothing to variable-rate deletion, ensuring sound certification with respect to edit distance. We achieve strong empirical results in natural language tasks, observing up to 30 orders of magnitude improvement to median cardinality of the certified region, over state-of-the-art certifications.

TMLR Journal 2025 Journal Article

RS-Reg: Probabilistic and Robust Certified Regression through Randomized Smoothing

• Aref Miri Rekavandi
• Olga Ohrimenko
• Benjamin I. P. Rubinstein

Randomized smoothing has shown promising certified robustness against adversaries in classification tasks. Despite such success with only zeroth-order access to base models, randomized smoothing has not been extended to a general form of regression. By defining robustness in regression tasks flexibly through probabilities, we demonstrate how to establish upper bounds on input data point perturbation (using the $\ell_2$ norm) for a user-specified probability of observing valid outputs. Furthermore, we showcase the asymptotic property of a basic averaging function in scenarios where the regression model operates without any constraint. We then derive a certified upper bound of the input perturbations when dealing with a family of regression models where the outputs are bounded. Our simulations verify the validity of the theoretical results and reveal the advantages and limitations of simple smoothing functions, i.e., averaging, in regression tasks. The code is publicly available at \url{https://github.com/arekavandi/Certified_Robust_Regression}.

NeurIPS Conference 2024 Conference Paper

Certified Adversarial Robustness via Randomized $\alpha$-Smoothing for Regression Models

• Aref M. Rekavandi
• Farhad Farokhi
• Olga Ohrimenko
• Benjamin I. Rubinstein

Certified adversarial robustness of large-scale deep networks has progressed substantially after the introduction of randomized smoothing. Deep net classifiers are now provably robust in their predictions against a large class of threat models, including $\ell_1$, $\ell_2$, and $\ell_\infty$ norm-bounded attacks. Certified robustness analysis by randomized smoothing has not been performed for deep regression networks where the output variable is continuous and unbounded. In this paper, we extend the existing results for randomized smoothing into regression models using powerful tools from robust statistics, in particular, $\alpha$-trimming filter as the smoothing function. Adjusting the hyperparameter $\alpha$ achieves a smooth trade-off between desired certified robustness and utility. For the first time, we propose a benchmark for certified robust regression in visual positioning systems using the Cambridge Landmarks dataset where robustness analysis is essential for autonomous navigation of AI agents and self-driving cars. Code is publicly available at \url{https: //github. com/arekavandi/Certified_adv_RRegression/}.

ECAI Conference 2023 Conference Paper

Fingerprint Attack: Client De-Anonymization in Federated Learning

• Qiongkai Xu
• Trevor Cohn
• Olga Ohrimenko

Federated Learning allows collaborative training without data sharing in settings where participants do not trust the central server and one another. Privacy can be further improved by ensuring that communication between the participants and the server is anonymized through a shuffle; decoupling the participant identity from their data. This paper seeks to examine whether such a defense is adequate to guarantee anonymity, by proposing a novel fingerprinting attack over gradients sent by the participants to the server. We show that clustering of gradients can easily break the anonymization in an empirical study of learning federated language models on two language corpora. We then show that training with differential privacy can provide a practical defense against our fingerprint attack.

NeurIPS Conference 2023 Conference Paper

RS-Del: Edit Distance Robustness Certificates for Sequence Classifiers via Randomized Deletion

• Zhuoqun Huang
• Neil G Marchant
• Keane Lucas
• Lujo Bauer
• Olga Ohrimenko
• Benjamin Rubinstein

Randomized smoothing is a leading approach for constructing classifiers that are certifiably robust against adversarial examples. Existing work on randomized smoothing has focused on classifiers with continuous inputs, such as images, where $\ell_p$-norm bounded adversaries are commonly studied. However, there has been limited work for classifiers with discrete or variable-size inputs, such as for source code, which require different threat models and smoothing mechanisms. In this work, we adapt randomized smoothing for discrete sequence classifiers to provide certified robustness against edit distance-bounded adversaries. Our proposed smoothing mechanism randomized deletion (RS-Del) applies random deletion edits, which are (perhaps surprisingly) sufficient to confer robustness against adversarial deletion, insertion and substitution edits. Our proof of certification deviates from the established Neyman-Pearson approach, which is intractable in our setting, and is instead organized around longest common subsequences. We present a case study on malware detection—a binary classification problem on byte sequences where classifier evasion is a well-established threat model. When applied to the popular MalConv malware detection model, our smoothing mechanism RS-Del achieves a certified accuracy of 91% at an edit distance radius of 128 bytes.

ICML Conference 2023 Conference Paper

Tight Data Access Bounds for Private Top-k Selection

• Hao Wu 0057
• Olga Ohrimenko
• Anthony Wirth

We study the top-$k$ selection problem under the differential privacy model: $m$ items are rated according to votes of a set of clients. We consider a setting in which algorithms can retrieve data via a sequence of accesses, each either a random access or a sorted access; the goal is to minimize the total number of data accesses. Our algorithm requires only $O(\sqrt{mk})$ expected accesses: to our knowledge, this is the first sublinear data-access upper bound for this problem. Our analysis also shows that the well-known exponential mechanism requires only $O(\sqrt{m})$ expected accesses. Accompanying this, we develop the first lower bounds for the problem, in three settings: only random accesses; only sorted accesses; a sequence of accesses of either kind. We show that, to avoid $\Omega(m)$ access cost, supporting both kinds of access is necessary, and that in this case our algorithm’s access cost is optimal.

NeurIPS Conference 2019 Conference Paper

An Algorithmic Framework For Differentially Private Data Analysis on Trusted Processors

• Joshua Allen
• Bolin Ding
• Janardhan Kulkarni
• Harsha Nori
• Olga Ohrimenko
• Sergey Yekhanin

Differential privacy has emerged as the main definition for private data analysis and machine learning. The global model of differential privacy, which assumes that users trust the data collector, provides strong privacy guarantees and introduces small errors in the output. In contrast, applications of differential privacy in commercial systems by Apple, Google, and Microsoft, use the local model. Here, users do not trust the data collector, and hence randomize their data before sending it to the data collector. Unfortunately, local model is too strong for several important applications and hence is limited in its applicability. In this work, we propose a framework based on trusted processors and a new definition of differential privacy called Oblivious Differential Privacy, which combines the best of both local and global models. The algorithms we design in this framework show interesting interplay of ideas from the streaming algorithms, oblivious algorithms, and differential privacy.

NeurIPS Conference 2019 Conference Paper

Oblivious Sampling Algorithms for Private Data Analysis

• Sajin Sasy
• Olga Ohrimenko

We study secure and privacy-preserving data analysis based on queries executed on samples from a dataset. Trusted execution environments (TEEs) can be used to protect the content of the data during query computation, while supporting differential-private (DP) queries in TEEs provides record privacy when query output is revealed. Support for sample-based queries is attractive due to \emph{privacy amplification} since not all dataset is used to answer a query but only a small subset. However, extracting data samples with TEEs while proving strong DP guarantees is not trivial as secrecy of sample indices has to be preserved. To this end, we design efficient secure variants of common sampling algorithms. Experimentally we show that accuracy of models trained with shuffling and sampling is the same for differentially private models for MNIST and CIFAR-10, while sampling provides stronger privacy guarantees than shuffling.

NeurIPS Conference 2018 Conference Paper

Contamination Attacks and Mitigation in Multi-Party Machine Learning

• Jamie Hayes
• Olga Ohrimenko

Machine learning is data hungry; the more data a model has access to in training, the more likely it is to perform well at inference time. Distinct parties may want to combine their local data to gain the benefits of a model trained on a large corpus of data. We consider such a case: parties get access to the model trained on their joint data but do not see each others individual datasets. We show that one needs to be careful when using this multi-party model since a potentially malicious party can taint the model by providing contaminated data. We then show how adversarial training can defend against such attacks by preventing the model from learning trends specific to individual parties data, thereby also guaranteeing party-level membership privacy.

I&C Journal 2018 Journal Article

Verifying the consistency of remote untrusted services with conflict-free operations

• Christian Cachin
• Olga Ohrimenko

A group of mutually trusting clients outsources a computation to a remote server, which they do not fully trust and that may be subject to attacks. The clients do not communicate with each other and would like to verify the correctness of the remote computation and the consistency of the server's responses. This paper presents the Conflict-free Operation verification Protocol (COP) that ensures linearizability when the server is correct and preserves fork-linearizability otherwise. Clients that observe each other's operations are consistent and their operations are linearizable. If the server forks two clients by hiding an operation, however, they never again see operations of each other. COP is wait-free in the sense that when executed with a correct server, non-conflicting operations can run without waiting for other clients. The paper gives a precise model for the guarantees of COP and includes a formal analysis that these are achieved.

SODA Conference 2012 Conference Paper

Privacy-preserving group data access via stateless oblivious RAM simulation

• Michael T. Goodrich
• Michael Mitzenmacher
• Olga Ohrimenko
• Roberto Tamassia