Access the full text.
Sign up today, get DeepDyve free for 14 days.
P. Subramani, Nicholas Vadivelu, Gautam Kamath (2020)
Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorization
per-formant vectorised mapping capabilities. Having encapsulated the sampling log-probability of the Gaussian mixture NumPyro Distribution , we can easily
M. Wainwright, Michael Jordan (2008)
Graphical Models, Exponential Families, and Variational InferenceFound. Trends Mach. Learn., 1
Yann LeCun, L. Bottou, Yoshua Bengio, P. Haffner (1998)
Gradient-based learning applied to document recognitionProc. IEEE, 86
(2015)
TensorFlow: Large-scale machine learning on heterogeneous systems
I. Gazeau, D. Miller, C. Palamidessi (2013)
Preserving differential privacy under finite-precision semanticsTheor. Comput. Sci., 655
Michael Jordan, Zoubin Ghahramani, T. Jaakkola, L. Saul (1999)
An Introduction to Variational Methods for Graphical ModelsMachine Learning, 37
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, N. Gimelshein, L. Antiga, Alban Desmaison, Andreas Köpf, E. Yang, Zach DeVito, Martin Raison, A. Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, Soumith Chintala (2019)
PyTorch: An Imperative Style, High-Performance Deep Learning Library
(2021)
CUDA-Shuffle: GPU shuf-fle using bijective functions
冯利芳 (2020)
FacebookThe SAGE International Encyclopedia of Mass Media and Society
Rory Mitchell, Daniel Stokes, E. Frank, G. Holmes (2021)
Bandwidth-Optimal Random Shuffling for GPUsACM Transactions on Parallel Computing, 9
This is implemented for batched data by the log_prob method in Listing 5 using JAX’s highly per-formant
Martín Abadi, Andy Chu, I. Goodfellow, H. McMahan, Ilya Mironov, Kunal Talwar, Li Zhang (2016)
Deep Learning with Differential PrivacyProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security
(2019)
Introducing Tensor- Flow privacy: Learning with differential privacy for training data. TensorFlow Blog, https://blog.tensorflow.org/2019/ 03/introducing-tensorflow-privacy-learning.html, 2019
S. Garfinkel, Philip Leclerc (2020)
Randomness Concerns when Deploying Differential PrivacyProceedings of the 19th Workshop on Privacy in the Electronic Society
and use the Inverse Gamma distribution
Michalis Titsias, M. Lázaro-Gredilla (2014)
Doubly Stochastic Variational Bayes for non-Conjugate Inference
it in our model shown in Listing 6 and only specify the prior distributions for the parameters
method for computing the log-probability of the Gaussian mixture where we marginalise out the latent variables to avoid these issues
Du Phan, Neeraj Pradhan, M. Jankowiak (2019)
Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyroArXiv, abs/1912.11554
H. Feistel (1973)
Cryptography and Computer PrivacyScientific American, 228
Romain Lopez, Pierre Boyeau, N. Yosef, Michael Jordan, J. Regier (2020)
AUTO-ENCODING VARIATIONAL BAYES
(2018)
JAX: composable transformations of Python+NumPy programs
Rami Al-Rfou, Guillaume Alain, Amjad Almahairi, Christof Angermüller, Dzmitry Bahdanau, Nicolas Ballas, Frédéric Bastien, Justin Bayer, A. Belikov, A. Belopolsky, Yoshua Bengio, Arnaud Bergeron, J. Bergstra, Valentin Bisson, Josh Snyder, Nicolas Bouchard, Nicolas Boulanger-Lewandowski, Xavier Bouthillier, A. Brébisson, Olivier Breuleux, P. Carrier, Kyunghyun Cho, J. Chorowski, P. Christiano, Tim Cooijmans, Marc-Alexandre Côté, Myriam Côté, Aaron Courville, Yann Dauphin, Olivier Delalleau, Julien Demouth, Guillaume Desjardins, S. Dieleman, Laurent Dinh, Mélanie Ducoffe, Vincent Dumoulin, S. Kahou, D. Erhan, Ziye Fan, Orhan Firat, M. Germain, Xavier Glorot, I. Goodfellow, M. Graham, Çaglar Gülçehre, P. Hamel, Iban Harlouchet, J. Heng, Balázs Hidasi, S. Honari, Arjun Jain, Sébastien Jean, Kai Jia, Mikhail Korobov, Vivek Kulkarni, Alex Lamb, Pascal Lamblin, Eric Larsen, César Laurent, S. Lee, S. Lefrançois, S. Lemieux, Nicholas Léonard, Zhouhan Lin, J. Livezey, C. Lorenz, J. Lowin, Qianli Ma, Pierre-Antoine Manzagol, Olivier Mastropietro, R. McGibbon, R. Memisevic, B. Merrienboer, Vincent Michalski, Mehdi Mirza, A. Orlandi, C. Pal, Razvan Pascanu, M. Pezeshki, Colin Raffel, D. Renshaw, M. Rocklin, Adriana Romero, Markus Roth, Peter Sadowski, J. Salvatier, F. Savard, Jan Schlüter, J. Schulman, Gabriel Schwartz, Iulian Serban, Dmitriy Serdyuk, S. Shabanian, Étienne Simon, Sigurd Spieckermann, S. Subramanyam, Jakub Sygnowski, Jérémie Tanguay, Gijs Tulder, Joseph Turian, S. Urban, Pascal Vincent, Francesco Visin, Harm Vries, David Warde-Farley, Dustin Webb, M. Willson, Kelvin Xu, Lijun Xue, Li Yao, Saizheng Zhang, Ying Zhang (2016)
Theano: A Python framework for fast computation of mathematical expressionsArXiv, abs/1605.02688
Dustin Tran, M. Hoffman, David Moore, Christopher Suter, Srinivas Vasudevan, Alexey Radul, Matthew Johnson, R. Saurous (2018)
Simple, Distributed, and Accelerated Probabilistic Programming
Bob Carpenter, A. Gelman, M. Hoffman, Daniel Lee, Ben Goodrich, M. Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, A. Riddell (2017)
Stan: A Probabilistic Programming Language.Journal of statistical software, 76
P. Kairouz, Ziyu Liu, T. Steinke (2021)
The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation
(2017)
Tensorflow distributions
Roy Frostig, Matthew Johnson, Chris Leary (2018)
Compiling machine learning programs via high-level tracing
Eli Bingham, Jonathan Chen, M. Jankowiak, F. Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, Noah Goodman (2018)
Pyro: Deep Universal Probabilistic ProgrammingArXiv, abs/1810.09538
Joonas Jälkö, Eemil Lagerspetz, J. Haukka, S. Tarkoma, Samuel Kaski, A. Honkela (2019)
Privacy-preserving data sharing via probabilistic modelingPatterns, 2
G. Wong, W. Mason (1985)
The Hierarchical Logistic Regression Model for Multilevel AnalysisJournal of the American Statistical Association, 80
Anand Patil, David Huard, C. Fonnesbeck (2010)
PyMC: Bayesian Stochastic Modelling in Python.Journal of statistical software, 35 4
(2019)
Introducing TensorFlow privacy: Learning with differential privacy for training data
Ú. Erlingsson, Ilya Mironov, A. Raghunathan, Shuang Song (2019)
That which we call privateArXiv, abs/1908.03566
A. Koskela, Joonas Jälkö, A. Honkela (2019)
Computing Tight Differential Privacy Guarantees Using FFT
Perturb by adding zero-mean Gaussian noise with variance C 2 σ 2
Aggregate per-instance gradients
Diederik Kingma, Jimmy Ba (2014)
Adam: A Method for Stochastic OptimizationCoRR, abs/1412.6980
C. Dwork, Aaron Roth (2014)
The Algorithmic Foundations of Differential PrivacyFound. Trends Theor. Comput. Sci., 9
Fabian Pedregosa, G. Varoquaux, Alexandre Gramfort, V. Michel, B. Thirion, O. Grisel, Mathieu Blondel, Gilles Louppe, P. Prettenhofer, Ron Weiss, Ron Weiss, J. Vanderplas, Alexandre Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay (2011)
Scikit-learn: Machine Learning in PythonArXiv, abs/1201.0490
C. Canonne, Gautam Kamath, T. Steinke (2020)
The Discrete Gaussian for Differential PrivacyArXiv, abs/2004.00010
Shuang Song, Kamalika Chaudhuri, A. Sarwate (2013)
Stochastic gradient descent with differentially private updates2013 IEEE Global Conference on Signal and Information Processing
S. Papson (1981)
“Model”Humanity & Society, 5
Ilya Mironov, Omkant Pandey, Omer Reingold, S. Vadhan (2009)
Computational Differential Privacy
Charles Harris, K. Millman, S. Walt, R. Gommers, Pauli Virtanen, D. Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel Smith, Robert Kern, Matti Picus, Stephan Hoyer, M. Kerkwijk, M. Brett, A. Haldane, Jaime R'io, Marcy Wiebe, Pearu Peterson, Pierre G'erard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, C. Gohlke, T. Oliphant (2020)
Array programming with NumPyNature, 585
M. Luby, C. Rackoff (1986)
How to Construct Pseudo-Random Permutations from Pseudo-Random Functions (Abstract)SIAM J. Comput., 17
Han Xiao, Kashif Rasul, Roland Vollgraf (2017)
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning AlgorithmsArXiv, abs/1708.07747
C. Dwork, K. Kenthapadi, Frank McSherry, Ilya Mironov, M. Naor (2006)
Our Data, Ourselves: Privacy Via Distributed Noise Generation
Joonas Jälkö, A. Honkela, O. Dikmen (2016)
Differentially Private Variational Inference for Non-conjugate ModelsArXiv, abs/1610.08749
log p ( x i ) = log K X j =1 ( π j N ( x i ; µ j , Σ j )
A. Koskela, Joonas Jälkö, Lukas Prediger, A. Honkela (2020)
Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT
The code in Listing 6 reflects this using the same imperative sampling instructions demonstrated in the earlier examples. We note that, in principle
Ilya Mironov (2012)
On significance of the least significant bits for differential privacyProceedings of the 2012 ACM conference on Computer and communications security
A. Krizhevsky (2009)
Learning Multiple Layers of Features from Tiny Images
C. Dwork, Frank McSherry, Kobbi Nissim, Adam Smith (2006)
Calibrating Noise to Sensitivity in Private Data Analysis
AbstractWe present d3p, a software package designed to help fielding runtime efficient widely-applicable Bayesian inference under differential privacy guarantees. d3p achieves general applicability to a wide range of probabilistic modelling problems by implementing the differentially private variational inference algorithm, allowing users to fit any parametric probabilistic model with a differentiable density function. d3p adopts the probabilistic programming paradigm as a powerful way for the user to flexibly define such models. We demonstrate the use of our software on a hierarchical logistic regression example, showing the expressiveness of the modelling approach as well as the ease of running the parameter inference. We also perform an empirical evaluation of the runtime of the private inference on a complex model and find a ~10 fold speed-up compared to an implementation using TensorFlow Privacy.
Proceedings on Privacy Enhancing Technologies – de Gruyter
Published: Apr 1, 2022
Keywords: differential privacy; JAX; NumPyro; probabilistic programming; variational inference
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.