The High-Level Practical Overview of Open-Source Privacy-Preserving Machine Learning Solutions
Abstract
This paper aims to provide a high-level overview of practical approaches to machine-learning respecting the privacy and confidentiality of customer information, which is called Privacy-Preserving Machine Learning. First, the security approaches in offline-learning privacy methods are assessed. Those focused on modern cryptographic methods, such as Homomorphic Encryption and Secure Multi-Party Computation, as well as on dedicated combined hardware and software platforms like Trusted Execution Environment - Intel® Software Guard Extensions (Intel® SGX). Combining the security approaches with different machine learning architectures leads to our Proof of Concept in which the accuracy and speed of the security solutions will be examined. The next step was exploring and comparing the Open-Source Python-based solutions for PPML. Four solutions were selected from almost 40 separate, state-of-the-art systems: SyMPC, TF-Encrypted, TenSEAL, and Gramine. Three different Neural Network architectures were designed to show different libraries’ capabilities. The POC solves the image classification problem based on the MNIST dataset. As the computational results show, the accuracy of all considered secure approaches is similar. The maximum difference between non-secure and secure flow does not exceed 1.2%. In terms of secure computations, the most effective Privacy-Preserving Machine Learning library is based on Trusted Execution Environment, followed by Secure Multi-Party Computation and Homomorphic Encryption. However, most of those are at least 1000 times slower than the non-secure evaluation. Unfortunately, it is not acceptable for a real-world scenario. Future work could combine different security approaches, explore other new and existing state-of-the-art libraries or implement support for hardware-accelerated secure computation.
Full Text:
PDFReferences
F. Newsroom. (2018) Fda permits marketing of artificial intelligence-
based device to detect certain diabetes-related eye problems. [Online].
Available: https://www.fda.gov/news-events/press-announcements/fda-
permits-marketing-artificial-intelligence-based-device-detect-certain-
diabetes-related-eye
FDA. Artificial intelligence and machine learning (ai/ml)-enabled
medical devices. [Online]. Available: https://www.fda.gov/medical-
devices/software-medical-device-samd/artificial-intelligence-and-
machine-learning-aiml-enabled-medical-devices
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J.
Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in
nd International Conference on Learning Representations, ICLR 2014,
Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings,
Y. Bengio and Y. LeCun, Eds., 2014. [Online]. Available: http:
//arxiv.org/abs/1312.6199
A. Shafahi, W. R. Huang, M. Najibi, O. Suciu, C. Studer,
T. Dumitras, and T. Goldstein, “Poison frogs! targeted clean-
label poisoning attacks on neural networks,” in Advances
in Neural Information Processing Systems 31: Annual Conference on
Neural Information Processing Systems 2018, NeurIPS 2018, December
-8, 2018, Montr ́eal, Canada, S. Bengio, H. M. Wallach, H. Larochelle,
K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., 2018, pp.
–6116. [Online]. Available: https://proceedings.neurips.cc/paper/
/hash/22722a343513ed45f14905eb07621686-Abstract.html
X. Yuan, P. He, Q. Zhu, and X. Li, “Adversarial examples: Attacks
and defenses for deep learning,” IEEE Trans. Neural Networks Learn.
Syst., vol. 30, no. 9, pp. 2805–2824, 2019. [Online]. Available:
https://doi.org/10.1109/TNNLS.2018.2886017
D. M. Bamasoud, A. S. Al-Dossary, N. M. Al-Harthy, R. A.
Al-Shomrany, G. S. Alghamdi, and R. O. Algahmdi, “Privacy and
security issues in cloud computing: A survey paper,” in International
Conference on Information Technology, ICIT 2021, Amman, Jordan, July
-15, 2021. IEEE, 2021, pp. 387–392. [Online]. Available: https:
//doi.org/10.1109/ICIT52682.2021.9491632
Y. Zhang and R. Sion, “Speculative execution attacks and cloud
security,” in Proceedings of the 2019 ACM SIGSAC Conference on
Cloud Computing Security Workshop, CCSW@CCS 2019, London, UK,
November 11, 2019, R. Sion and C. Papamanthou, Eds. ACM, 2019,
p. 201. [Online]. Available: https://doi.org/10.1145/3338466.3360287
Y. Alghofaili, A. Albattah, N. Alrajeh, M. A. Rassam, and B. A. S.
Al-rimy, “Secure cloud infrastructure: A survey on issues, current
solutions, and open challenges,” Applied Sciences, vol. 11, no. 19, 2021.
[Online]. Available: https://www.mdpi.com/2076-3417/11/19/9005
N. Dowlin, R. Gilad-Bachrach, K. Laine, K. Lauter, M. Naehrig, and
J. Wernsing, “Cryptonets: Applying neural networks to encrypted data
with high throughput and accuracy,” Tech. Rep. MSR-TR-2016-3,
[Online]. Available: https://www.microsoft.com/en-us/research/
publication/cryptonets-applying-neural-networks-to-encrypted-data-
with-high-throughput-and-accuracy/
J. Alvarez-Valle, P. Bhatu, N. Chandran, D. Gupta, A. Nori, A. Rastogi,
M. Rathee, R. Sharma, and S. Ugare, “Secure medical image analysis
with cryptflow,” 2020.
A. Soin, P. Bhatu, R. Takhar, N. Chandran, D. Gupta, J. Alvarez-Valle,
R. Sharma, V. Mahajan, and M. P. Lungren, “Multi-institution encrypted
medical imaging ai validation without data sharing,” 2021.
M. H. M. Elham Tabassi (NIST), Kevin Burns (MITRE). A taxonomy
and terminology of adversarial machine learning. [Online]. Available:
https://csrc.nist.gov/publications/detail/nistir/8269/draft
Y. LeCun and C. Cortes, “MNIST handwritten digit database,” 2010.
[Online]. Available: http://yann.lecun.com/exdb/mnist/
F. Boemer, A. Costache, R. Cammarota, and C. Wierzynski, “ngraph-
he2: A high-throughput framework for neural network inference on
encrypted data,” 2019.
F. Boemer, Y. Lao, R. Cammarota, and C. Wierzynski, “ngraph-he: A
graph compiler for deep learning on homomorphically encrypted data,”
A. Benaissa, B. Retiat, B. Cebere, and A. E. Belfedhal, “Tenseal: A
library for encrypted tensor operations using homomorphic encryption,”
S. Carpov, P. Dubrulle, and R. Sirdey, “Armadillo: A compilation
chain for privacy preserving applications,” in Proceedings of the 3rd
International Workshop on Security in Cloud Computing, ser. SCC ’15.
Association for Computing Machinery, 2015, p. 13–19. [Online].
Available: https://doi.org/10.1145/2732516.2732520
I. Chillotti, N. Gama, M. Georgieva, and M. Izabach`ene, “Faster fully
homomorphic encryption: Bootstrapping in less than 0.1 seconds,” Cryp-
tology ePrint Archive, Report 2016/870, 2016, https://ia.cr/2016/870.
S. S. Magara, C. Yildirim, F. Yaman, B. Dilekoglu, F. R. Tutas,
E. ̈Ozt ̈urk, K. Kaya, ̈O. Tastan, and E. Savas, “Ml with he: Privacy
preserving machine learning inferences for genome studies,” 2021.
R. Dathathri, O. Saarikivi, H. Chen, K. Laine, K. Lauter, S. Maleki,
M. Musuvathi, and T. Mytkowicz, “Chet: an optimizing compiler for
fully-homomorphic neural-network inferencing,” in Proceedings of the
th ACM SIGPLAN Conference on Programming Language Design and
Implementation, 2019, pp. 142–156.
E. Hesamifard, H. Takabi, M. Ghasemi, and R. N. Wright, “Privacy-
preserving machine learning as a service,” Proc. Priv. Enhancing
Technol., vol. 2018, no. 3, pp. 123–142, 2018. [Online]. Available:
https://doi.org/10.1515/popets-2018-0024
C. Boura, N. Gama, M. Georgieva, and D. Jetchev, “Chimera: Combin-
ing ring-lwe-based fully homomorphic encryption schemes,” Cryptology
ePrint Archive, Report 2018/758, 2018, https://ia.cr/2018/758.
Q. Lou, B. Feng, G. C. Fox, and L. Jiang, “Glyph: Fast and accurately
training deep neural networks on encrypted data,” 2020.
OpenMined. (2021) Tenseal library. [Online]. Available: https:
//github.com/OpenMined/TenSEAL
J. H. Cheon, D. Kim, D. Kim, H. H. Lee, and K. Lee, “Numerical
method for comparison on homomorphically encrypted numbers,” Cryp-
tology ePrint Archive, Report 2019/417, 2019, https://ia.cr/2019/417.
J. H. Cheon, D. Kim, and D. Kim, “Efficient homomorphic comparison
methods with optimal complexity,” Cryptology ePrint Archive, Report
/1234, 2019, https://ia.cr/2019/1234.
U. Michelucci, Advanced applied deep learning : convolutional neural
networks and object detection. Apress, 2019.
A. Dalskov, D. Escudero, and M. Keller, “Secure evaluation of quantized
neural networks,” Cryptology ePrint Archive, Report 2019/131, 2019,
https://ia.cr/2019/131.
OpenMined. (2021) Sympc library. [Online]. Available: https://
github.com/OpenMined/SyMPC
N. Kumar, M. Rathee, N. Chandran, D. Gupta, A. Rastogi, and
R. Sharma, “Cryptflow: Secure tensorflow inference,” 2020.
EXAMPLE OF ARTICLE FOR INTERNATIONAL JOURNALS OF ELECTRONICS AND TELECOMMUNICATIONS 7
D. Rathee, M. Rathee, N. Kumar, N. Chandran, D. Gupta, A. Rastogi,
and R. Sharma, “Cryptflow2: Practical 2-party secure inference,”
Proceedings of the 2020 ACM SIGSAC Conference on Computer and
Communications Security, 2020. [Online]. Available: http://dx.doi.org/
1145/3372297.3417274
D. Rathee, M. Rathee, R. K. K. Goli, D. Gupta, R. Sharma, N. Chandran,
and A. Rastogi, “Sirnn: A math library for secure rnn inference,” Cryp-
tology ePrint Archive, Report 2021/459, 2021, https://ia.cr/2021/459.
B. Knott, S. Venkataraman, A. Hannun, S. Sengupta, M. Ibrahim, and
L. van der Maaten, “Crypten: Secure multi-party computation meets
machine learning,” in arXiv 2109.00984, 2021.
M. Dahl, J. Mancuso, Y. Dupis, B. Decoste, M. Giraud, I. Livingstone,
J. Patriquin, and G. Uhma, “Private machine learning in tensorflow using
secure computation,” 2018.
W. Henecka, S. K ̈ogl, A.-R. Sadeghi, T. Schneider, and I. Wehrenberg,
“Tasty: Tool for automating secure two-party computations,” Cryptology
ePrint Archive, Report 2010/365, 2010, https://ia.cr/2010/365.
P. Mohassel and P. Rindal, “Aby¡sup¿3¡/sup¿: A mixed protocol
framework for machine learning,” in Proceedings of the 2018 ACM
SIGSAC Conference on Computer and Communications Security, ser.
CCS ’18. Association for Computing Machinery, 2018, p. 35–52.
[Online]. Available: https://doi.org/10.1145/3243734.3243760
S. Wagh, D. Gupta, and N. Chandran, “Securenn: Efficient and private
neural network training,” Cryptology ePrint Archive, Report 2018/442,
, https://ia.cr/2018/442.
W. Zheng, R. Deng, W. Chen, R. A. Popa, A. Panda, and I. Stoica,
“Cerebro: A platform for Multi-Party cryptographic collaborative
learning,” in 30th USENIX Security Symposium (USENIX Security 21).
USENIX Association, 2021, pp. 2723–2740. [Online]. Available:
https://www.usenix.org/conference/usenixsecurity21/presentation/zheng
S. Wagh, S. Tople, F. Benhamouda, E. Kushilevitz, P. Mittal, and
T. Rabin, “Falcon: Honest-majority maliciously secure framework for
private deep learning,” 2020.
M. S. Riazi, M. Samragh, H. Chen, K. Laine, K. Lauter, and F. Koushan-
far, “Xonn: Xnor-based oblivious deep neural network inference,” 2019.
M. S. Riazi, C. Weinert, O. Tkachenko, E. M. Songhori, T. Schneider,
and F. Koushanfar, “Chameleon: A hybrid secure computation frame-
work for machine learning applications,” 2018.
A.-R. Sadeghi and T. Schneider, “Generalized universal circuits for
secure evaluation of private functions with application to data clas-
sification,” Cryptology ePrint Archive, Report 2008/453, 2008, https:
//ia.cr/2008/453.
M. Barni, P. Failla, R. Lazzeretti, A.-R. Sadeghi, and T. Schneider,
“Privacy-preserving ecg classification with branching programs and
neural networks,” IEEE Transactions on Information Forensics and
Security, vol. 6, no. 2, pp. 452–468, 2011.
P. Mohassel and Y. Zhang, “Secureml: A system for scalable privacy-
preserving machine learning,” in 2017 IEEE Symposium on Security and
Privacy (SP), 2017, pp. 19–38.
N. Koti, A. Patra, R. Rachuri, and A. Suresh, “Tetrad: Actively secure
pc for secure training and inference,” Cryptology ePrint Archive,
Report 2021/755, 2021, https://ia.cr/2021/755.
A. Patra and A. Suresh, “Blaze: Blazing fast privacy-preserving machine
learning,” Proceedings 2020 Network and Distributed System Security
Symposium, 2020. [Online]. Available: http://dx.doi.org/10.14722/
ndss.2020.24202
N. Koti, M. Pancholi, A. Patra, and A. Suresh, “Swift: Super-fast and
robust privacy-preserving machine learning,” 2021.
EzPC. (2021) Ezpc. [Online]. Available: https://github.com/mpc-msri/
EzPC
PySyft. (2021) Pysyft. [Online]. Available: https://github.com/
OpenMined/PySyft
T. Ryffel, P. Tholoniat, D. Pointcheval, and F. Bach, “Ariann: Low-
interaction privacy-preserving deep learning via function secret sharing,”
D. Labs. (2021) tf-encrypted library. [Online]. Available: https:
//github.com/tf-encrypted/tf-encrypted
gramine. (2021) gramine, library. [Online]. Available: https:
//github.com/gramineproject/gramine
D. Labs. (2021) tf-trusted, library. [Online]. Available: https://
github.com/capeprivacy/tf-trusted
F. Tram`er and D. Boneh, “Slalom: Fast, verifiable and private execution
of neural networks in trusted hardware,” 2019.
F. Mo, H. Haddadi, K. Katevas, E. Marin, D. Perino, and N. Kourtel-
lis, “Ppfl: Privacy-preserving federated learning with trusted execution
environments,” 2021.
F. Mo, A. S. Shamsabadi, K. Katevas, S. Demetriou, I. Leontiadis,
A. Cavallaro, and H. Haddadi, “Darknetz,” Proceedings of the 18th
International Conference on Mobile Systems, Applications, and Services,
[Online]. Available: http://dx.doi.org/10.1145/3386901.3388946
J. J. Dai, Y. Wang, X. Qiu, D. Ding, Y. Zhang, Y. Wang, X. Jia,
L. C. Zhang, Y. Wan, Z. Li, J. Wang, S. Huang, Z. Wu, Y. Wang,
Y. Yang, B. She, D. Shi, Q. Lu, K. Huang, and G. Song, “Bigdl: A
distributed deep learning framework for big data,” in Proceedings of
the ACM Symposium on Cloud Computing, ser. SoCC’19. Association
for Computing Machinery, 2019, pp. 50–60. [Online]. Available:
https://arxiv.org/pdf/1804.05839.pdf
M. Orenbach, P. Lifshits, M. Minkin, and M. Silberstein, “Eleos: Exitless
os services for sgx enclaves,” in Proceedings of the Twelfth European
Conference on Computer Systems, ser. EuroSys ’17. Association
for Computing Machinery, 2017, p. 238–253. [Online]. Available:
https://doi.org/10.1145/3064176.3064219
R. Kunkel, D. L. Quoc, F. Gregor, S. Arnautov, P. Bhatotia, and
C. Fetzer, “Tensorscone: A secure tensorflow framework using intel sgx,”
W. Ozga, D. L. Quoc, and C. Fetzer, “Perun: Secure multi-stakeholder
machine learning framework with gpu support,” 2021.
A. Mondal, Y. More, R. H. Rooparaghunath, and D. Gupta, “Flatee:
Federated learning across trusted execution environments,” 2021.
LeNET. (2021) Lenet. [Online]. Available: https://en.wikipedia.org/
wiki/LeNet
Refbacks
- There are currently no refbacks.
International Journal of Electronics and Telecommunications
is a periodical of Electronics and Telecommunications Committee
of Polish Academy of Sciences
eISSN: 2300-1933
