research-article

EMShepherd: Detecting Adversarial Samples via Side-channel Leakage

Authors:

Yunsi FeiAuthors Info & Claims

ASIA CCS '23: Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security

Pages 300 - 313

https://doi.org/10.1145/3579856.3582827

Published: 10 July 2023 Publication History

Abstract

Deep Neural Networks (DNN) are vulnerable to adversarial perturbations — small changes crafted deliberately on the input to mislead the model for wrong predictions. Adversarial attacks have disastrous consequences for deep learning empowered critical applications. Existing defense and detection techniques both require extensive knowledge of the model, testing inputs and even execution details. They are not viable for general deep learning implementations where the model internal is unknown, a common ‘black-box’ scenario for model users. Inspired by the fact that electromagnetic (EM) emanations of a model inference are dependent on both operations and data and may contain footprints of different input classes, we propose a framework, EMShepherd, to capture EM traces of model execution, perform processing on traces and exploit them for adversarial detection. Only benign samples and their EM traces are used to train the adversarial detector: a set of EM classifiers and class-specific unsupervised anomaly detectors. When the victim model system is under attack by an adversarial example, the model execution will be different from executions for the known classes, and the EM trace will be different. We demonstrate that our air-gapped EMShepherd can effectively detect different adversarial attacks on a commonly used FPGA deep learning accelerator for both Fashion MNIST and CIFAR-10 datasets. It achieves a detection rate on most types of adversarial samples, which is comparable to the state-of-the-art ‘white-box’ software-based detectors.

References

[1]

2019. F-Score Definition | DeepAI. https://deepai.org/machine-learning-glossary-and-terms/f-score. (Accessed on 01/23/2022).

[2]

2020. Running Average Power Limit Energy Reporting. https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/advisory-guidance/running-average-power-limit-energy-reporting.html.

[3]

2020. Zynq DPU v3.2 IP Product Guide. https://www.xilinx.com/support/documentation/ip_documentation/dpu/v3_2/pg338-dpu.pdf. (Accessed on 01/23/2022).

[4]

2021. TensorFlow. https://www.tensorflow.org [Online; accessed 24. Jan. 2022].

[5]

2021. Welcome to Ultra96-PYNQ’s documentation! — Ultra96-PYNQ v2.6 documentation. https://ultra96-pynq.readthedocs.io/en/latest [Online; accessed 23. Jan. 2022].

[6]

2022. Probe Set PBS 2 (incl. Preamplifier). https://aaronia-shop.com/products/probe-set-pbs-2-incl-preamplifier [Online; accessed 23. Jan. 2022].

[7]

2022. Ultra96-V2 | Avnet Boards. https://www.avnet.com/wps/portal/us/products/avnet-boards/avnet-board-families/ultra96-v2 [Online; accessed 23. Jan. 2022].

[8]

2022. Vitis AI. https://www.xilinx.com/products/design-tools/vitis/vitis-ai.html [Online; accessed 24. Jan. 2022].

[9]

Dakshi Agrawal, Bruce Archambeault, Josyula R Rao, and Pankaj Rohatgi. 2002. The EM side-channel (s). In Int. Workshop on Cryptographic Hardware & Embedded Systems. 29–45.

[10]

D. Agrawal, B. Archambeault, J. R. Rao, and P. Rohatgi. 2002. The EM side-channels. In Int. WkShp on Cryptographic Hardware & Embedded Systems.

[11]

Lejla Batina, Shivam Bhasin, Dirmanto Jap, and Stjepan Picek. 2019. { CSI}{ NN} : Reverse Engineering of Neural Network Architectures Through Electromagnetic Side Channel. In USENIX Security Symp.515–532.

[12]

Arjun Nitin Bhagoji, Daniel Cullina, and Prateek Mittal. 2017. Dimensionality reduction as a defense against evasion attacks on machine learning classifiers. arXiv preprint arXiv:1704.02654 2 (2017), 1.

[13]

Shobhit Bhatnagar, Deepanway Ghosal, and Maheshkumar H Kolekar. 2017. Classification of fashion article images using convolutional neural networks. In Int. Conf. on Image Information Processing (ICIIP). 1–6.

[14]

Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016).

[15]

Nicholas Carlini and David Wagner. 2017. Adversarial examples are not easily detected: Bypassing ten detection methods. In ACM Workshop on Artificial Intelligence & Security. 3–14.

Digital Library

[16]

Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In IEEE Symp. on Security & Privacy. IEEE, 39–57.

[17]

Suresh Chari, Josyula R Rao, and Pankaj Rohatgi. 2002. Template attacks. In Int. Workshop on Cryptographic Hardware & Embedded Systems. Springer, 13–28.

[18]

Łukasz Chmielewski and Léo Weissbart. 2021. On reverse engineering neural network implementation on gpu. In International Conference on Applied Cryptography and Network Security. Springer, 96–113.

Digital Library

[19]

Debayan Das, Anupam Golder, Josef Danial, Santosh Ghosh, Arijit Raychowdhury, and Shreyas Sen. 2019. X-DeepSCA: Cross-device deep learning side channel attack. In Proc. Design Automation Conf.1–6.

Digital Library

[20]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization.Journal of machine learning research 12, 7 (2011).

[21]

Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. 2017. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410 (2017).

[22]

David Forsyth and Jean Ponce. 2011. Computer vision: A modern approach.Prentice hall.

[23]

Kenneth R Foster, Robert Koprowski, and Joseph D Skufca. 2014. Machine learning, medical diagnosis, and biomedical engineering research-commentary. Biomedical engineering online 13, 1 (2014), 1–9.

[24]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. J. Machine Learning Research 17, 1 (2016), 2096–2030.

Digital Library

[25]

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).

[26]

Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. 2017. On the (statistical) detection of adversarial examples. arXiv preprint arXiv:1702.06280 (2017).

[27]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. arxiv:1512.03385 [cs.CV]

[28]

Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. aXiv:1704.04861 (2017).

[29]

Jin Huang and Charles X Ling. 2005. Using AUC and accuracy in evaluating learning algorithms. IEEE Transactions on knowledge and Data Engineering 17, 3 (2005), 299–310.

Digital Library

[30]

Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei. 2021. Layercam: Exploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing 30 (2021), 5875–5888.

Digital Library

[31]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[32]

Paul Kocher, Joshua Jaffe, and Benjamin Jun. 1999. Differential power analysis. In Annual Int. Cryptology Conf. Springer, 388–397.

[33]

Alexey Kurakin, Ian Goodfellow, Samy Bengio, 2016. Adversarial examples in the physical world.

[34]

[34] Teledyne LeCroy. 2022. http://cdn.teledynelecroy.com/files/pdf/waverunner_6_zi_datasheet.pdf[Online; accessed 23. Jan. 2022].

[35]

Xin Li and Fuxin Li. 2017. Adversarial examples detection in deep networks with convolutional filter statistics. In Proc. IEEE Int. Conf. on Computer Vision. 5764–5772.

[36]

Moritz Lipp, Andreas Kogler, David Oswald, Michael Schwarz, Catherine Easdon, Claudio Canella, and Daniel Gruss. 2021. PLATYPUS: Software-based Power Side-Channel Attacks on x86. In 2021 IEEE Symposium on Security and Privacy (SP). IEEE.

[37]

Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh. 2018. Towards robust neural networks via random self-ensemble. In Proceedings of the European Conference on Computer Vision (ECCV). 369–385.

Digital Library

[38]

Shiqing Ma and Yingqi Liu. 2019. Nic: Detecting adversarial samples with neural network invariant checking. In Proc. Network & Distributed System Security Symposium (NDSS 2019).

[39]

Xingjun Ma, Bo Li, Yisen Wang, Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E Houle, and James Bailey. 2018. Characterizing adversarial subspaces using local intrinsic dimensionality. arXiv preprint arXiv:1801.02613 (2018).

[40]

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017).

[41]

Christopher Manning and Hinrich Schutze. 1999. Foundations of statistical natural language processing. MIT press.

Digital Library

[42]

Dongyu Meng and Hao Chen. 2017. Magnet: a two-pronged defense against adversarial examples. In ACM SIGSAC Conf. on Computer & Communications Security. 135–147.

Digital Library

[43]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. Deepfool: a simple and accurate method to fool deep neural networks. In Proc. IEEE Conf. Computer Vision & Pattern Recognition. 2574–2582.

[44]

Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep face recognition. British Machine Vision Association.

[45]

Jonas Rauber, Wieland Brendel, and Matthias Bethge. 2017. Foolbox: A Python toolbox to benchmark the robustness of machine learning models. In Workshop on Reliable Machine Learning in the Wild. http://arxiv.org/abs/1707.04131

[46]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You Only Look Once: Unified, Real-Time Object Detection. arXiv preprint arXiv:1506.02640 (2016).

[47]

Jonathan G Richens, Ciarn M Lee, and Saurabh Johri. 2020. Improving the accuracy of medical diagnosis with causal machine learning. Nature communications 11, 1 (2020), 1–9.

[48]

Ramprasaath R Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. 2016. Grad-CAM: Why did you say that?arXiv preprint arXiv:1611.07450 (2016).

[49]

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 3–18.

[50]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[51]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. arxiv:1409.1556 [cs.CV]

[52]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.

Digital Library

[53]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2014. Going Deeper with Convolutions. arXiv preprint arXiv:1409.4842 (2014).

[54]

Guanhong Tao, Shiqing Ma, Yingqi Liu, and Xiangyu Zhang. 2018. Attacks meet interpretability: Attribute-steered detection of adversarial samples. arXiv preprint arXiv:1810.11580 (2018).

[55]

Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. 2020. On adaptive attacks to adversarial example defenses. Advances in Neural Information Processing Systems 33 (2020), 1633–1645.

[56]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE.Journal of machine learning research 9, 11 (2008).

[57]

Kira Vinogradova, Alexandr Dibrov, and Gene Myers. 2020. Towards interpretable semantic segmentation via gradient-weighted class activation mapping (student abstract). In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 13943–13944.

[58]

Siyue Wang, Xiao Wang, Pu Zhao, Wujie Wen, David Kaeli, Peter Chin, and Xue Lin. 2018. Defensive dropout for hardening deep neural networks under adversarial attacks. In Proceedings of the International Conference on Computer-Aided Design. 1–8.

Digital Library

[59]

Xiao Wang, Siyue Wang, Pin-Yu Chen, Yanzhi Wang, Brian Kulis, Xue Lin, and Sang Peter Chin. 2019. Protecting Neural Networks with Hierarchical Random Switching: Towards Better Robustness-Accuracy Trade-off for Stochastic Defenses. In IJCAI.

[60]

Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang Song, Qian Wang, and Hairong Qi. 2019. Beyond inferring class representatives: User-level privacy leakage from federated learning. In IEEE Conf. on Computer Communications. IEEE, 2512–2520.

Digital Library

[61]

Weilin Xu, David Evans, and Yanjun Qi. 2017. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155 (2017).

[62]

Yijun Yang, Ruiyuan Gao, Yu Li, Qiuxia Lai, and Qiang Xu. 2022. What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction. arXiv preprint arXiv:2201.09650 (2022).

[63]

Honggang Yu, Haocheng Ma, Kaichen Yang, Yiqiang Zhao, and Yier Jin. 2020. DeepEM: Deep Neural Networks Model Recovery through EM Side-Channel Information Leakage. In IEEE Int. Symp. on Hardware Oriented Security & Trust (HOST). IEEE, 209–218.

[64]

Yicheng Zhang, Rozhin Yasaei, Hao Chen, Zhou Li, and Mohammad Abdullah Al Faruque. 2021. Stealing neural network structure through remote fpga side-channel analysis. IEEE Trans. on Information Forensics & Security 16 (2021), 4377–4388.

[65]

Dawei Zhou, Nannan Wang, Chunlei Peng, Xinbo Gao, Xiaoyu Wang, Jun Yu, and Tongliang Liu. 2021. Removing adversarial noise in class activation feature space. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7878–7887.

[66]

Jiang Zhu, Lizan Wang, Haolin Liu, Shujuan Tian, Qingyong Deng, and Jianqi Li. 2020. An Efficient Task Assignment Framework to Accelerate DPU-Based Convolutional Neural Network Inference on FPGAs. IEEE Access 8 (2020), 83224–83237. https://doi.org/10.1109/ACCESS.2020.2988311

Cited By

Islam MAlouani IKhasawneh K(2024)Hardware Support for Trustworthy Machine Learning: A Survey2024 25th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED60706.2024.10528373(1-6)Online publication date: 3-Apr-2024
https://doi.org/10.1109/ISQED60706.2024.10528373

Index Terms

EMShepherd: Detecting Adversarial Samples via Side-channel Leakage
1. Security and privacy
  1. Security in hardware
    1. Embedded systems security
    2. Hardware attacks and countermeasures
      1. Side-channel analysis and countermeasures

Recommendations

SoK: Deep Learning-based Physical Side-channel Analysis
Side-channel attacks represent a realistic and serious threat to the security of embedded devices for already almost three decades. A variety of attacks and targets they can be applied to have been introduced, and while the area of side-channel attacks ...
On the Challenges of Detecting Side-Channel Attacks in SGX
RAID '22: Proceedings of the 25th International Symposium on Research in Attacks, Intrusions and Defenses

Existing tools to detect side-channel attacks on Intel SGX are grounded on the observation that attacks affect the performance of the victim application. As such, all detection tools monitor the potential victim and raise an alarm if the witnessed ...
Preventing and detecting cache side-channel attacks in cloud computing
ICC '17: Proceedings of the Second International Conference on Internet of things, Data and Cloud Computing

Cloud computing offers new cost effective services on-demand such as Software as a service (SaaS), Infrastructure as a service (IaaS) and Platform as a service (PaaS). However, with all of these services promising facilities and benefits, there are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASIA CCS '23: Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security

July 2023

1066 pages

ISBN:9798400700989

DOI:10.1145/3579856

Editors:
Joseph Liu
Monash University, Australia
,
Yang Xiang
Swinburne University of Technology, Australia
,
Surya Nepal
Data61, Australia
,
Gene Tsudik
University of California Irvine, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Distinguished Paper

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ASIA CCS '23

Sponsor:

SIGSAC

ASIA CCS '23: ACM Asia Conference on Computer and Communications Security

July 10 - 14, 2023

VIC, Melbourne, Australia

Acceptance Rates

Overall Acceptance Rate 418 of 2,322 submissions, 18%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
182
Total Downloads

Downloads (Last 12 months)129
Downloads (Last 6 weeks)7

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Islam MAlouani IKhasawneh K(2024)Hardware Support for Trustworthy Machine Learning: A Survey2024 25th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED60706.2024.10528373(1-6)Online publication date: 3-Apr-2024
https://doi.org/10.1109/ISQED60706.2024.10528373

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents