Adversarial machine learning

Adversarial machine learning is the study of the attacks on machine learning algorithms, and of the defenses against such attacks.^[1] A survey from May 2020 exposes the fact that practitioners report a dire need for better protecting machine learning systems in industrial applications.^[2]

Most machine learning techniques are mostly designed to work on specific problem sets, under the assumption that the training and test data are generated from the same statistical distribution (IID). However, this assumption is often dangerously violated in practical high-stake applications, where users may intentionally supply fabricated data that violates the statistical assumption.

Most common attacks in adversarial machine learning include evasion attacks,^[3] data poisoning attacks,^[4] Byzantine attacks^[5] and model extraction.^[6]

^ Kianpour, Mazaher; Wen, Shao-Fang (2020). "Timing Attacks on Machine Learning: State of the Art". Intelligent Systems and Applications. Advances in Intelligent Systems and Computing. Vol. 1037. pp. 111–125. doi:10.1007/978-3-030-29516-5_10. ISBN 978-3-030-29515-8. S2CID 201705926.
^ Siva Kumar, Ram Shankar; Nyström, Magnus; Lambert, John; Marshall, Andrew; Goertzel, Mario; Comissoneru, Andi; Swann, Matt; Xia, Sharon (May 2020). "Adversarial Machine Learning-Industry Perspectives". 2020 IEEE Security and Privacy Workshops (SPW). pp. 69–75. doi:10.1109/SPW50608.2020.00028. ISBN 978-1-7281-9346-5. S2CID 229357721.
^ Goodfellow, Ian; McDaniel, Patrick; Papernot, Nicolas (25 June 2018). "Making machine learning robust against adversarial inputs". Communications of the ACM. 61 (7): 56–66. doi:10.1145/3134599. ISSN 0001-0782.^{[permanent dead link]}
^ Geiping, Jonas; Fowl, Liam H.; Huang, W. Ronny; Czaja, Wojciech; Taylor, Gavin; Moeller, Michael; Goldstein, Tom (2020-09-28). Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching. International Conference on Learning Representations 2021 (Poster).
^ El-Mhamdi, El Mahdi; Farhadkhani, Sadegh; Guerraoui, Rachid; Guirguis, Arsany; Hoang, Lê-Nguyên; Rouault, Sébastien (2021-12-06). "Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)". Advances in Neural Information Processing Systems. 34. arXiv:2008.00742.
^ Tramèr, Florian; Zhang, Fan; Juels, Ari; Reiter, Michael K.; Ristenpart, Thomas (2016). Stealing Machine Learning Models via Prediction {APIs}. 25th USENIX Security Symposium. pp. 601–618. ISBN 978-1-931971-32-4.

[1] Kianpour, Mazaher; Wen, Shao-Fang (2020). "Timing Attacks on Machine Learning: State of the Art". Intelligent Systems and Applications. Advances in Intelligent Systems and Computing. Vol. 1037. pp. 111–125. doi:10.1007/978-3-030-29516-5_10. ISBN 978-3-030-29515-8. S2CID 201705926.

[:1-2] Siva Kumar, Ram Shankar; Nyström, Magnus; Lambert, John; Marshall, Andrew; Goertzel, Mario; Comissoneru, Andi; Swann, Matt; Xia, Sharon (May 2020). "Adversarial Machine Learning-Industry Perspectives". 2020 IEEE Security and Privacy Workshops (SPW). pp. 69–75. doi:10.1109/SPW50608.2020.00028. ISBN 978-1-7281-9346-5. S2CID 229357721.

[GoodfellowMcDaniel20182-3] Goodfellow, Ian; McDaniel, Patrick; Papernot, Nicolas (25 June 2018). "Making machine learning robust against adversarial inputs". Communications of the ACM. 61 (7): 56–66. doi:10.1145/3134599. ISSN 0001-0782.^{[permanent dead link]}

[4] Geiping, Jonas; Fowl, Liam H.; Huang, W. Ronny; Czaja, Wojciech; Taylor, Gavin; Moeller, Michael; Goldstein, Tom (2020-09-28). Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching. International Conference on Learning Representations 2021 (Poster).

[:13-5] El-Mhamdi, El Mahdi; Farhadkhani, Sadegh; Guerraoui, Rachid; Guirguis, Arsany; Hoang, Lê-Nguyên; Rouault, Sébastien (2021-12-06). "Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)". Advances in Neural Information Processing Systems. 34. arXiv:2008.00742.

[6] Tramèr, Florian; Zhang, Fan; Juels, Ari; Reiter, Michael K.; Ristenpart, Thomas (2016). Stealing Machine Learning Models via Prediction {APIs}. 25th USENIX Security Symposium. pp. 601–618. ISBN 978-1-931971-32-4.

[1]

[2]

[3]

[4]

[5]

[6]