Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/association-for-computing-machinery/verifying-the-safety-of-autonomous-systems-with-neural-network-dJ4V5oZbU8
References (64)
-
OpenAI. [n
OpenAI GymRetrieved from https://gym.openai.com.
-
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, D. Erhan, I. Goodfellow, R. Fergus (2013)
Intriguing properties of neural networksCoRR, abs/1312.6199
-
A. Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu (2017)
Towards Deep Learning Models Resistant to Adversarial AttacksArXiv, abs/1706.06083
-
Matthias Hein, Maksym Andriushchenko (2017)
Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation -
Diego Lopez, Patrick Musau, Hoang-Dung Tran, Souradeep Dutta, Taylor Carpenter, Radoslav Ivanov, Taylor Johnson (2022)
ARCH-COMP22 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems PlantsEPiC Series in Computing
-
Shiqi Wang, Kexin Pei, J. Whitehouse, Junfeng Yang, S. Jana (2018)
Efficient Formal Safety Analysis of Neural NetworksArXiv, abs/1809.08098
-
G. Sacks (2003)
A DECISION METHOD FOR ELEMENTARY ALGEBRA AND GEOMETRY -
Radoslav Ivanov, Taylor Carpenter, James Weimer, R. Alur, George Pappas, Insup Lee (2019)
Case study: verifying the safety of an autonomous racing car with a neural network controllerProceedings of the 23rd International Conference on Hybrid Systems: Computation and Control
-
Hoang-Dung Tran, Stanley Bak, Weiming Xiang, Taylor Johnson (2020)
Verification of Deep Convolutional Neural Networks Using ImageStarsComputer Aided Verification, 12224
-
Chao Huang, Jiameng Fan, Wenchao Li, Xin Chen, Qi Zhu (2019)
ReachNNACM Transactions on Embedded Computing Systems (TECS), 18
-
Timon Gehr, M. Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, Martin Vechev (2018)
AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation2018 IEEE Symposium on Security and Privacy (SP)
-
Tsui-Wei Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, D. Boning, I. Dhillon, L. Daniel (2018)
Towards Fast Computation of Certified Robustness for ReLU NetworksArXiv, abs/1804.09699
-
Moustapha Cissé, Piotr Bojanowski, Edouard Grave, Yann Dauphin, Nicolas Usunier (2017)
Parseval Networks: Improving Robustness to Adversarial ExamplesArXiv, abs/1704.08847
-
Weiming Xiang, Hoang-Dung Tran, Taylor Johnson (2017)
Output Reachable Set Estimation and Verification for Multilayer Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 29
-
Rüdiger Ehlers (2017)
Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks -
T. Lillicrap, Jonathan Hunt, A. Pritzel, N. Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra (2015)
Continuous control with deep reinforcement learningCoRR, abs/1509.02971
-
Xin Chen, Erika Abraham, Sriram Sankaranarayanan (2012)
Taylor model flowpipe construction for non-linear hybrid systemsProceedings of the 2012 IEEE 33rd Real-Time Systems Symposium (RTSS’12). IEEE, 2012
-
Mahyar Fazlyab, Alexander Robey, Hamed Hassani, M. Morari, George Pappas (2019)
Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks -
Weiming Xiang, Patrick Musau, A. Wild, Diego Lopez, Nathaniel Hamilton, Xiaodong Yang, Joel Rosenfeld, Taylor Johnson (2018)
Verification for Machine Learning, Autonomy, and Neural Networks SurveyArXiv, abs/1810.01989
-
G. Lafferriere, George Pappas, S. Yovine (1999)
A New Class of Decidable Hybrid Systems -
K. Makino, M. Berz (2007)
SUPPRESSION OF THE WRAPPING EFFECT BY TAYLOR MODEL - BASED VERIFIED INTEGRATORS: THE SINGLE STEP -
Gurobi Optimization
[nGurobi Optimizer. Retrieved from https://gurobi.com.
-
Cumhur Erkan Tuncali, Georgios Fainekos, Hisahiro Ito, James Kapinski (2018)
Simulation-based adversarial test generation for autonomous vehicles with machine learning componentsArXiv:1804.06760. Retrieved from https://arxiv.org/abs/1804.06760.
-
T. Dreossi, A. Donzé, S. A. Seshia (2017)
Compositional falsification of cyber-physical systems with machine learning componentsProceedings of the NASA Formal Methods Symposium. Springer
-
R. Rajamani (2012)
Lateral Vehicle Dynamics -
Alexey Kurakin, I. Goodfellow, Samy Bengio (2016)
Adversarial examples in the physical worldArXiv, abs/1607.02533
-
A. Wilkie (1997)
Schanuel’s Conjecture and the Decidability of the Real Exponential Field -
K. D. Julian, J. Lopez, J. S. Brush, M. P. Owen, M. J. Kochenderfer (2016)
Policy compression for aircraft collision avoidance systemsProceedings of the 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC’16). IEEE, 2016
-
Scott Fujimoto, H. Hoof, D. Meger (2018)
Addressing Function Approximation Error in Actor-Critic Methods -
Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato, Lior Wolf (2014)
DeepFace: Closing the Gap to Human-Level Performance in Face Verification2014 IEEE Conference on Computer Vision and Pattern Recognition
-
Mariusz Bojarski, D. Testa, Daniel Dworakowski, Bernhard Firner, B. Flepp, Prasoon Goyal, L. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, Karol Zieba (2016)
End to End Learning for Self-Driving CarsArXiv, abs/1604.07316
-
Parasara Duggirala, S. Mitra, Mahesh Viswanathan, Matthew Potok (2015)
C2E2: A Verification Tool for Stateflow Models -
Kevin Eykholt, I. Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, D. Song (2018)
Robust Physical-World Attacks on Deep Learning Visual Classification2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
-
Xin Chen, E. Ábrahám, S. Sankaranarayanan (2013)
Flow*: An Analyzer for Non-linear Hybrid Systems -
H. Tran, S. Bak, W. Xiang, T. T. Johnson (2020)
Verification of deep convolutional neural networks using ImageStarsProceedings of the 32nd International Conference on Computer-Aided Verification (CAV’20). Springer.
-
G. Erdogan, L. Alexander, R. Rajamani (2011)
Closed-loop snowplow applicator control using road condition measurementsVehicle System Dynamics, 49
-
Soonho Kong, Sicun Gao, Wei Chen, E. Clarke (2015)
dReach: δ-Reachability Analysis for Hybrid Systems -
Hoang-Dung Tran, Feiyang Cai, Diego Lopez, Patrick Musau, Taylor Johnson, X. Koutsoukos (2019)
Safety Verification of Cyber-Physical Systems with Reinforcement Learning ControlACM Transactions on Embedded Computing Systems (TECS), 18
-
Radoslav Ivanov, James Weimer, R. Alur, George Pappas, Insup Lee (2018)
Verisig: verifying safety properties of hybrid systems with neural network controllersProceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control
-
Philip Polack, Florent Altché, B. d'Andréa-Novel, A. Fortelle (2017)
The kinematic bicycle model: A consistent model for planning feasible trajectories for autonomous vehicles?2017 IEEE Intelligent Vehicles Symposium (IV)
-
Guy Katz, Clark Barrett, D. Dill, Kyle Julian, Mykel Kochenderfer (2017)
Reluplex: An Efficient SMT Solver for Verifying Deep Neural NetworksArXiv, abs/1702.01135
-
Vicenç Royo, David Fridovich-Keil, Sylvia Herbert, C. Tomlin (2018)
Classification-based Approximate Reachability with Guarantees Applied to Safe Trajectory TrackingArXiv, abs/1803.03237
-
T. Gehr, M. Mirman, D. Drachsler-Cohen, P. Tsankov, S. Chaudhuri, M. Vechev (2018)
AI2: Safety and robustness certification of neural networks with abstract interpretationProceedings of the IEEE Symposium on Security and Privacy (SP’18).
-
Souradeep Dutta, Susmit Jha, S. Sankaranarayanan, A. Tiwari (2018)
Output Range Analysis for Deep Feedforward Neural Networks -
Kexin Pei, Yinzhi Cao, Junfeng Yang, S. Jana (2017)
DeepXplore: Automated Whitebox Testing of Deep Learning SystemsProceedings of the 26th Symposium on Operating Systems Principles
-
OpenAI -
R. Alur, C. Courcoubetis, N. Halbwachs, T. Henzinger, Pei-Hsin Ho, X. Nicollin, A. Olivero, J. Sifakis, S. Yovine (1995)
The Algorithmic Analysis of Hybrid SystemsTheor. Comput. Sci., 138
-
Prajit Ramachandran, Barret Zoph, Quoc Le (2018)
Searching for Activation FunctionsArXiv, abs/1710.05941
-
M. Althoff (2015)
An Introduction to CORA 2015 -
Cumhur Tuncali, Georgios Fainekos, Hisahiro Ito, J. Kapinski (2018)
Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components2018 IEEE Intelligent Vehicles Symposium (IV)
-
Xin Chen, E. Ábrahám, S. Sankaranarayanan (2012)
Taylor Model Flowpipe Construction for Non-linear Hybrid Systems2012 IEEE 33rd Real-Time Systems Symposium
-
Gurobi Optimization -
Xiaowu Sun, Haitham Khedr, Yasser Shoukry (2018)
Formal verification of neural network controlled autonomous systemsProceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control
-
Souradeep Dutta, Xin Chen, S. Sankaranarayanan (2019)
Reachability analysis for neural feedback systems using regressive polynomial rule inferenceProceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control
-
A. Moore (1990)
Efficient memory-based learning for robot control -
M. Mirman, Timon Gehr, Martin Vechev (2018)
Differentiable Abstract Interpretation for Provably Robust Neural Networks -
Goran Frehse, C. Guernic, Alexandre Donzé, Scott Cotton, Rajarshi Ray, Olivier Lebeltel, Rodolfo Ripado, A. Girard, T. Dang, O. Maler (2011)
SpaceEx: Scalable Verification of Hybrid Systems -
A. Minai, Ronald Williams (1993)
On the derivatives of the sigmoidNeural Networks, 6
-
T. Dreossi, Alexandre Donzé, S. Seshia (2017)
Compositional Falsification of Cyber-Physical Systems with Machine Learning ComponentsJournal of Automated Reasoning, 63
-
Chao Huang, Jiameng Fan, Wenchao Li, Xin Chen, Qi Zhu (2019)
ReachNN: Reachability analysis of neural-network controlled systemsACM Trans. Embedd. Comput. Syst., 18
-
F1/10 Autonomous Racing Competition -
Kyle Julian, Jessica Lopez, J. Brush, Michael Owen, Mykel Kochenderfer (2016)
Policy compression for aircraft collision avoidance systems2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC)
-
R. Rajamani (2005)
Vehicle dynamics and control -
Ronan Collobert, J. Weston, L. Bottou, Michael Karlen, K. Kavukcuoglu, P. Kuksa (2011)
Natural Language Processing (Almost) from ScratchArXiv, abs/1103.0398
- Publisher
- Association for Computing Machinery
- Copyright
- Copyright © 2020 ACM
- ISSN
- 1539-9087
- eISSN
- 1558-3465
- DOI
- 10.1145/3419742
- Publisher site
- See Article on Publisher Site
Abstract
This article addresses the problem of verifying the safety of autonomous systems with neural network (NN) controllers. We focus on NNs with sigmoid/tanh activations and use the fact that the sigmoid/tanh is the solution to a quadratic differential equation. This allows us to convert the NN into an equivalent hybrid system and cast the problem as a hybrid system verification problem, which can be solved by existing tools. Furthermore, we improve the scalability of the proposed method by approximating the sigmoid with a Taylor series with worst-case error bounds. Finally, we provide an evaluation over four benchmarks, including comparisons with alternative approaches based on mixed integer linear programming as well as on star sets.
Journal
ACM Transactions on Embedded Computing Systems (TECS) – Association for Computing Machinery
Published: Dec 7, 2020
Keywords: Neural network verification