Access the full text.
Sign up today, get DeepDyve free for 14 days.
Daniel J. Fremont, Johnathan Chiu, Dragos D. Margineantu, Denis Osipychev, Sanjit A. Seshia (2020)
Formal analysis and redesign of a neural network-based aircraft taxiing system with VerifAI32nd Intl. Conf. on CAV.
Nilesh Dalvi, Pedro Domingos, Mausam, Sumit Sanghai, D. Verma (2004)
Adversarial classificationProceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining
Daniel Fremont, Johnathan Chiu, D. Margineantu, Denis Osipychev, S. Seshia (2020)
Formal Analysis and Redesign of a Neural Network-Based Aircraft Taxiing System with VerifAIComputer Aided Verification, 12224
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, P. Frossard (2015)
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, D. Erhan, I. Goodfellow, R. Fergus (2013)
Intriguing properties of neural networksCoRR, abs/1312.6199
Thomas A. Henzinger, Jean-François Raskin (2000)
Robust undecidability of timed and hybrid systemsIntl. Workshop on HSCC. Springer
Shakiba Yaghoubi, Georgios Fainekos (2018)
Gray-box adversarial testing for control systems with machine learning componentsProceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control
T. Henzinger, Jean-François Raskin (2000)
Robust Undecidability of Timed and Hybrid Systems
Liren Yang, Xiaofan Cui, A. Avestruz, N. Ozay (2019)
Correct-by-construction control synthesis for buck converters with event-triggered state measurement2019 American Control Conference (ACC)
J. Lasserre (2014)
Tractable approximations of sets defined with quantifiersMathematical Programming, 151
Masaki Waga (2020)
Falsification of cyber-physical systems with robustness-guided black-box checkingProceedings of the 23rd International Conference on Hybrid Systems: Computation and Control
Petter Nilsson, Omar Hussien, Ayca Balkan, Yuxiao Chen, Aaron D. Ames, Jessy W. Grizzle, Necmiye Ozay, Huei Peng, Paulo Tabuada (2015)
Correct-by-construction adaptive cruise control: Two approachesIEEE TCST, 24
R. Majumdar, N. Ozay, Anne-Kathrin Schmuck (2020)
On abstraction-based controller design with output feedbackProceedings of the 23rd International Conference on Hybrid Systems: Computation and Control
G. Ernst, Sean Sedwards, Zhenya Zhang, I. Hasuo (2018)
Fast Falsification of Hybrid Systems using Probabilistically Adaptive InputArXiv, abs/1812.04159
Xiao Wang, Saasha Nair, Matthias Althoff (2020)
Falsification-Based Robust Adversarial Reinforcement Learningarxiv:2007.00691 (2020).
Sarah Dean, Andrew Taylor, Ryan Cosner, B. Recht, A. Ames (2020)
Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functions
T. Dreossi, Daniel Fremont, Shromona Ghosh, Edward Kim, H. Ravanbakhsh, Marcell Vazquez-Chanlatte, S. Seshia (2019)
VerifAI: A Toolkit for the Formal Design and Analysis of Artificial Intelligence-Based Systems
E. Plaku, L. Kavraki, Moshe Vardi (2009)
Falsification of LTL safety properties in hybrid systemsInternational Journal on Software Tools for Technology Transfer, 15
A. Bose, Gauthier Gidel, Hugo Berrard, Andre Cianflone, Pascal Vincent, Simon Lacoste-Julien, William Hamilton (2020)
Adversarial Example GamesArXiv, abs/2007.00720
A. Dosovitskiy, G. Ros, Felipe Codevilla, Antonio López, V. Koltun (2017)
CARLA: An Open Urban Driving Simulator
Petter Nilsson, Omar Hussien, Ayca Balkan, Yuxiao Chen, A. Ames, J. Grizzle, N. Ozay, H. Peng, P. Tabuada (2016)
Correct-by-Construction Adaptive Cruise Control: Two ApproachesIEEE Transactions on Control Systems Technology, 24
A. Bhatia, Emilio Frazzoli (2004)
Incremental Search Methods for Reachability Analysis of Continuous and Hybrid Systems
Shromona Ghosh, H. Ravanbakhsh, S. Seshia (2019)
Counterexample-Guided Synthesis of Perception Models and Control2021 American Control Conference (ACC)
Yashwanth Annpureddy, Che Liu, Georgios Fainekos, Sriram Sankaranarayanan (2011)
S-taliro: A tool for temporal logic falsification for hybrid systemsIntl. Conf. on TACAS. Springer
Yinan Li, Jun Liu (2018)
Invariance Control Synthesis for Switched Nonlinear Systems: An Interval Analysis ApproachIEEE Transactions on Automatic Control, 63
Xiao Wang, Saasha Nair, M. Althoff (2020)
Falsification-Based Robust Adversarial Reinforcement Learning2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA)
M. Herceg, M. Kvasnica, C. Jones, M. Morari (2013)
Multi-Parametric Toolbox 3.02013 European Control Conference (ECC)
Yashwanth Annpureddy, Che Liu, Georgios Fainekos, S. Sankaranarayanan (2011)
S-TaLiRo: A Tool for Temporal Logic Falsification for Hybrid Systems
Liren Yang, N. Ozay (2020)
Efficient Safety Control Synthesis with Imperfect State Information2020 59th IEEE Conference on Decision and Control (CDC)
Dimitri Bertsekas (1972)
Infinite time reachability of state-space regions by using feedback controlIEEE TAC, 17
Tarik Nahhal, Thao Dang (2007)
Test coverage for continuous and hybrid systemsIntl. Conf. on CAV. Springer
Yinan Li, Jun Liu (2017)
Invariance control synthesis for switched nonlinear systems: An interval analysis approachIEEE TAC, 63
Xin Chen, E. Ábrahám, S. Sankaranarayanan (2013)
Flow*: An Analyzer for Non-linear Hybrid Systems
Shromona Ghosh, Hadi Ravanbakhsh, Sanjit A. Seshia (2019)
Counterexample-guided synthesis of perception models and controlarxiv:1911.01523 (2019).
Tarik Nahhal, T. Dang (2007)
Test Coverage for Continuous and Hybrid Systems
Sadra Sadraddini, Russ Tedrake (2019)
Linear Encodings for Polytope Containment Problems2019 IEEE 58th Conference on Decision and Control (CDC)
A. Gleave, Michael Dennis, Neel Kant, Cody Wild, S. Levine, Stuart Russell (2019)
Adversarial Policies: Attacking Deep Reinforcement LearningArXiv, abs/1905.10615
Anand Balakrishnan, Aniruddh Puranic, Xin Qin, Adel Dokhanchi, Jyotirmoy Deshmukh, H. Amor, Georgios Fainekos (2019)
Specifying and Evaluating Quality Metrics for Vision-based Perception Systems2019 Design, Automation & Test in Europe Conference & Exhibition (DATE)
Ian Mitchell, A. Bayen, C. Tomlin (2005)
A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic gamesIEEE Transactions on Automatic Control, 50
Erion Plaku, Lydia E Kavraki, Moshe Y. Vardi (2009)
Falsification of LTL safety properties in hybrid systemsIntl. Conf. on TACAS. Springer
I. Goodfellow, Jonathon Shlens, Christian Szegedy (2014)
Explaining and Harnessing Adversarial ExamplesCoRR, abs/1412.6572
Glen Chou, Yunus Sahin, Liren Yang, Kwesi Rutledge, Petter Nilsson, N. Ozay (2018)
Using Control Synthesis to Generate Corner Cases: A Case Study on Autonomous DrivingIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 37
Jyotirmoy Deshmukh, Xiaoqing Jin, J. Kapinski, O. Maler (2015)
Stochastic Local Search for Falsification of Hybrid Systems
Kexin Pei, Yinzhi Cao, Junfeng Yang, S. Jana (2017)
DeepXplore: Automated Whitebox Testing of Deep Learning SystemsProceedings of the 26th Symposium on Operating Systems Principles
Jernej Kos, D. Song (2017)
Delving into adversarial attacks on deep policiesArXiv, abs/1705.06452
Shromona Ghosh, Felix Berkenkamp, G. Ranade, S. Qadeer, Ashish Kapoor (2018)
Verifying Controllers Against Adversarial Examples with Bayesian Optimization2018 IEEE International Conference on Robotics and Automation (ICRA)
Alexandre Donzé (2010)
Breach, A Toolbox for Verification and Parameter Synthesis of Hybrid Systems
Sandy Huang, Nicolas Papernot, I. Goodfellow, Yan Duan, P. Abbeel (2017)
Adversarial Attacks on Neural Network PoliciesArXiv, abs/1702.02284
Cumhur Tuncali, Georgios Fainekos, Hisahiro Ito, J. Kapinski (2018)
Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components2018 IEEE Intelligent Vehicles Symposium (IV)
S. Levine, Chelsea Finn, Trevor Darrell, P. Abbeel (2015)
End-to-End Training of Deep Visuomotor PoliciesJ. Mach. Learn. Res., 17
Jyotirmoy Deshmukh, Marko Horvat, Xiaoqing Jin, R. Majumdar, Vinayak Prabhu (2017)
Testing Cyber-Physical Systems through Bayesian OptimizationACM Transactions on Embedded Computing Systems (TECS), 16
M. Wulf, L. Doyen, Jean-François Raskin (2006)
A Lattice Theory for Solving Games of Imperfect Information
Glen Chou, Yunus Emre Sahin, Liren Yang, Kwesi J. Rutledge, Petter Nilsson, Necmiye Ozay (2018)
Using control synthesis to generate corner cases: A case study on autonomous drivingIEEE TCAD, 37
T. Dreossi, Alexandre Donzé, S. Seshia (2017)
Compositional Falsification of Cyber-Physical Systems with Machine Learning ComponentsJournal of Automated Reasoning, 63
F. Clarke, R. Stern, G. Sabidussi (1999)
Nonlinear Analysis, Differential Equations and Control
Eduardo Sontag (1999)
Stability and stabilization: discontinuities and the effect of disturbancesarXiv: Optimization and Control
Ian M. Mitchell, Alexandre M. Bayen, Claire J. Tomlin (2005)
A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic gamesIEEE TAC, 50
Jongwoo Kim, J. Esposito, Vijay Kumar (2005)
An RRT-Based Algorithm for Testing and Validating Multi-Robot Controllers
D. Bertsekas (1972)
Infinite time reachability of state-space regions by using feedback controlIEEE Transactions on Automatic Control, 17
Alexandre Donzé, O. Maler (2010)
Robust Satisfaction of Temporal Logic over Real-Valued Signals
Alexandre Donzé, Oded Maler (2010)
Robust satisfaction of temporal logic over real-valued signalsIntl. Conf. on FORMATS. Springer
In this paper, we study feedback dynamical systems with memoryless controllers under imperfect information. We develop an algorithm that searches for “adversarial scenarios”, which can be thought of as the strategy for the adversary representing the noise and disturbances, that lead to safety violations. The main challenge is to analyze the closed-loop system's vulnerabilities with a potentially complex or even unknown controller in the loop. As opposed to commonly adopted approaches that treat the system under test as a black-box, we propose a synthesis-guided approach, which leverages the knowledge of a plant model at hand. This hence leads to a way to deal with gray-box systems (i.e., with known plant and unknown controller). Our approach reveals the role of the imperfect information in the violation. Examples show that our approach can find non-trivial scenarios that are difficult to expose by random simulations. This approach is further extended to incorporate model mismatch and to falsify vision-in-the-loop systems against finite-time reach-avoid specifications.
ACM Transactions on Embedded Computing Systems (TECS) – Association for Computing Machinery
Published: Sep 22, 2021
Keywords: Adversarial scenarios
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.