Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Worst-case Satisfaction of STL Specifications Using Feedforward Neural Network Controllers

Worst-case Satisfaction of STL Specifications Using Feedforward Neural Network Controllers In this paper, a reinforcement learning approach for designing feedback neural network controllers for nonlinear systems is proposed. Given a Signal Temporal Logic (STL) specification which needs to be satisfied by the system over a set of initial conditions, the neural network parameters are tuned in order to maximize the satisfaction of the STL formula. The framework is based on a max-min formulation of the robustness of the STL formula. The maximization is solved through a Lagrange multipliers method, while the minimization corresponds to a falsification problem. We present our results on a vehicle and a quadrotor model and demonstrate that our approach reduces the training time more than 50 percent compared to the baseline approach. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Embedded Computing Systems (TECS) Association for Computing Machinery

Worst-case Satisfaction of STL Specifications Using Feedforward Neural Network Controllers

Download PDF
20 pages

Loading next page...
 
/lp/association-for-computing-machinery/worst-case-satisfaction-of-stl-specifications-using-feedforward-neural-2sNrH8KNtC

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2019 ACM
ISSN
1539-9087
eISSN
1558-3465
DOI
10.1145/3358239
Publisher site
See Article on Publisher Site

Abstract

In this paper, a reinforcement learning approach for designing feedback neural network controllers for nonlinear systems is proposed. Given a Signal Temporal Logic (STL) specification which needs to be satisfied by the system over a set of initial conditions, the neural network parameters are tuned in order to maximize the satisfaction of the STL formula. The framework is based on a max-min formulation of the robustness of the STL formula. The maximization is solved through a Lagrange multipliers method, while the minimization corresponds to a falsification problem. We present our results on a vehicle and a quadrotor model and demonstrate that our approach reduces the training time more than 50 percent compared to the baseline approach.

Journal

ACM Transactions on Embedded Computing Systems (TECS) – Association for Computing Machinery

Published: Oct 8, 2019

Keywords: Reinforcement learning

References

  • Safe at any speed: A simulation-based test harness for autonomous vehicles
    Abbas, Houssam; O’Kelly, Matthew; Rodionova, Alena; Mangharam, Rahul
  • Classification and coverage-based falsification for embedded control systems
    Adimoolam, Arvind; Dang, Thao; Donzé, Alexandre; Kapinski, James; Jin, Xiaoqing
  • An introduction to CORA 2015
    Althoff, Matthias
  • S-taliro: A tool for temporal logic falsification for hybrid systems
    Annpureddy, Yashwanth; Liu, Che; Fainekos, Georgios; Sankaranarayanan, Sriram
  • Specification-based monitoring of cyber-physical systems: A survey on theory, tools and applications
    Bartocci, Ezio; Deshmukh, Jyotirmoy; Donze, Alexandre; Fainekos, Georgios; Maler, Oded; Nivckovic, Dejan; Sankaranarayanan, Sriram
  • Constrained Optimization and Lagrange Multiplier Methods
    Bertsekas, Dimitri P.
  • Reinforcement learning and optimal control
    Bertsekas, Dimitri P.
  • Flow*: An analyzer for non-linear hybrid systems
    Chen, Xin; Ábrahám, Erika; Sankaranarayanan, Sriram
  • Learning-based model predictive control under signal temporal logic specifications
    Cho, Kyunghoon; Oh, Songhwai
  • Trajectory tracking control for robotic vehicles using counterexample guided training of neural networks
    Claviere, Arthur; Dutta, Souradeep; Sankaranarayanan, Sriram
  • Nonlinear model predictive control with neural network optimization for autonomous autorotation of small unmanned helicopters
    Dalamagkidis, Konstantinos; Valavanis, Kimon P; Piegl, Les A.
  • Neural network solution for forward kinematics problem of HEXA parallel robot
    Dehghani, M.; Ahmadi, M.; Khayatian, A.; Eghtesad, M.; Farid, M.
  • PILCO: A model-based and data-efficient approach to policy search
    Deisenroth, Marc; Rasmussen, Carl E.
  • Breach, a toolbox for verification and parameter synthesis of hybrid systems
    Donzé, Alexandre
  • Robust satisfaction of temporal logic over real-valued signals
    Donzé, Alexandre; Maler, Oded
  • Compositional falsification of cyber-physical systems with machine learning components
    Dreossi, Tommaso; Donzé, Alexandre; Seshia, Sanjit A
  • Counterexample-guided data augmentation
    Dreossi, Tommaso; Ghosh, Shromona; Yue, Xiangyu; Keutzer, Kurt; Sangiovanni-Vincentelli, Alberto; Seshia, Sanjit A.
  • Semantic adversarial deep learning
    Dreossi, Tommaso; Jha, Somesh; Seshia, Sanjit A.
  • Reachability analysis for neural feedback systems using regressive polynomial rule inference
    Dutta, Souradeep; Chen, Xin; Sankaranarayanan, Sriram
  • Output range analysis for deep neural networks
    Dutta, Souradeep; Jha, Susmit; Sanakaranarayanan, Sriram; Tiwari, Ashish
  • Learning and verification of feedback control systems using feedforward neural networks
    Dutta, Souradeep; Jha, Susmit; Sankaranarayanan, Sriram; Tiwari, Ashish
  • Robustness of temporal logic specifications for continuous-time signals
    Fainekos, Georgios E.; Pappas, George J.
  • SpaceEx: Scalable verification of hybrid systems
    Frehse, Goran; Guernic, Colas Le; Donzé, Alexandre; Cotton, Scott; Ray, Rajarshi; Lebeltel, Olivier; Ripado, Rodolfo; Girard, Antoine; Dang, Thao; Maler, Oded
  • Reduced variance deep reinforcement learning with temporal logic specifications
    Gao, Qitong; Hajinezhad, Davood; Zhang, Yan; Kantaros, Yiannis; Zavlanos, Michael M.
  • An introduction to the use of neural networks in control systems
    Hagan, Martin T; Demuth, Howard B; De Jesús, Orlando
  • Evaluating the CMA evolution strategy on multimodal test functions
    Hansen, Nikolaus; Kern, Stefan
  • Completely derandomized self-adaptation in evolution strategies
    Hansen, Nikolaus; Ostermeier, Andreas
  • Learning an approximate model predictive controller with guarantees
    Hertneck, Michael; Köhler, Johannes; Trimpe, Sebastian; Allgöwer, Frank
  • Multilayer feedforward networks are universal approximators
    Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert
  • Verisig: Verifying safety properties of hybrid systems with neural network controllers
    Ivanov, Radoslav; Weimer, James; Alur, Rajeev; Pappas, George J.; Lee, Insup
  • Neural network guidance for UAVs
    Julian, Kyle D.; Kochenderfer, Mykel J.
  • Policy compression for aircraft collision avoidance systems
    Julian, Kyle D.; Lopez, Jessica; Brush, Jeffrey S.; Owen, Michael P.; Kochenderfer, Mykel J.
  • Nonlinear systems
    Khalil, Hassan K.; Grizzle, Jessy W.
  • Specifying real-time properties with metric temporal logic
    Koymans, Ron
  • Learning neural network policies with guided policy search under unknown dynamics
    Levine, Sergey; Abbeel, Pieter
  • A policy search method for temporal logic specified reinforcement learning tasks
    Li, Xiao; Ma, Yao; Belta, Calin
  • Monitoring temporal properties of continuous signals
    Maler, Oded; Nickovic, Dejan
  • Timestamp temporal logic (TTL) for testing the timing of cyber-physical systems
    Mehrabian, Mohammadreza
  • Guided policy search via approximate mirror descent
    Montgomery, William H; Levine, Sergey
  • Dynamic time warping
    Müller, Meinard
  • Neural network based optimization approach for energy demand prediction in smart grid
    Muralitharan, K.; Sakthivel, Rathinasamy; Vishnuvarthan, R.
  • Smooth operator: Control using the smooth robustness of temporal logic
    Pant, Yash Vardhan; Abbas, Houssam; Mangharam, Rahul
  • Fly-by-logic: Control of multi-drone fleets with temporal logic objectives
    Pant, Yash Vardhan; Abbas, Houssam; Quaye, Rhudii A.; Mangharam, Rahul
  • Understanding the exploding gradient problem
    Pascanu, Razvan; Mikolov, Tomas; Bengio, Yoshua
  • On the difficulty of training recurrent neural networks
    Pascanu, Razvan; Mikolov, Tomas; Bengio, Yoshua
  • Deepxplore: Automated whitebox testing of deep learning systems
    Pei, Kexin; Cao, Yinzhi; Yang, Junfeng; Jana, Suman
  • Model predictive control with signal temporal logic specifications
    Raman, Vasumathi; Donzé, Alexandre; Maasoumy, Mehdi; Murray, Richard M.; Sangiovanni-Vincentelli, Alberto; Seshia, Sanjit A.
  • Reactive synthesis from signal temporal logic specifications
    Raman, Vasumathi; Donzé, Alexandre; Sadigh, Dorsa; Murray, Richard M.; Seshia, Sanjit A.
  • Classification-based approximate reachability with guarantees applied to safe trajectory tracking
    Royo, Vicenc Rubies; Fridovich-Keil, David; Herbert, Sylvia; Tomlin, Claire J.
  • Applications of neural networks in high assurance systems
    Schumann, Johann; Liu, Yan
  • Simulation-based adversarial test generation for autonomous vehicles with machine learning components
    Tuncali, Cumhur Erkan; Fainekos, Georgios; Ito, Hisahiro; Kapinski, James
  • Sampling-based synthesis of maximally-satisfying controllers for temporal logic specifications
    Vasile, Cristian-Ioan; Raman, Vasumathi; Karaman, Sertac
  • Generating dominant strategies for continuous two-player zero-sum games
    Vazquez-Chanlatte, Marcell J.; Ghosh, Shromona; Raman, Vasumathi; Sangiovanni-Vincentelli, Alberto; Seshia, Sanjit A.
  • Information theoretic MPC for model-based reinforcement learning
    Williams, Grady; Wagener, Nolan; Goldfain, Brian; Drews, Paul; Rehg, James M.; Boots, Byron; Theodorou, Evangelos A.
  • Verification for machine learning, autonomy, and neural networks survey
    Xiang, Weiming; Musau, Patrick; Wild, Ayana A.; Lopez, Diego Manzanas; Hamilton, Nathaniel; Yang, Xiaodong; Rosenfeld, Joel; Johnson, Taylor T.
  • Falsification of temporal logic requirements using gradient based local search in space and time
    Yaghoubi, Shakiba; Fainekos, Georgios
  • Gray-box adversarial testing for control systems with machine learning components
    Yaghoubi, Shakiba; Fainekos, Georgios
  • Learning deep control policies for autonomous aerial vehicles with mpc-guided policy search
    Zhang, Tianhao; Kahn, Gregory; Levine, Sergey; Abbeel, Pieter
  • Design of deep neural networks as add-on blocks for improving impromptu trajectory tracking
    Zhou, Siqi; Helwa, Mohamed K.; Schoellig, Angela P.