MAE Seminar – Safety Analysis of AI-enabled Cyber-Physical Systems (CPS): A Formal Approach


12:45 pm-1:45 pm
Add to Outlook/iCal
Add to Google Calendar


Room 450, Wertheim Lab for Engineering Excellence
527 Gale Lemerand Drive
Gainesville, FL 32611


MAE Seminar – Safety Analysis of AI-enabled Cyber-Physical Systems (CPS): A Formal Approach

Monday, April 22, 2024, at 12:50pm, Location: WERT 450

Pavithra Prabhakar
Professor, Peggy and Gary Edwards Chair in Engineering, Kansas State University
Program Director, Software and Hardware Foundations Cluster, Computer and Information Science and Engineering Directorate, National Science Foundation

AI-based components have become an integral part of Cyber-Physical Systems enabling transformative functionalities. With the ubiquitous use of Machine Learning components in perception, control and decision making in safety critical application domains such as automotive and aerospace, rigorous analysis of these systems has become imperative toward real-world deployment. In this talk, we will present a formal approach to verifying the safety of AI-enabled CPS. We consider a closed-loop system consisting of a dynamical system model of the physical system and a neural network model of the perception/control modules and analyze the safety of this system through reachable set computation.

One of the main challenges with reachable set computation of neural network-controlled CPS is the scalability of the methods to large networks and complex dynamics. We present a novel abstraction technique for neural network size reduction that provides soundness guarantees for safety analysis and indicates a promising direction for scalable analysis of the closed loop system. Specifically, our abstraction consists of constructing a simpler neural network with fewer neurons, albeit with interval weights called interval neural network (INN), which over-approximates the output range of the given neural network. We present several methods for computing the output range analysis problem on the INNs, including one which consists of reducing it to solving a mixed integer linear programming problem, and the other that involves a symbolic computation of the reach set using a novel data structure called the interval star set. Our experimental results highlight the trade-off between the computation time and the precision of the computed output set. We will discuss other foundational questions on neural network size reduction by exploring the notion of equivalence and approximate equivalence. We will conclude by pointing to ongoing work on incorporating a camera model along with a neural network for perception in the closed loop system framework.

Pavithra Prabhakar is professor in the department of computer science, and the Peggy and Gary Edwards Chair in Engineering at Kansas State University. She is currently serving the National Science Foundation as a Program Director in the Software and Hardware Foundations Cluster in the Computer and Information Science and Engineering Directorate, where she manages formal methods and verification portfolio. Specifically, she leads the Formal Methods in the Field (FMitF) program, has been a founding program director for the Safe Learning Enabled Systems (SLES) program and is a cognizant program director for the Foundations of Robotics Research (FRR) and the Cyber-Physical Systems (CPS) program. She obtained her doctorate in computer science and a master’s degree in applied mathematics from the University of Illinois at Urbana-Champaign, followed by a CMI postdoctoral fellowship at the California Institute of Technology. Prior to coming to K-State, she spent four years at the IMDEA Software Institute in Spain as a tenure-track assistant professor. She is the recipient of a Marie Curie Career Integration Grant from the European Union (2014), an NSF CAREER Award (2016), an ONR Young Investigator Award (2017), NITW distinguished young alumnus award (2021), and an Amazon Research Award (2022).

MAE Faculty Host: Dr. Yu Wang


Hosted by

Dr. Yu Wang