MAE Affiliate Seminar – Verify-then-Monitor: Calibration Guarantees for Safety Confidence

Details

Verify-then-Monitor: Calibration Guarantees for Safety Confidence

Tuesday, September 19, 2023, at 12:50 pm
Location: In-Person MAE-A, Room 303

Ivan Ruchkin, PhD
Assistant Professor
UF Department of Electrical and Computer Engineering

Abstract
Autonomous cyber-physical systems (CPS) are increasingly deployed in complex and safety-critical environments. To help CPS interact with such environments, learning-enabled components, typically implemented with neural networks, perform perception and control functions. Unfortunately, the complexity of the environments and learning components is a major challenge to ensuring the safety of CPS. An emerging assurance paradigm prescribes two steps: (i) verifying as much of the CPS as possible at design time, and then (ii) monitoring the probability of safety at run time in case of unexpected situations. But how can we guarantee that the monitor produces a probability that is well-calibrated to the true chance of safety? This talk will summarize our recent answers in two settings. The first setting combines Bayesian filtering with probabilistic model checking of Markov decision processes, instantiated in the context of controlling critical infrastructure. The second setting focuses on confidence monitoring of formalized assumptions behind closed-loop neural-network verification in the context of an autonomous underwater vehicle.

Biography
Dr. Ivan Ruchkin is an assistant professor at the Department of Electrical and Computer Engineering of the University of Florida, where he leads the Trustworthy Engineered Autonomy (TEA) Lab. His research makes autonomous systems safer and more trustworthy by combining techniques from formal methods and artificial intelligence. Ivan received his Ph.D. degree in Software Engineering from Carnegie Mellon University and completed his postdoctoral training at the University of Pennsylvania. His contributions were recognized with multiple Best Paper awards, a Gold Medal in the ACM Student Research Competition, and the Frank Anger Memorial Award for the crossover of ideas between the software engineering and embedded systems communities. More information can be found at https://ivan.ece.ufl.edu.

MAE Faculty Host: Yu Wang