LEAD: Learning-Enabled Assistive Driving: Formal Assurances during Operation and Training
Current driver-assist systems are the epitomy
of CPS as they integrate the physical and digital domains by
incorporating a myriad of Electronic Control Units (ECUs) running
sophisticated planning and control algorithms, so much so that they
are aptly considered as “computers on wheels”. With the
proliferation of data-driven algorithms,
future self-driving vehicles will incorporate even more advanced
technologies, such as four-wheel independent steering,
drive-by-wire, brake-by-wire, perception modules, autonomous
navigation, lane-change, platooning, etc. and will be able to solve
many critical tasks such as perception, decision making, and
adaptive control in unstructured environments and move towards
complete autonomy. However, due to the fragility of present machine
learning algorithms in adapting to unseen environments or
unaccounted contextual situations, human intervention is still
necessary. What is not clear is how to align the decisions taken by
the vehicle’s autonomous or active safety control system with the
human’s intentions, goals, and skill set. This is paramount for
passenger vehicles operating in an “information-rich” world that
require the smooth and safe interaction between machine, computer,
and human at all levels: the driver, the vehicle, and the traffic.
While the modeling and control aspects at each level have received
their due attention in the literature, we argue that the interaction
between these three levels demarcates the cyber-physical nature of
the problem
Current driving autonomy solutions leverage advanced machine learning algorithms to perform challenging pattern recognition, planning and high-level decision tasks. In particular, reinforcement learning (RL) has proved successful in learning strategies and tasks, often out-performing humans. At the same time, these RL algorithms must align the agent’s goals with the human driver without sacrificing safety. Such interactions are vital for safe operation, yet they have not been deeply explored thus far.
There is currently a need to quantify the impact of the human driver within the autonomy loop, both from an individual experiential perspective, as well as in terms of safety. In this research, we propose to increase the performance and safety guarantees of deep neural network architectures operating within a feedback loop that includes the driver by: a) using redundant architectures that blend model-free and model-based processing pipelines, and b) by adding safety guarantees both during training and during execution by leveraging recent advances of formal methods for safety-critical applications..
The proposed effort makes fundamental contributions to the challenging problem of safe operation of (deep) neural network-based learning architectures both during training and during execution. Our methodology is based on four key novel ingredients that guide our technical approach. The first ingredient is the introduction of a novel deep NN architecture that combines both model-based representation sensor measurements to enhance robustness during training. The second ingredient is the incorporation of signal temporal logic (STL) specifications within the proposed DNN architecture via the introduction of a bi-directional recurrent neural network (RNN) to ensure safety during training of learning-enabled advanced driver assist systems (ADAS), while also considering the driver’s habits and driving skills. The third ingredient is the development of new learning-from-demonstration techniques that enable population-based training of controllers to perform a context- or task-specific blending of individual, heterogeneous driver policies. Finally, we utilize efficient reachable set approximations to formulate an optimization-based runtime assurance (RTA) mechanism to ensure the satisfaction of the STL specifications during execution via the use of the theory of mixed monotonicity for efficient reachable set calculations of NNs.
The project is in collaboration with Sam Coogan and Matthew Gombolay.
Please also visit our VIP project website.
Sponsors
This project is supported by NSF.
Selected Publications
- S. Jafarpour, A. Harapanahalli, S. Coogan, "Interval Reachability of Nonlinear Dynamical Systems with Neural Network Controllers", 5th Annual Learning for Dynamics & Control Conference, Accepted, 2023. https://arxiv.org/abs/2301.07912
- A. Harapanahalli, S. Jafarpour, S. Coogan, "Contraction-Guided Adaptive Partitioning for Reachability Analysis of Neural Network Controlled Systems", IEEE Conference on Decision and Control, Submitted, 2023. https://arxiv.org/abs/2304.03671
- Yin, J., Dawson, C., Fan, C., and Tsiotras, P., “Shield Model Predictive Path Integral: A Computationally Efficient Robust MPC Approach Using Control Barrier Functions,'' IEEE Robotics and Automation Letters}, Vol. 8, No. 11, pp. 7106-7113, 2023. Doi: 10.1109/LRA.2023.3315211
- Baird, Luke, Akash Harapanahalli, and Samuel Coogan. "Interval Signal Temporal Logic from Natural Inclusion Functions." IEEE Control Systems Letters (2023). Doi: 10.1109/LCSYS.2023.3337744
- Harapanahalli, Akash, Saber Jafarpour, and Samuel Coogan. "Forward Invariance in Neural Network Controlled Systems." IEEE Control Systems Letters (2023). DOI: 10.1109/LCSYS.2023.3341980
- Pak, Andrey, Hemanth Manjunatha, Dimitar Filev, and Panagiotis Tsiotras. "CARNet: A Dynamic Autoencoder for Learning Latent Dynamics in Autonomous Driving Tasks." arXiv preprint arXiv:2205.08712 (2022).
- Manjunatha, Hemanth, Andrey Pak, and Panagiotis Tsiotras (2022). "Improving Autonomous Driving Policy Generalization via Neural Network Over-Parameterization.". 1th Workshop on Safe Learning for Autonomous Driving (SL4AD) International Conference on Machine Learning.
- Manjunatha, H., Ghanei, M., Pak, A., and Tsiotras, P., ``Improving Autonomous Driving Policy Generalization via Auxiliary Tasks and Latent Modeling,'' 5th Multi-disciplinary Conference on Reinforcement Learning and Decision Making, Providence, RI, June 8--11, 2022
- Chen, Letian, Sravan Jayanthi, Rohan Paleja, Daniel Martin, Viacheslav Zakharov, and Matthew Gombolay. "Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations." arXiv preprint arXiv:2209.11908 (2023). 6th Conference on Robot Learning (CoRL 2022), Auckland, New Zealand.
- Yang, Yue, Letian Chen, and Matthew Gombolay."Safe Inverse Reinforcement Learning via Control Barrier Function." arXiv preprint arXiv:2212.02753 (2022)
- Yang, Yue, Letian Chen, Zulfiqar Zaidi, Sanne van Waveren, Arjun Krishna, and Matthew Gombolay. "Enhancing Safety in Learning from Demonstration Algorithms via Control Barrier Function Shielding." In Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, pp. 820-829. 2024. https://doi.org/10.1145/3610977.3635002
- Yin, J., Zhang, Z., and Tsiotras, P., “Risk-Aware Model Predictive Path Integral Using Conditional Value-at-Risk,'' International Conference on Robotics and Automation, London, UK, May 29--June 2, 2023, pp. 7937-7943, doi: 10.1109/ICRA48891.2023.10161100
- 3. Yin, J., Dawson, C., Fan, C., and Tsiotras, P., “Shield Model Predictive Path Integral: A Computationally Efficient Robust MPC Approach Using Control Barrier Functions,'' International Conference on Robotics and Automation, Yokohama, Japan, May 13--17, 2024.
- Harapanahalli, Akash, Saber Jafarpour, and Samuel Coogan. "Contraction-guided adaptive partitioning for reachability analysis of neural network controlled systems." 2023 62nd IEEE Conference on Decision and Control (CDC). IEEE, 2023. Doi: 10.1109/CDC49753.2023.10383360
- Harapanahalli, Akash, Saber Jafarpour, and Samuel Coogan. "immrax: A Parallelizable and Differentiable Toolbox for Interval Analysis and Mixed Monotone Reachability in JAX." IFAC-PapersOnLine 58.11 (2024): 75-80. Doi: 10.1016/j.ifacol.2024.07.428
- Harapanahalli, Akash, and Samuel Coogan. "Efficient Reachable Sets on Lie Groups Using Lie Algebra Monotonicity and Tangent Intervals." Control and Decision Conference, (2024). Doi: 10.1109/CDC56724.2024.10886065
- Yin, J., Tsiotras, P., and Berntorp, K., “Chance-Constrained Information-Theoretic Stochastic Model Predictive Control with Safety Shielding,'' 63rd IEEE Conference on Decision and Control, Milan, Italy, Dec. 16-19, 2024, pp. 653-658, doi:10.1109/CDC56724.2024.10885840
- Parashar, A., Yin, J., Dawson, C., Tsiotras, P., and Fan, C., “Learning-based Bayesian Inference for Testing of Autonomous Systems,'' IEEE International Conference on Robotics and Automation, Atlanta, GA, May 19-23, 2025, doi:10.1109/LRA.2024.3455782
- Sivaramakrishnan, V., Kalagarla,K. C., Devonport, R., Pilipovsky, J., Tsiotras, P., and Oishi, M., “SAVER: A Toolbox for Sampling-Based, Probabilistic Verification of Neural Networks,'' Hybrid Systems: Computation and Control, Irvine, CA, May 6-9, 2025. doi: 10.1145/3716863.371804
- Yin, J., So, O., Yang E. Y., Fan, C., and Tsiotras, P., “Safe Beyond the Horizon: Efficient Sampling-based MPC with Neural Control Barrier Functions,'' Robotics: Science and Systems, Los Angeles, CA, June 21-25, 2025