AI Seminar Series: Dr. Suchi Saria, Safe and Reliable AI (April 16)
Dr. Suchi Saria presents "Safe and Reliable AI: Overview and Novel Approach Approaches for Learning and Monitoring" at the AI Seminar (April 16, 2021).
The Artificial Intelligence (AI) Seminar is a weekly meeting at the University of Alberta where researchers interested in AI can share their research. Presenters include both local speakers from the University of Alberta and visitors from other institutions. Topics related in any way to artificial intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems, are explored.
Abstract: As the use of machine learning in safety-critical domains becomes widespread, the importance of proactively addressing sources of failure and evaluating model reliability has increased. Achieving this, however, can be difficult because the performance and reliability of ML models are vulnerable to being overly dependent on the "context" (i.e., artifacts specific to the training dataset) on which the model was trained. In this talk, we will overview work in this area over the last five years and describe in more detail two example state-of-the-art approaches tackling challenges in building safe and reliable AI. The first describes causally-inspired learning algorithms which allow model developers to specify potentially problematic changes in context and then learn models which are guaranteed to be stable to these shifts. The second tackles monitoring for safety: to be able to evaluate the stability of a model to changes in setting or population, typically requires applying the model to a large number of independent datasets. Since the cost of collecting such datasets is often prohibitive, we will describe a distributionally robust framework for evaluating this type of robustness using the fixed, available evaluation data.
This talk will be jointly presented by Prof. Suchi Saria and Adarsh Subbaswamy.
Bio: Dr. Suchi Saria is the John C. Malone Associate professor of computer science and statistics at the Whiting School of Engineering and of health policy at the Bloomberg School of Public Health. She is also the founding Research Director of the Malone Center for Engineering in Healthcare at Hopkins and the founder of Bayesian Health, which leverages state-of-the-art machine learning and behavior change expertise to unlock improved patient care outcomes at scale by providing real-time precise, patient-specific, and actionable insights at the point of care.
Recently, Dr. Saria won a grant from the FDA and is collaborating with them in the development of frameworks for the evaluation of safety and reliability of AI. She was named by IEEE Intelligent Systems as Artificial Intelligence’s “10 to Watch” (2015), MIT Technology Review’s ‘35 Innovators under 35’ (2017), World Economic Forum’s Young Global Leader (2018), DARPA Young Faculty Awardee (2016) and a Sloan Research Fellow (2018). She was invited to join the National Academy of Engineering’s Frontiers of Engineering (2017) and the National Academy of Medicine’s Emerging Leaders in Health and Medicine (2018). She has given over 250 invited talks and is on the editorial board of the Journal of Machine Learning Research.
Dr. Suchi Saria presents “Safe and Reliable AI: Overview and Novel Approach Approaches for Learning and Monitoring” at the AI Seminar (April 16, 2021).
The Artificial Intelligence (AI) Seminar is a weekly meeting at the University of Alberta where researchers interested in AI can share their research. Presenters include both local speakers from the University of Alberta and visitors from other institutions. Topics related in any way to artificial intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems, are explored.
Abstract: As the use of machine learning in safety-critical domains becomes widespread, the importance of proactively addressing sources of failure and evaluating model reliability has increased. Achieving this, however, can be difficult because the performance and reliability of ML models are vulnerable to being overly dependent on the “context” (i.e., artifacts specific to the training dataset) on which the model was trained. In this talk, we will overview work in this area over the last five years and describe in more detail two example state-of-the-art approaches tackling challenges in building safe and reliable AI. The first describes causally-inspired learning algorithms which allow model developers to specify potentially problematic changes in context and then learn models which are guaranteed to be stable to these shifts. The second tackles monitoring for safety: to be able to evaluate the stability of a model to changes in setting or population, typically requires applying the model to a large number of independent datasets. Since the cost of collecting such datasets is often prohibitive, we will describe a distributionally robust framework for evaluating this type of robustness using the fixed, available evaluation data.
This talk will be jointly presented by Prof. Suchi Saria and Adarsh Subbaswamy.
Bio: Dr. Suchi Saria is the John C. Malone Associate professor of computer science and statistics at the Whiting School of Engineering and of health policy at the Bloomberg School of Public Health. She is also the founding Research Director of the Malone Center for Engineering in Healthcare at Hopkins and the founder of Bayesian Health, which leverages state-of-the-art machine learning and behavior change expertise to unlock improved patient care outcomes at scale by providing real-time precise, patient-specific, and actionable insights at the point of care.
Recently, Dr. Saria won a grant from the FDA and is collaborating with them in the development of frameworks for the evaluation of safety and reliability of AI. She was named by IEEE Intelligent Systems as Artificial Intelligence’s “10 to Watch” (2015), MIT Technology Review’s ‘35 Innovators under 35’ (2017), World Economic Forum’s Young Global Leader (2018), DARPA Young Faculty Awardee (2016) and a Sloan Research Fellow (2018). She was invited to join the National Academy of Engineering’s Frontiers of Engineering (2017) and the National Academy of Medicine’s Emerging Leaders in Health and Medicine (2018). She has given over 250 invited talks and is on the editorial board of the Journal of Machine Learning Research.
