Dr. Suchi Saria presents “Safe and Reliable AI: Overview and Novel Approach Approaches for Learning and Monitoring” at the AI Seminar (April 16, 2021). The Artificial Intelligence (AI) Seminar is a weekly meeting at the University of Alberta where researchers interested in AI can share their research. Presenters include both local speakers from the University of Alberta and visitors from other institutions. Topics related in any way to artificial intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems, are explored. Abstract: As the use of machine learning in safety-critical domains becomes widespread, the importance of proactively addressing sources of failure and evaluating model reliability has increased. Achieving this, however, can be difficult because the performance and reliability of ML models are vulnerable to being overly dependent on the “context” (i.e., artifacts specific to the training dataset) on which the model was trained. In this talk, we will overview work in this area over the last five years and describe in more detail two example state-of-the-art approaches tackling challenges in building safe and reliable AI. The first describes causally-inspired learning algorithms which allow model developers to specify potentially problematic changes in context and then learn models which are guaranteed to be stable to these shifts. The second tackles monitoring for safety: to be able to evaluate the stability of a model to changes in setting or population, typically requires applying the model to a large number of independent datasets. Since the cost of collecting [More]