Reinforcement Learning in the Real World: Making Predictions Online for Water Treatment – Oct. 30, 2023

Session Description

Join CANSSI Ontario at the Statistical Sciences Applied Research and Education Seminar (ARES) with Martha White, Associate Professor, Department of Computing Science, University of Alberta.

In this talk I will discuss how we used reinforcement-learning based prediction approaches for a real drinking-water treatment plant. I will first describe this dataset, and highlight challenges with seasonality, non- stationarity, partial observability, and heterogeneity across sensors and operation modes of the plant. I will then explain General Value Function (GVF) predictions—discounted cumulative sums of observations–and highlight why they might be preferable to classical n-horizon predictions common in time series prediction. One important conclusion from this work is to demonstrate the importance of learning in deployment: an agent trained purely offline with no online updating performs more poorly than an agent that learns online. I will conclude with some general learnings about using reinforcement learning for real systems.

This event is hosted by Ontario Regional Centre of the Canadian Statistical Sciences Institute (CANSSI).