Reinforcement Learning in the Real World: Making Predictions Online for Water Treatment – Oct. 30, 2023

Join CANSSI Ontario at the Statistical Sciences Applied Research and Education Seminar (ARES) with Martha White, Associate Professor, Department of Computing Science, University of Alberta.

In this talk I will discuss how we used reinforcement-learning based prediction approaches for a real drinking-water treatment plant. I will first describe this dataset, and highlight challenges with seasonality, non- stationarity, partial observability, and heterogeneity across sensors and operation modes of the plant. I will then explain General Value Function (GVF) predictions—discounted cumulative sums of observations–and highlight why they might be preferable to classical n-horizon predictions common in time series prediction. One important conclusion from this work is to demonstrate the importance of learning in deployment: an agent trained purely offline with no online updating performs more poorly than an agent that learns online. I will conclude with some general learnings about using reinforcement learning for real systems.

This event is hosted by Ontario Regional Centre of the Canadian Statistical Sciences Institute (CANSSI).