The goal of this project is to address these gaps via two orthogonal approaches: design of more efficient algorithms for learning, and development of scalable exploration techniques that can lead to efficient learning.
Prof Meyn is developing new approaches to reinforcement learning (RL) and distributed control with wide-ranging applications. This project concerns the development of this theory, and its application to reliable control and resource allocation for demand dispatch (an alternative to demand response for smart grid applications).
The objective of the proposed research is to develop fundamental mathematics for general state space Markov processes and controlled interacting particle systems. The mathematical goal pertains to the development of existence, uniqueness and regularity theory for a Poisson equation, clarification of the underlying assumptions, regularity estimates, and relationship to Lyapunov exponents. Several representations of the gradient of the solution of the Poisson equation are discussed—based on the theory of elliptic PDEs along with certain compact embedding arguments for Sobolev spaces, a Lyapunov based construction, a representation in terms of the generalized resolvent, and a construction where the semigroup is approximated in terms of a diffusion map. These representations of the gradient are used to obtain new algorithms for both reinforcement learning and nonlinear filtering: These include a kernel-based algorithm based on the diffusion map approximation, and a stochastic approximation algorithm rooted in ideas from approximate value iteration. A goal of the proposed research will be to explore connections between these approximations and more broadly between the underlying mathematical concepts.
Natural and man-made networked systems are all around us. The power grid and the Internet are two examples of apparently complex interconnected systems, in which millions of “agents” are eager to extract value in the form of energy or bandwidth. While these systems are complex when measured in graph-theoretic terms, the behavior of communication and energy systems appears simple and highly predictable to the end users (in most of the world). This success is due in part to distributed control loops that manage system-wide supply-demand balance. An example of distributed control in the Internet is TCP/IP, and automatic generation control (AGC) in most electric power grids. While distributed control protocols are highly developed and widely accepted in communication applications, this is less true in other networked systems such as electric power and natural gas distribution.
This project aims to advance control theory for complex interconnected systems. The application focus is on power systems, but the control techniques are general and are likely to have far broader impact.
The goal of this project is to create virtual energy storage resources via demand dispatch to be used for grid-level regulation, ramping, peak smoothing, and even recovery from contingencies such as generation faults, while ensuring that QoS to consumers obeys strict constraints. Demand dispatch can only be realized by devising distributed control algorithms that meet multiple, potentially conflicting objectives: the grid needs high quality resources for regulation; the consumer expects that water supply is not interrupted, fish in the refrigerator stays fresh, and the climate within a building remains within desired bounds. The project aims to create a science for demand dispatch based on three essential ingredients.
The ultimate goal of the project is to help the electric grid become more reliable even when a large amount of electricity is generated from green, but intermittent – sources such as solar and wind. To deal with this intermittency, inexpensive source of energy storage are required. Instead of investing in batteries, this project seeks to obtain cheap storage by manipulating power demand in consumer loads through intelligent decision-making algorithms. By varying power demand up and down from what a load would nominally consume, the load can be made to behave like a battery, effectively creating a source of Virtual Energy Storage (VES). This kind of virtual storage is cheaper than batteries since it is a software-based solution; little additional hardware is needed. Another aspect of the project is to develop decision-making algorithms to cope with operational issues faced by the power distribution networks (that deliver electricity to neighborhoods) due to increasing use of intermittent solar power.