Stochastic Optimal Control with MarkovBounds.jl

We present MarkovBounds.jl -- a meta-package to SumOfSquares.jl which enables the computation of guaranteed bounds on the optimal value of a large class of stochastic optimal control problems via a high-level, practitioner-friendly interface.

The optimal control of stochastic processes is arguably one of the most fundamental questions in the context of decision-making under uncertainty. When the controlled process is a jump-diffusion process characterized by polynomial data (drift- and diffusion coefficient, jumps, etc.), it is well known that polynomial optimization and the machinery of the moment-sum-of-squares (SOS) hierarchy provides a systematic way to construct informative (and often tight) convex relaxations for the associated optimal control problems. While the JuMP ecosystem offers with SumOfSquares.jl in principle everything that is required to study stochastic optimal control problems from this perspective, it remains a cumbersome and error-prone process to translate a concrete stochastic optimal control problem into its SOS relaxation. Moreover, this translation process requires expert knowledge, rendering it inaccessible to a large audience. MarkovBounds.jl is intended to close this gap by providing a high-level interface which allows the user to define stochastic optimal control problems in symbolic form using for example DynamicPolynomials.jl or Symbolics.jl, and automates subsequent translation to associated SOS relaxations. Finite and (discounted) infinite horizon problems are supported as well as several common objective function types. Furthermore, MarkovBounds.jl supports the combination of standard SOS relaxations with discretization approaches to tighten the relaxations.

In this talk, we will briefly review the conceptual ideas behind constructing SOS relaxations for stochastic optimal control problems and showcase the use of MarkovBounds.jl for the optimal control of populations in a predator-prey system, expression of protein in a stochastic bio circuit and the bounding of rare event probabilities.