Toggle Main Menu Toggle Search

Open Access padlockePrints

Symbolic controller synthesis for Büchi specifications on stochastic systems

Lookup NU author(s): Dr Sadegh SoudjaniORCiD



We consider the policy synthesis problem for continuous-state controlled Markov processes evolving in discrete time, when the specification is given as a Buchi condition (visit a set of states infinitely often). We decompose computation of the maximal probability of satisfying the Buchi condition into two steps. The first step is to compute the maximal qualitative winning set, from where the Buchi condition can be enforced with probability one. The second step is to find the maximal probability of reaching the already computed qualitative winning set. In contrast with finite-state models, we show that such a computation only gives a lower bound on the maximal probability where the gap can be non-zero. In this paper we focus on approximating the qualitative winning set, while pointing out that the existing approaches for unbounded reachability computation can solve the second step. We provide an abstraction-based technique to approximate the qualitative winning set by simultaneously using an over- and under-approximation of the probabilistic transition relation. Since we are interested in qualitative properties, the abstraction is non-probabilistic; instead, the probabilistic transitions are assumed to be under the control of a (fair) adversary. Thus, we reduce the original policy synthesis problem to a Buchi game under a fairness assumption and characterize upper and lower bounds on winning sets as nested fixed point expressions in the mu-calculus. This characterization immediately provides a symbolic algorithm scheme. Further, a winning strategy computed on the ab- stract game can be refined to a policy on the controlled Markov process. We describe a concrete abstraction procedure and demonstrate our algorithm on two case studies. We show that our techniques are able to provide tight approximations to the qualitative winning set for the Van der Pol oscillator and a 3-d Dubin’s vehicle.

Publication metadata

Author(s): Majumdar R, Mallik K, Soudjani S

Publication type: Conference Proceedings (inc. Abstract)

Publication status: Published

Conference Name: 23rd ACM International Conference on Hybrid Systems: Computation and Control

Year of Conference: 2020

Pages: 1-11

Print publication date: 21/04/2020

Online publication date: 22/04/2020

Acceptance date: 27/12/2019

Date deposited: 14/01/2020

Publisher: ACM


DOI: 10.1145/3365365.3382214

Library holdings: Search Newcastle University Library for this item

ISBN: 9781450370189