Decomposition of Time Scales in Linear Systems and Markovian Decision Processes

Abstract

The presence of "slow" and "fast" dynamics in large scale systems has motivated the use of singular perturbations as a means of obtaining reduced order models for analysis and control law design. In this thesis we establish how systems having this "two-time-scale" property can use singular perturbation modeling to make this property explicit enabling various reduced order analysis and design techniques to be applied. For deterministic linear time-invariant systems, various techniques for obtaining reduced order models are unified through left and right eigenspace decompositions. A general two stage control design procedure for separate fast and slow subsystems is developed which can be applied to both continuous and discrete time models. Finally, Markov chain models of stochastic systems with "weak" and "strong" transition probabilities lead to a singularly perturbed model from which we obtain the concept of the reduced order "aggregate" chain. For controlled Markov chains the aggregate model is used to develop decentralized optimization algorithms for the discounted and average cost per stage problems.