Global PAC Bounds for Learning Discrete Time Markov Chains - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2020

Global PAC Bounds for Learning Discrete Time Markov Chains

Jun Sun
  • Function : Author
  • PersonId : 1085966


Learning models from observations of a system is a powerful tool with many applications. In this paper, we consider learning Discrete Time Markov Chains (DTMC), with different methods such as frequency estimation or Laplace smoothing. While models learnt with such methods converge asymptotically towards the exact system, a more practical question in the realm of trusted machine learning is how accurate a model learnt with a limited time budget is. Existing approaches provide bounds on how close the model is to the original system, in terms of bounds on local (transition) probabilities, which has unclear implication on the global behavior. In this work, we provide global bounds on the error made by such a learning process, in terms of global behaviors formalized using temporal logic. More precisely, we propose a learning process ensuring a bound on the error in the probabilities of these properties. While such learning process cannot exist for the full LTL logic, we provide one ensuring a bound that is uniform over all the formulas of CTL. Further, given one time-to-failure property, we provide an improved learning algorithm. Interestingly, frequency estimation is sufficient for the latter, while Laplace smoothing is needed to ensure non-trivial uniform bounds for the full CTL logic.


Other [cs.OH]
Fichier principal
Vignette du fichier
BGJS20.pdf (433.85 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03065571 , version 1 (14-12-2020)


  • HAL Id : hal-03065571 , version 1


Hugo Bazille, Blaise Genest, Cyrille Jegourel, Jun Sun. Global PAC Bounds for Learning Discrete Time Markov Chains. CAV 2020 - 32nd International Conference on Computer-Aided Verification, Jul 2020, Los Angeles, United States. pp.304-326. ⟨hal-03065571⟩
49 View
64 Download


Gmail Facebook Twitter LinkedIn More