We study a non-zero sum game considered on the solutions of a hybrid dynamical system that evolves in continuous time and that is subjected to abrupt changes of parameters. The changes of the parameters are synchronized with (and determined by) the changes of the states/actions of two Markov decision processes, each of which is controlled by a player that aims at minimizing his or her objective function. The lengths of the time intervals between the " jumps " of the parameters are assumed to be small. We show that an asymptotic Nash equilibrium of such hybrid game can be constructed on the basis of a Nash equilibrium of a deterministic averaged dynamic game.