In this work we face the problem of finding strong Stackelberg equilibrium in stochastic games. We study a familiy of stochastic games where equilibrium in stationary policies exist and prove the convergence of value iteration and policy iteration procedures. Preliminary computational results evaluate the performance of these algorithms for stochastic games in the form of security games. Finally, we show that is not always possible to achieve strong Stackelberg equilibrium via dynamic programming.