Urban areas are subject to heavy road traffic, creating congestion that has a negative impact on the economy and greenhouse gas emissions. To alleviate this problem, traffic light systems (TLS) have been developed. Existing evaluations of these systems suffer from several biases, reducing the credibility of the results. In this paper, we develop a stochastic protocol for evaluating TLS. The protocol consists of simulating each system a large number of times on a single road network, randomly varying vehicle flows and system parameters. On the basis of average travel times and CO2 emissions per travel, we develop two new metrics for analyzing the potential and the difficulty of setting of the systems. Based on an example of the use of this protocol, we demonstrate that the use of stochastic simulations makes it possible to obtain objective results, unlike existing protocols. We believe that this test method will help road infrastructure managers to select a traffic light management system with greater confidence.