We introduce the notion of syzygy for a set of reduction operators and relate it to the notion of syzygy for presentations of algebras. We give a method for constructing a linear basis of the space of syzygies for a set of reduction operators. We interpret these syzygies in terms of the confluence property from rewriting theory. This enables us to optimise the completion procedure for reduction operators based on a criterion for detecting useless reductions. We illustrate this criterion with an example of construction of commutative Gröbner basis.