The fundamental problems in the traditional frequency do- main approaches to convolutive blind source separation are 1) arbitrary permutations and 2) arbitrary scaling in each frequency bin of the esti- mated filters or sources. These ambiguities are corrected by taking into account some specific properties of the filters or sources, or both. This paper focusses on the filter permutation problem, assuming the absence of the scaling ambiguity, investigating the use of temporal sparsity of the filters as a property to aid permutation correction. Theoretical and ex- perimental results bring out the potential as well as the extent to which sparsity can be used as a hypothesis to formulate a well posed permuta- tion problem.