Inverse problems arising in (geo)magnetism are typically ill-posed, in particular they exhibit non-uniqueness. Nevertheless, there exist nontrivial model spaces on which the problem is uniquely solvable. Our goal is here to describe such spaces that accommodate constraints suited for applications. In this paper we treat the inverse magnetization problem on a Lipschitz domain with fairly general topology. We characterize the subspace of L^2 vector fields that causes non-uniqueness, and identify a subspace of harmonic gradients on which the inversion becomes unique. This classification has consequences for applications and we present some of them in the context of geo-sciences. In the second part of the paper, we discuss the space of piecewise constant vector fields. This vector space is too large to make the inversion unique. But as we show, it contains a dense subspace in L^2 on which the problem becomes uniquely solvable, i.e., magnetizations from this subspace are uniquely determined by their magnetic potential.