It is frequently stated that the maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimates of a continuous parameter are not invariant to arbitrary «reparametrizations» of the parameter space . This report clarifies the issues surrounding this problem, by pointing out the difference between coordinate invariance, which is a sine qua non for a mathematically well-defined problem, and diffeomorphism invariance, which is a substantial issue, and provides a solution. We first show that the presence of a metric structure on can be used to define coordinate-invari- ant MAP and MMSE estimates, and we argue that this is the natural and necessary way to proceed. The estimation problem and related geometrical quantities are all defined in a manifestly coordinate-invariant way. We show that the same MAP estimate results from `density maximization' or from using an invariantly-defined delta function loss. We then discuss the choice of a metric structure on . By imposing an invariance criterion natural within a Bayesian framework, we show that this choice is essentially unique. It does not necessarily correspond to a choice of coordinates. The resulting MAP estimate coincides with the minimum message length (MML) estimate, but no discretization or approximation is used in its derivation.