This paper introduces an image region descriptor and applies it to the problem of appearance matching. The proposed descriptor can be seen as a natural extension of covariance. Driven by recent studies in mathematical statistics related to Brownian motion, we design the Brownian descriptor. In contrast to the classical covariance descriptor, which measures the degree of linear relationship between features, our novel descriptor measures the degree of all kinds of possible relationships between features. We argue that the proposed covariance is a richer descriptor than the classical covariance, especially when fusing non-linearly dependent features. We demonstrate that the Brownian descriptor significantly outperforms the classical covariance in terms of matching accuracy and efficiency, opening new directions for future research.