Matrix factorization (MF) models have been widely used in data analysis. Even though they have been shown to be useful in many applications, classical MF models often fall short when the observed data are impulsive and contain outliers. In this study, we present αMF, a MF model with α-stable observations. Stable distributions are a family of heavy-tailed distributions that is particularly suited for such impulsive data. We develop a Markov Chain Monte Carlo method, namely a Gibbs sampler, for making inference in the model. We evaluate our model on both synthetic and real audio applications. Our experiments on speech enhancement show that αMF yields superior performance to a popular audio processing model in terms of objective measures. Furthermore, αMF provides a theoretically sound justification for recent empirical results obtained in audio processing.