This paper deals with the stereo matching problem, while moving away from the traditional fronto-parallel assumption. We propose an algorithm that provides disparities in accordance with the surface properties of the scene under consideration. To do so, we carry out cooperatively both disparity and surface normal estimations by setting the two tasks in a unified Markovian framework. A novel joint probabilistic model is defined through two Markov Random Fields (MRF) to favor both intra field (within neighboring disparities and neighboring normals) and inter field (between disparities and normals) consistency. Geometric contextual information is introduced in the pair-wise Markovian regularizing term used in both MRFs. Segmentation and plane fitting procedures, usually performed as final steps to increase the quality results are here explicitly used in one of the MRF data terms. We then design an appropriate alternating maximization procedure based on standard Belief Propagation. We illustrate the performance of our approach on synthetic and real data. The results obtained are comparable to the state-of-the-art and show improvement in many cases.