This paper deals with model-based pose estimation (or camera localization). The model is rendered as a virtual image and we propose a direct approach that takes into account the image as a whole. For this, we consider a sim- ilarity measure, the mutual information. Mutual information is a measure of the quantity of information shared by two signals (or two images in our case). Exploiting this measure allows our method to deal with different image modalities (real and synthetic). Furthermore, it handles occlusions and illu- mination changes. Results with synthetic (benchmark) and real image sequences, with static or mobile camera, demonstrate the robustness of the method and its ability to produce stable and precise pose estimations.