Contrary to many application domains, recommending items within a museum is not only a question of preferences. Of course, the visitors expect suggestions that are likely to interest or please them. However, additional factors should be taken into account. Recent works use the visiting styles or the shortest distance between items to adapt the list of recommendations. But, as far as we know, no model of the literature aims at inferring in real time a holistic user model which includes variables such as the crowd tolerance, the distance tolerance, the expected user control, the fatigue, the congestion points, etc. As a work-in-progress, we propose a new representation model which includes psychological, physical and social variables so as to increase user satisfaction and enjoyment. We show how we can infer these characteristics from the user observations (geolocalization over time, moving speed,. . .) and we discuss how we can use them jointly for a sequence recommendation purpose. This work is still in an early stage of development and remains more theoretical than experimental.