In recent years, virtual assistants gained a pervasive role in many domains and education was not different from others. However, although some implementation of conversational agents for supporting students have already been presented, they were ad hoc systems, built for specific courses and impossible to generalize. Also, there is a lack of research about the effects that the development of systems capable of interacting with both the students and the professors would have. In this paper, we introduce Rexy, a configurable application that can be used to build virtual teaching assistants for diverse courses, and present the results of a user study carried out using it as a virtual teaching assistant for an on-site course held at Politecnico di Milano. The qualitative analysis of the usage that was made of the assistant and the results of a post study questionnaire the students were asked to fill showed that they see conversational agents as useful tools for helping them in their studies.