During this internship, we worked on improving an open domain question answering system. We addressed the document selection part which is structured as a text ranking task. The first step was to explore classical methods, also known as sparse retrievers, and to test those algorithms on our evaluation dataset. These methods produced only minor differences in performance. The next step was to employ deep language models, namely the BERT based architectures. A variety of techniques and designs were considered. First, we tackled the lack of data to train such models on the French language, followed by the definition of the problem (classification or ranking), and finally, we addressed the problem of limited text length in BERT-based models. The final results show a 12% improvement in performance over the original model.