Towards dense 3D reconstruction for mixed reality in healthcare: classical multi-view stereo vs deep learning
Résumé
Faithfully reproducing surroundings in 3D is a key- component in Mixed Reality for medical training in neona- tology, where a user sees a hospital room in a Virtual Real- ity helmet while retaining tangible interaction with a baby mannequin and various medical tools. Deep learning solu- tions have high claims against classical methods but their performance in real-life application remains unclear. To fill this blank, we present a comparative study of depth map based Multi-View Stereo methods for dense 3D reconstruc- tion. We compare classical state-of-the-art methods to their learned counterparts and assess their robustness to weakly- textured and reflective surfaces as well as accuracy on thin structures both globally and locally. We also analyze the ef- fect of depth filtering along with computational effort. Our experiments reveal various factors which contribute to the performance gap between the methods that we discuss in detail. This study is the first to evaluate traditional dense geometry reconstruction methods against brand-new deep learning models. It helps to better understand what suits best the challenges of hospital environments. Furthermore, it builds a solid analytic ground to underscore the strengths and weaknesses of the learned methods.
Fichier principal
TowardsDense3DReconstructionForMixedRealityInHealthcare_ClassicalMultiViewStereoVSDeepLearning_NoteIEEE.pdf (3.18 Mo)
Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)