Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2016

Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera

Résumé

This paper presents a mirror-like augmented reality (AR) system to display the internal anatomy of a user. Using a single Microsoft V2.0 Kinect, we animate in real-time a user-specific internal anatomy according to the user’s motion and we superimpose it onto the user’s color map. The user can visualize his anatomy moving as if he was able to look inside his own body in real-time. A new calibration procedure to set up and attach a user-specific anatomy to the Kinect body tracking skeleton is introduced. At calibration time, the bone lengths are estimated using a set of poses. By using Kinect data as input, the practical limitation of skin correspondance in prior work is overcome. The generic 3D anatomical model is attached to the internal anatomy registration skeleton, and warped on the depth image using a novel elastic deformer, subject to a closest-point registration force and anatomical constraints. The noise in Kinect outputs precludes any realistic human display. Therefore, a novel filter to reconstruct plausible motions based on fixed length bones as well as realistic angular degrees of freedom (DOFs) and limits is introduced to enforce anatomical plausibility. Anatomical constraints applied to the Kinect body tracking skeleton joints are used to maximize the physical plausibility of the anatomy motion, while minimizing the distance to the raw data. At run-time, a simulation loop is used to attract the bones towards the raw data, and skinning shaders efficiently drag the resulting anatomy to the user’s tracked motion. Our user-specific internal anatomy model is validated by comparing the skeleton with segmented MRI images. A user study is established to evaluate the believability of the animated anatomy.
Fichier principal
Vignette du fichier
mig_2016.pdf (24.08 Mo) Télécharger le fichier
mig_submission_id_01.mp4 (27.19 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01366704 , version 1 (16-09-2016)

Identifiants

Citer

Armelle Bauer, Ali-Hamadi Dicko, François Faure, Olivier Palombi, Jocelyne Troccaz. Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera. ACM SIGGRAPH Conference on Motion in Games, Oct 2016, San Francisco, United States. pp.113-122, ⟨10.1145/2994258.2994259⟩. ⟨hal-01366704⟩

Relations

1537 Consultations
593 Téléchargements

Altmetric

Partager

More