In the era of big data, one may apply generic learning algorithms for medical computer vision. But such algorithms are often "black-boxes" and as such, hard to interpret. We still need new constructive models, which could eventually feed the big data framework. Where can one find inspiration for new models in medical computer vision? The emerging field of Neurogeometry provides innovative ideas. Neurogeometry models the visual cortex through modern Differential Geometry: the neuronal architecture is represented as a sub-Riemannian manifold R2 x S1. On the one hand, Neurogeometry explains visual phenomena like human perceptual completion. On the other hand, it provides efficient algorithms for computer vision. Examples of applications are image completion (in-painting) and crossing-preserving smoothing. In medical image computer vision, Neurogeometry is less known although some algorithms exist. One reason is that one often deals with 3D images, whereas Neurogeometry is essentially 2D (our retina is 2D). Moreover, the generalization of (2D)-Neurogeometry to 3D is not straight-forward from the mathematical point of view. This article presents the theoretical framework of a 3D-Neurogeometry inspired by the 2D case. We survey the mathematical structures and a standard frame for algorithms in 3D- Neurogeometry. The aim of the paper is to provide a "theoretical toolbox" and inspiration for new algorithms in 3D medical computer vision.