Simulating the deformation of the human anatomy is a central element of Medical Image Computing and Computer Assisted Interventions. Such simulations play a key role in non-rigid registration, augmented reality, and several other applications. Although the Finite Element Method is widely used as a numerical approach in this area, it is often hindered by the need for an optimal meshing of the domain of interest. The derivation of meshes from imaging modalities such as CT or MRI can be cumbersome and time-consuming. In this paper we use the Immersed Boundary Method (IBM) to bridge the gap between these imaging modalities and the fast simulation of soft tissue deformation on complex shapes represented by a surface mesh directly retrieved from binary images. A high resolution surface, that can be obtained from binary images using a marching cubes approach, is embedded into a hexahedral simulation grid. The details of the surface mesh are properly taken into account in the hexahedral mesh by adapting the Mirtich integration method. In addition to not requiring a dedicated meshing approach, our method results in higher accuracy for less degrees of freedom when compared to other element types. Examples on brain deformation demonstrate the potential of our method.