In this paper, we reconstruct 3D objects with a heterogeneous sensor network of Range Imaging(RIM) sensors and high-res camcorders. With this setup, we first carry out simple but effective depth calibration for the RIM cameras. We then combine the camcorder silhouette cues and RIM camera depth information, for the reconstruction. Our main contribution is the proposal of a sensor fusion framework so that the computation is general, simple and scalable. Although we only discuss the camcorders and RIM cameras in this paper, the proposed framework can be applied to any type of vision sensors. It uses a space occupancy grid as a probabilistic 3D representation of scene contents. After defining sensing models for each type of sensors, the reconstruction is simply a Bayesian inference problem, and can be solved robustly. The experiments show that the recover full 3D closed shapes substantially improved the quality of the noisy RIM sensor measurement.