In this paper parallelepipeds and their use in camera calibration and 3D reconstruction processes are studied. Parallelepipeds naturally characterize rigidity constraints present in a scene, such as parallelism and orthogonality. A subclass of parallelepipeds-the cuboids-has been frequently used over the past to partially calibrate cameras. However, the full potential of parallelepipeds, in camera calibration as well as in scene reconstruction, has never been clearly established. We propose a new framework for the use of parallelepipeds which is based on an extensive study of this potential. In particular, we exhibit the complete duality that exists between the intrinsic metric characteristics of a parallelepiped and the intrinsic parameters of a camera. Our framework allows to fully exploit parallelepipeds and thus overcomes several limitations of calibration approaches based on cuboids. To illustrate this framework, we present an original and very efficient interactive method for 3D reconstruction from single images. This method allows to quickly build a scene model from a single uncalibrated image.