In this paper, the method for the creation of an anatomically and mechanically realistic brain phantom from polyvinyl alcohol cryogel (PVA-C) is proposed for validation of image processing methods such as segmentation, reconstruction, registration, and denoising. PVA-C is material widely used in medical imaging phantoms because of its mechanical similarities to soft tissues. The phantom was cast in a mold designed using the left hemisphere of the Colin27 brain dataset [C. Holmes et al., "Enhancement of MR images using registration for signal averaging," J. Comput. Assist. Tomogr. 22(2), 324 (1998)]. Marker spheres and inflatable catheters were also implanted to enable good registration comparisons and to simulate tissue deformation, respectively. The phantom contained deep sulci, a complete insular region, and an anatomically accurate left ventricle. It was found to provide good contrast in triple modality imaging, consisting of computed tomography, ultrasound, and magnetic resonance imaging. Multiple sets of multimodal data were acquired from this phantom. The methods for building the anatomically accurate, multimodality phantom were described in this work. All multimodal data are made available freely to the image processing community (http://pvabrain.inria.fr). We believe the phantom images could allow for the validation and further aid in the development of novel medical image processing techniques.