The use of synthetic sequences is one of the most promising tools for advanced in silico evaluation of the quantification of cardiac deformation and strain through 3D ultrasound (US) and magnetic resonance (MR) imaging. In this paper, we propose the first simulation framework which allows the generation of realistic 3D synthetic cardiac US and MR (both cine and tagging) image sequences from the same virtual patient. A state-of-the-art electromechanical (E/M) model was exploited for simulating groundtruth cardiac motion fields ranging from healthy to various pathological cases including both ventricular dyssynchrony and myocardial ischemia. The E/M groundtruth along with template MR/US images and physical simulators were combined in a unified framework for generating synthetic data. We efficiently merged several warping strategies to keep full control of myocardial deformations while preserving realistic image texture. In total, we generated 18 virtual patients, each with synthetic 3D US, cine MR and tagged MR sequences. The simulated images were evaluated both qualitatively by showing realistic textures and quantitatively by observing myocardial intensity distributions similar to real data. In particular, the US simulation showed a smoother myocardium/background interface than the state-of-the-art. We also assessed the mechanical properties. The pathological subjects were discriminated from the healthy ones by both global indexes (ejection fraction and the global circumferential strain) and regional strain curves. The synthetic database is comprehensive in terms of both pathology and modality, and has a level of realism sufficient for validation purposes. All the 90 sequences are made publicly available to the research community via an open-access database.