We present a novel physically-based approach for simulating realistic brittle fracture of impacting bodies in real-time. Our method is mainly composed of two novel parts: (1) a fracture initiation method based on modal analysis, (2) a fast energybased fracture propagation algorithm. We propose a way to compute the contact durations and the contact forces between stiff bodies to simulate the damped deformation wave that is responsible for fracture initiation. As a consequence, our method naturally takes into account the damping properties of the bodies as well as the contact properties to simulate the fracture. To obtain a complete fracture pipeline, we present an efficient way to generate the fragments and their geometric surfaces. These surfaces are sampled on the edges of the physical mesh, to visually represent the actual fracture surface computed. As shown in our results, the computation time performances and realism of our method are well-suited for physically-based interactive applications.