Concrete is a widely used construction material by virtue of its cost and mechanical properties. Due to its low tensile strength however, concrete is very sensitive to crack formation. Cracks in concrete are responsible for significant inspection, maintenance and repair costs. In order to optimize structural health management, Non-Destructive Testing (NDT) has been extensively studied. Among all NDT techniques, ultrasonic methods are considered advantageous by providing information on mechanical properties in areas not directly accessible from the surface. Recent studies have led to developing nonlinear ultrasonic methods to increase the sensitivity to damage making possible the detection of large cracks/notches and the monitoring of crack evolution. However, the detection of small cracks in concrete remains a great challenge for NDT techniques. In this study, an ultrasonic method, based on nonlinear acoustic mixing of coda waves by lower-frequency swept pump waves, providing for an efficient global detection of small cracks in concrete is presented. By simultaneous comparison, for uncracked and cracked mortars, of ultrasonic velocity variations and decorrelation coefficient between the unperturbed and the perturbed signals for different pump amplitude, the method allows to accurately detect very small cracks with widths of around 20 mm correlated with velocity variations of approximately 0.01%. This method is reproducible and able to provide a simple means for differentiating damaged and sound concrete. Attention must be paid however to the material evolution during the time span of both a single experiment and the entire experimental campaign as a consequence of the presumed high sensitivity of the observables. Several applications of this technique could be developed in the field of civil engineering, although the power of the pump source would constitute a limitation. For example, the detection of small cracks causing leakage could be performed without any need for percolating fluid.