This paper is devoted to tackling constrained multi-objective optimisation under uncertainty problems. A Surrogate-Assisted Bounding-Box approach (SABBa) is formulated here to deal with approximated robustness and reliability measures, which can be adaptively refined. A Bounding-Box is defined as a multi-dimensional product of intervals, centred on the estimated objectives and constraints, that contains the true underlying values. The accuracy of these estimations can be tuned throughout the optimisation so as to reach high levels only on promising designs, which allows quick convergence toward the optimal area. In SABBa, this approach is supplemented with a Surrogate-Assisting (SA) strategy, which permits to further reduce the overall computational cost. The adaptive refinement within the Bounding-Box approach is guided by the computation of the Pareto Optimal Probability (POP) of each box. We first assess the proposed method on several analytical uncertainty-based optimisation test-cases with respect to an a priori metamodel approach in terms of a probabilistic modified Hausdorff distance to the true Pareto optimal set. The method is then applied to three engineering applications: the design of two-bar truss in structural mechanics, the shape optimisation of an Organic Rankine Cycle turbine blade and the design of a thermal protection system for atmospheric reentry.