We propose a novel hybrid high-order method (HHO) to approximate singularly perturbed fourthorder PDEs on domains with a possibly curved boundary. The two key ideas in devising the method are the use of a Nitsche-type boundary penalty technique to weakly enforce the boundary conditions and a scaling of the weighting parameter in the stabilization operator that compares the singular perturbation parameter to the square of the local mesh size. With these ideas in hand, we derive stability and optimal error estimates over the whole range of values for the singular perturbation parameter, including the zero value for which a second-order elliptic problem is recovered. Numerical experiments illustrate the theoretical analysis.