In this paper we study the so-called spin-boson system, namely {a two-level system} in interaction with a distinguished mode of a quantized bosonic field. We give a brief description of the controlled Rabi and Jaynes--Cummings models and we discuss their appearance in the mathematics and physics literature. We then study the controllability of the Rabi model when the control is an external field acting on the bosonic part. Applying geometric control techniques to the Galerkin approximation and using perturbation theory to guarantee non-resonance of the spectrum of the drift operator, we prove approximate controllability of the system, for almost every value of the interaction parameter.