With the advent of cloud computing and the need to satisfy growing customers resource demands, cloud providers now operate increasing amounts of large data centers. In order to ease the creation of private clouds, several open-source Infrastructure-as-a-Service (IaaS) cloud management frameworks (e.g., OpenNebula, Nimbus, Eucalyptus, OpenStack) have been proposed. However, all these systems are either highly centralized or have limited fault-tolerance support. Consequently, they all share common drawbacks: scalability is limited by a single master node and Single Point of Failure (SPOF). In this paper, we present the design, implementation and evaluation of a novel scalable and fault-tolerant virtual machine (VM) management framework called Snooze. For scalability our system utilizes a self-organizing hierarchical architecture and performs distributed VM management. Moreover, fault-tolerance is provided at all levels of the hierarchy, thus allowing the system to self-heal in case of failures. Our evaluation conducted on 144 physical machines of the Grid'5000 experimental testbed shows that the fault-tolerance features of the framework do not impact application performance. Moreover, negligible cost is involved in performing distributed VM management and the system remains highly scalable with increasing amounts of resources.