Building roof extraction from airborne laser scanning point clouds is significant for building modeling. The common method adopts a bottom-up strategy which requires a ground filtering process first, and the subsequent process of region growing based on a single seed point easily causes oversegmentation problem. This paper proposes a novel method to extract roofs. A top-down strategy based on cloth simulation is first used to detect seed point sets with semantic information; then, the roof seed points are extracted instead of a single seed point for region-growing segmentation. The proposed method is validated by three point cloud datasets that contain different types of roof and building footprints. The results show that the top-down strategy directly extracts roof seed point sets, most roofs are extracted by the region-growing algorithm based on the seed point set, and the total errors of roof extraction in the test areas are 0.65%, 1.07%, and 1.45%. The proposed method simplifies the workflow of roof extraction, reduces oversegmentation, and determines roofs in advance based on the semantic seed point set, which suggests a practical solution for rapid roof extraction.