Context: Providers and consumers of computer vision algorithms need to test their solutions using realistic videos as input test data. The current practice is to find existing videos or to film them outdoors. However, the e ort of manually collecting or filming videos in diverse scenarios is highly resource intensive and not economically viable. In an industrial project, we faced the challenge of providing more automation and control to produce video variants using a video generator. A key problem is to capture and exploit information of what can vary within a video. Objective: This work seeks to describe the variability requirements in the video domain and to provide practical solutions for video variability modeling to support a generative approach for synthesizing video sequences. Method: In this paper, we report about a new domain-specific variability modeling and configuration language, called VM, resulting from the close collaboration with industrial partners. We expose the requirements and advanced variability constructs required to characterize and realize variations of physical properties of a video (such as objects speed or illumination). Results: The results of our experiments and industrial experience show that VM is e ective to model complex variability information and can be exploited to synthesize video variants. Conclusions: We concluded that basic variability mechanisms are useful but not enough, attributes and multi-features are of prior importance, and meta-information is relevant for e fficient variability analysis. In addition, we questioned the existence of one-size-fits-all variability modeling solution applicable in any industry. Yet, some common needs for modeling variability are becoming apparent such as support for attributes and multi-features.