Besides copyright protection, digital video watermarking is also applied in non-security applications like second screen annotation, where high robustness against geometric distortions and high watermark payload are required. Robustness against geometric distortions, however, is still one of the major challenging issues in video watermarking, in particular for the schemes in compressed domain. In this paper, we propose a video watermarking scheme that can resist geometric attacks. The watermark embedding is performed in Fourier domain using patchwork method which is able to handle high embedding payload. Fast transform between block DCT and DFT enables the proposed scheme to be applicable directly in the compressed domain, significantly reducing the computation cost. Perceptual masking is applied in both DFT and DCT domains to ensure high visual quality. Experimental results demonstrate that the proposed scheme achieves satisfactory robustness against all kinds of attacks, including geometric distortions and frame dropping and swapping.