The implementation of real-time audio Digital Signal Processing (DSP) on FPGA has been extensively studied in the past. Up to now, Audio IPs were designed either “by hand” in VHDL or using predefined IPs in block synthesis environments. The advent of High Level Synthesis (HLS) allows for a real compilation flow from high-level audio DSP specifications down to FPGA bit-streams. This paper presents the principles and the implementation of the first “audio DSP compiler” targeting FPGAs. Our fully open-source system compiles audio DSP programs down to FPGA hardware and up to actual sound production. Many parameters such as the number of output channels, sampling rate, etc. are adjusted automatically by the compiler. Software interfaces can be generated to control the system in real-time. This compilation flow presents two important technological breakthroughs for audio programmers: achieving ultra-low latency real- time audio DSP (few micro-seconds) and the possibility of easily deploying systems with a large number of audio channels.