This paper introduces the reasoning and concepts behind a generic and modular approach to obstacle avoidance based on geofencing. The issue of UAV safety is tackled from a practical point of view and a collision avoidance system based on practical usage and safety considerations is presented. Before doing so, the need for a reference model of geographic geometry is discussed, looking at global positioning in different reference systems and how to switch between the associated coordinates systems. The concept of fence is defined and the practical aspects of these fences definitions and mapping are detailed. A collision avoidance strategy is proposed that works in the case of autonomous flight as well as in the case of piloted flight. To our knowledge, this second scenario has not been extensively covered and we feel it deserves some attention since most of the civilian UAVs flying today are piloted manually by an operator. This avoidance is based on constrained optimization to find an input for the controller which prevents collision while staying as true as possible to the original command from the human operator. This strategy allows for “smooth” collision avoidance since it gently and continuously restricts the flight possibilities of the UAV while giving as much control freedom as possible to the operator. We also discuss how this work can be extended by taking the actual vehicle dynamic into consideration in order to make it work for a larger variety of controllers. The concept behind the “closest point” approach used above is detailed, and its limitations and possible extensions are discussed. Finally, we present the simulation framework used to test this geofencing strategy and discuss the simulation results as a proof of concept for our future implementation.