For the purpose of autonomous UAV flight control, cameras are ubiquitously exploited as a cheap and effective onboard sensor for obtaining non-metric position or velocity measurements. Since the metric scale cannot be directly re- covered from visual input only, several methods have been proposed in the recent literature to overcome this limitation by exploiting independent "metric" information from additional onboard sensors. The flexibility of most approaches is, however, often limited by the need of constantly tracking over time a certain set of features in the environment, thus potentially suffering from possible occlusions or loss of tracking during flight. In this respect, in this paper we address the problem of estimating the scale of the observed linear velocity in the UAV body frame from direct measurement of the instantaneous (and non-metric) optical flow, and the integration of an on- board Inertial Measurement Unit (IMU) for providing (metric) acceleration readings. To this end, two different estimation techniques are developed and critically compared: a standard Extended Kalman Filter (EKF) and a novel nonlinear observer stemming from the adaptive control literature. Results based on simulated and real data recorded during a quadrotor UAV flight demonstrate the effectiveness of the approach.