In this paper we present a method to control the displacement of a robot arm with no proprioceptive sensor. The joint positions are not available and this manipulator is usually open-loop controlled. In order to get a more efficient control interface, we propose a closed-loop system based on an eye-to-hand visual servoing approach. We show that, using such an approach, measurement of the manipulator motion with proprioceptive sensors is not required to precisely control the end-effector motion. We propose solutions for position-based control and velocity control of the manipulator. To maintain the end effector in the camera field of view, the camera orientation is also controlled. Various results show the validity and the efficiency of the approach.