In this paper, we propose a method to control the motion of soft robots able to manipulate objects or roll from one place to another. We use the Finite Element Method (FEM) to simulate the deformations of the soft robot, its actuators, and surroundings when deformable. To find the inverse model of the robot interacting with obstacles, and with constraints on its actuators, we write the problem as a quadratic program with complementarity constraints. The novelty of this work is that friction contacts (sticking contact only) are taken into account in the optimization process, allowing the control of these specific tasks that are locomotion and manipulation. We propose a formulation that simplifies the optimization problem, together with a dedicated solver. The algorithm has real-time performance and handles evolving environments as long as we know them. To show the effectiveness of the method, we present several numerical examples, and a demonstration on a real robot.