Agents are of interest mainly when confronted with complex tasks. We propose a methodology for the automated design of such agents (in the framework of Markov Decision Processes) in the case where the global task can be decomposed into simpler -possibly concurrent- sub-tasks. This is accomplished by automatically combining basic behaviors using Reinforcement Learning methods. \correction[Basic behaviors]{The main idea is to build a global policy using a weighted combination of basic policies, the weights being learned by the agent (using Simulated Annealing in our case). These basic behaviors} can either be learned or reused from previous tasks since they will not need to be tuned to the new task. Furthermore, the agents designed by our methodology are highly scalable as, without further refinement of the global behavior, they can automatically combine several instances of the same basic behavior to take into account concurrent occurences of the same subtask.