Agents, especially in the context of Multi-Agents Systems, are confronted to complex tasks. We propose a methodology for the automated design of such agents in the case where the global task can be decomposed into simpler sub-tasks that can be concurrent. This is accomplished by automatically combining basic behaviors using Reinforcement Learning methods. Basic behaviors are either learned or reused from previous tasks as they do not need to be tuned to the specific task being learned. Furthermore, the agents designed by our methodology are highly scalable as, without further refinement of the global behavior, they can automatically combine several instances of the same basic behavior to take into account concurrent occurences of the same subtask.