Markov Decision processes have been widely used to control the execution of a static set of tasks with limited resources. But little attention has been paid to adapt these techniques to cope with changes in the environment. This is a problem of the dynamic optimisation of resource allocation to a changing set of tasks. We transform this problem into a dynamic composition of local policies (each of which controls a task) to approximate the optimal control policy. Our main claim in this paper is that it is possible to dynamically compute good decisions without completely calculating the optimal policy. We develop an approach which provides more flexibility for MDPs to deal with dynamic environments. This approach is made up of two steps. The first step consists of an off-line pre-processing of tasks and the compilation of policies for all possible available resources. The second step concerns a quick on-line approximation of the policy of executing the current task taking into account the current state of the queue. We present a preliminary analysis of the performance of our approach.