We investigate mathematically a nonlinear approximation type approach recently introduced in [A. Ammar et al., J. Non-Newtonian Fluid Mech., 2006] to solve high dimensional partial differential equations. We show the link between the approach and the greedy algorithms of approximation theory studied e.g. in [R.A. DeVore and V.N. Temlyakov, Adv. Comput. Math., 1996]. On the prototypical case of the Poisson equation, we show that a variational version of the approach, based on minimization of energies, converges. On the other hand, we show various theoretical and numerical difficulties arising with the non variational version of the approach, consisting of simply solving the first order optimality equations of the problem. Several unsolved issues are indicated in order to motivate further research.