In a wide range of problems arising in source separation, large scale optimization problems need to be solved. The objective of this chapter is to introduce the theoretical background which makes it possible to develop efficient algorithms to successfully address these problems. This chapter will be mainly focused on nonlinear optimization tools for dealing with convex and nonconvex problems. Proximal tools, parallel splitting techniques, Majorization-Minimization strategies and alternating minimization approaches will be presented. Illustrations of these methods on various problems will be provided.