The retrieval of information on the coefficient in Electrical Impedance Tomography is a severely ill-posed problem, and often leads to inaccurate solutions. It is well known that numerical methods provide only low-resolution reconstructions. The aim of this work is to analyze the Factorization Method in the case of inhomogeneous background. We propose a numerical scheme to solve the dipole-like Neumann boundary-value problem, when the background coefficient is inhomogeneous. Several numerical tests show that the method is capable of recovering the location and the shape of the inclusions, in many cases where the diffusion coefficient is nonlinearly space-dependent. In addition, we test the numerical scheme after adding artificial noise.