This extended abstract presents a review of various constructs and evaluation frameworks proposed in the literature for e-Retailers. Our study shows that the existing frameworks apply to either the technical or the non-technical elements of an e-Retailing site and not both of them. Therefore, a comprehensive framework covering all aspects of quality is what is required. Furthermore, for the long run sustenance and growth of e-Retailing, it is necessary to focus on the service provided and not only the technical aspects. In the Indian context, where e-Retailing is beginning to enter a rapid growth phase, evaluation methods and metrics which are appropriate are necessary. Our extended abstract highlights the key issues that will help define these constructs.