The Internet of Things plays a central role in the foreseen shift of the Internet to the Future Internet, as it incarnates the drastic expansion of the In- ternet network with non-classical ICT devices. It will further be a major source of evolution of usage, due to the penetration in the user's life. As such, we en- vision that the Internet of Things will cooperate with the Internet of Services to provide users with services that are aware of their surrounding environment. The supporting service-oriented middleware shall then abstract the functionalities of Things as services as well as provide the needed interoperability and flexibil- ity, through a loose coupling of components and composition of services. Still, core functionalities of the middleware, namely service discovery and composi- tion, need to be revisited to meet the challenges posed by the Internet of Things. Challenges in particular relate to the ultra large scale, heterogeneity and dynam- ics of the Internet of Things that are far beyond the ones of today's Internet of Services. In addition, new challenges also arise, pertaining to the physical-world aspect that is central to the IoT. In this paper, we survey the major challenges posed to service-oriented middleware towards sustaining a service-based Internet of Things, together with related state of the art. We then concentrate on the spe- cific solutions that we are investigating within the INRIA ARLES project team as part of the CHOReOS European project, discussing new approaches to overcome the challenges particular to the Internet of Things.