In this paper, we review the recently finished CARETAKER project outcomes from a system point of view. The IST FP6-027231 CARETAKER project aimed at studying, developing and assessing multimedia knowledge-based content analysis, knowledge extraction components, and metadata management sub-systems in the context of automated situation awareness and decision support. More precisely, CARETAKER focused on the extraction of a structured knowledge from large multimedia collections recorded over surveillance networks of camera and microphones deployed in real sites. Indeed, the produced audio-visual streams, in addition to security and safety issues, represent a useful source of information when stored and automatically analysed, for instance in urban planning or resource optimisation. In this paper, we overview the communication architecture developed for the project, and detail the different innovative content analysis components developed within the test-beds. We also highlight the different technical concerns encountered for each individual brick, which are common issues in distributed media applications.