The increased use of multiple types of smart devices in several application domains, opens the pathways for the collection of humongous volumes of data. At the same time, the need for processing of only a subset of these data by applications in order to quickly conclude tasks execution and knowledge extraction, has resulted in the adoption of a very high number of queries set into distributed datasets. As a result, a significant process is the efficient response to these queries both in terms of time and the appropriate data. In this paper, we present a hierarchical query-driven clustering approach, for performing efficient data mapping in remote datasets for the management of future queries. Our work differs from other current methods in the sense that it combines a Query-Based Learning (QBL) model with a hierarchical clustering in the same methodology. The performance of the proposed model is assessed by a set of experimental scenarios while we present the relevant numerical outcomes.