Brain-inspired Hyperdimensional Computing (HDC), a machine learning (ML) model featuring high energy efficiency and fast adaptability, provides a promising solution to many real-world tasks on resource-limited devices. This paper introduces an HDC-based user adaptation framework, which requires efficient fine-tuning of HDC models to boost accuracy. Specifically, we propose two techniques for HDC, including the learnable projection and the fusion mechanism for the Associative Memory (AM). Compared with the user adaptation framework based on the original HDC, our proposed framework shows 4.8% and 3.5% of accuracy improvements on two benchmark datasets, including the ISOLET dataset and the UCIHAR dataset, respectively.