The use of the heat kernel on graphs has recently given rise to a family of so-called Diffusion-Wasserstein distances which resort to the Optimal Transport theory for comparing attributed graphs. In this paper, we address the open problem of optimizing the diffusion time used in these distances and which plays a key role in several machine learning settings, including graph domain adaptation or graph classification. Inspired from the notion of triplet-based constraints used, e.g., in metric learning, we design a loss function that aims at bringing two graphs closer together while keeping an impostor away, this latter taking the form of a Wasserstein barycenter. After a thorough analysis of the properties of this function, we show on synthetic and real-world data that the resulting Diffusion-Wasserstein distances outperforms the Gromov and Fused-Gromov Wasserstein distances on unsupervised graph domain adaptation tasks. Additionally, we give evidence in such a setting that our method for optimizing the diffusion parameter allows to overcome the limitation of the widely used circular validation strategy.