Does Large Pretrained Dataset Always Help? On the Effect of Dataset Size on Big Transfer Model - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Does Large Pretrained Dataset Always Help? On the Effect of Dataset Size on Big Transfer Model

Résumé

Transfer learning often refers to an approach concerning machine learning in which the programmers relocate an initially developed model as the starting point for a model on a consequent task. It is evident that deep neural networks entail massive datasets to build and train feasible models. To gain massive and qualified datasets, nonetheless, seems expensive and time-consuming. This paper firstly examines the scale of a dataset so as to train an effective model as well as pertinent reactions corresponding to partial changes made to the scale of the dataset. In the practical experiments via training deep neural networks, we simplify hyperparameter tuning via transfer of pre-trained representations, hence promoting sample efficiency. To verify the usefulness of pre-trained models in datasets of different sizes, we have done relevant experiments on two benchmark datasets, cifar10 and cifar100. The results demonstrate that the larger the size of the pre-trained model, the better the fine-tuning effect of the network. With detailed analysis of primary elements contributing to high transfer performance, we aim to utilize pre-trained models with more efficient performance on dataset named as ImageNet-21k to benefit the computer vision research, in contrast to traditional models pre-trained on the smaller dataset, ILSVRC-2012.
Fichier sous embargo
Fichier sous embargo
0 5 14
Année Mois Jours
Avant la publication
mercredi 1 janvier 2025
Fichier sous embargo
mercredi 1 janvier 2025
Connectez-vous pour demander l'accès au fichier

Dates et versions

hal-04178721 , version 1 (08-08-2023)

Licence

Identifiants

Citer

Rui Li, Yang Tian, Qibin Chen, Xiangyu Zhu, Yongfei Jia, et al.. Does Large Pretrained Dataset Always Help? On the Effect of Dataset Size on Big Transfer Model. 12th International Conference on Intelligent Information Processing (IIP), May 2022, Qingdao, China. pp.136-147, ⟨10.1007/978-3-031-03948-5_12⟩. ⟨hal-04178721⟩
22 Consultations
2 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More