CONFERENCE PROCEEDINGS
Advanced Search
Home > Proceedings > Volume 13395 > Article
Open Access Paper
12 November 2024 ViT lightweight training at IoT edge based on transfer learning
Zhixin Li, Yang Long, Jiahao Miao
Author Affiliations +
Proceedings Volume 13395, International Conference on Optics, Electronics, and Communication Engineering (OECE 2024) ; 133951J (2024) https://doi.org/10.1117/12.3048338
Event: International Conference on Optics, Electronics, and Communication Engineering, 2024, Wuhan, China
Abstract
Recently, the vision transformer (ViT) model of deep learning has achieved surprising performance in the field of computer vision and has been widely used in IoT edge devices. However, the training of ViT models requires a large amount of data and computing resources, which is a challenge for resource-constrained edge IoT devices. To solve the above problems, this paper proposed a lightweight ViT method based on transfer learning. The primary concept of this method is to train large-scale ViT models in the cloud (CloudViT) and deploying small-scale ViT models at the edge (EdgeViT). Firstly, through the method of transfer learning, some underlying parameters of CloudViT were utilized to construct EdgeViT. The purpose is to enable EdgeViT to learn from CloudViT, acquiring knowledge and improving its performance. Secondly, adding a randomly initialized LayerNorm layer before the MLPHead during the training process of EdgeViT, it can improve further model performance. Finally, Experiment results demonstrated that EdgeViT could achieve 91.3% of CloudViT’s performance with only half the parameters and floating-point operations (FLOPs). Moreover, finetuning EdgeViT with a 60% reduction in training time still allows it to achieve 81.3% of CloudViT’s performance. Relevant conclusions can provide technical support for the proposed method.
(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.
Citation Download Citation
Zhixin Li, Yang Long, and Jiahao Miao "ViT lightweight training at IoT edge based on transfer learning", Proc. SPIE 13395, International Conference on Optics, Electronics, and Communication Engineering (OECE 2024) , 133951J (12 November 2024); https://doi.org/10.1117/12.3048338
PROCEEDINGS
 6 PAGES
DOWNLOAD PAPER SAVE TO MY LIBRARY
GET CITATION
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Data modeling
Clouds
Internet of things
Performance modeling
Instrument modeling
Neural networks
Head
RELATED CONTENT
Subscribe to Digital Library
Receive Erratum Email Alert
Back to Top