Detailed info

Single-Layer Transformers for More Accurate Early Exits with Less Overhead

AuthorsArian Bakhtiarnia, Qi Zhang and Alexandros Iosifidis
TitleSingle-Layer Transformers for More Accurate Early Exits with Less Overhead
AbstractDeploying deep learning models in time-critical applications with limited computational resources, for instance in edge computing systems and IoT networks, is a challenging task that often relies on dynamic inference methods such as early exiting. In this paper, we introduce a novel architecture for early exiting based on the vision transformer architecture, as well as a fine-tuning strategy that significantly increase the accuracy of early exit branches compared to conventional approaches while introducing less overhead. Through extensive experiments on image and audio classification as well as audiovisual crowd counting, we show that our method works for both classification and regression problems, and in both single- and multi-modal settings. Additionally, we introduce a novel method for integrating audio and visual modalities within early exits in audiovisual data analysis, that can lead to a more fine-grained dynamic inference.
ISSN0893-6080
Publication type:Journal
Title of journalNeural Networks
Year of publication:2022
Pages:461-473
Number, date or frequency of journal:Volume 153
Publisher:Elsevier
Urlhttps://doi.org/10.5281/zenodo.6737409https://doi.org/10.5281/zenodo.6737409
DOI10.1016/j.neunet.2022.06.038

Key Facts

  • Project Coordinator: Dr. Sotiris Ioannidis
  • Institution: Foundation for Research and Technology Hellas (FORTH)
  • E-mail: marvel-info@marvel-project.eu 
  • Start: 01.01.2021
  • Duration: 36 months
  • Participating Organisations: 17
  • Number of countries: 12

Get Connected

Funding

eu FLAG

This project has received funding from the European Union’s Horizon 2020 Research and Innovation program under grant agreement No 957337. The website reflects only the view of the author(s) and the Commission is not responsible for any use that may be made of the information it contains.