DESCRIPTION :
The Inria Rennes - Bretagne Atlantique Centre is one of Inria's eight centres and has more than thirty research teams. The Inria Center is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc.
Mission confiée
While machine learning models have achieved impressive results in recent years on many individual tasks (e.g. object recognition, classification, language models), they are obtained with static models that are not capable of adapting their behavior over time. In a dynamic environment, adapting the behavior of a model would require restarting the training process each time new data becomes available, quickly becoming impractical due to constraints such as storage and privacy issues.
Continual learning [2] studies such problems stemming from an infinite/incremental stream of data and the need to extend their behavior to additional tasks. The major challenge is to learn without signif- icant degradation in accuracy for previously learned tasks, a problem known as catastrophic forgetting. An added complication is the use of low precision arithmetic (e.g. sub 16-bit floating-point formats such as FP8), that can also have an impact on the performance of the model and on the choice of continual learning approach. This aspect of the impact of arithmetic on task performance in a continual learning scenario seems to have so far received little to no attention [3, 4], although if continual learning systems are to be deployed in the real world, especially on embedded or edge devices, such considerations will become paramount.
The goal of this thesis is therefore to investigate the performance impact of using low preci- sion arithmetic in the context of training and deploying continual learning systems on edge devices and propose task-aware number format precision switching strategies and custom hardware archi- tectures for continual learning tasks. The starting point will be implementing, testing, and adapting various low precision variants of continual learning methods (replay, regularization and parameter iso- lation). To do so, we envision using the Avalanche [5] continual learning library, which will integrate the mptorch [1, 6] framework developed in the TARAN team for doing custom precision computations during DNN training and inference.
The second and main objective of the PhD thesis will then be to validate the developed techniques trough a prototype of an accelerator for training in the context of low-precision continual learning. Synthesis of the specialized architecture on a target hardware platform will demonstrate the gains in per- formance and energy of the automatically generated accelerators. This parallel architecture will include configurable arithmetic operators implementing various precision and number representations as defined by an exploration methodology. Concretely, the architecture will first be validated in an FPGA accelera- tor for training based on previous work in the team [1, 6]. Second, an ASIC prototype will be designed to reach the highest energy efficiency. The team has such experience in designing custom chips, eval- uating performance and power consumption in advanced technology, and even going down to a silicon prototype (even if hardly reachable in the frame
of a PhD thesis). Our main focus is on energy-efficient embedded systems, such as autonomous vehicles, or on ultra-low-power IoT (Internet of Things) de- vices. A heterogeneous host-accelerator model of computation in which a fast accelerator (e.g., FPGA or ASIC) with support for low-precision arithmetic is connected to a slower general purpose host device (e.g., a RISC-V CPU) that can perform high-precision arithmetic.
References
[1] S. B. Ali, S.-I. Filip, and O. Sentieys. A stochastic rounding-enabled low-precision floating-point mac for dnn training. In IEEE/ACM Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1-6, 2024.
[2] M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuyte- laars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44:3366-3385, 2021.
[3] C. F. S. Leite and Y. Xiao. Resource-Efficient Continual Learning for Sensor-Based Human Activity Recognition. ACM Transactions on Embedded Computing Systems, 21:1-25, 2022.
Code d'emploi : Rectifieur (h/f)
Domaine professionnel actuel : Conducteurs Régleurs de Machines à Commande Numérique
Temps partiel / Temps plein : Plein temps
Type de contrat : Contrat à durée indéterminée (CDI)
Compétences : Intelligence Artificielle, C ++ (Langage de Programmation), Programmation Informatique, FPGA, Conception de Matériel, Hardware Platform Interface, Python (Langage de Programmation), Machine Learning, Object Detection, Parallel Computing, Reconnaissance de Formes, Reduced Instruction Set Computing, Recherche du Radical, Verilog, Hardware Description Language Vhsic (Vhdl), Application Specific Integrated Circuits, Deep Learning, Technologies Informatiques, Adaptabilité, Créativité, Motivation Personnelle, Innovation, Algorithmes, Architecture, Arithmétique, Systèmes Automatisés, Systèmes Embarqués, Réalisation de Prototypes, Recherche et Développement, Véhicules Autonomes, Réalisation de Tests, Internet des Objets (IOT), Capteurs, Performance Energétique, Optimisation Continue
Courriel :
com-ren@inria.fr
Téléphone :
0299847100
Type d'annonceur : Employeur direct
