DESCRIPTION :
The STARS research team combines advanced theory with cutting-edge practice, with a focus on computer vision systems.
Team website: https://team.inria.fr/stars/
Scientific context
"Actions speak louder than words". Humans are complex beings, and they often convey a wealth of information not through their words but through their actions and demeanor. Non-verbal behaviors can offer crucial insights into their emotional state, pain level, or anxiety, often more eloquently than words alone [1]. The analysis of non-verbal communication is of critical importance in the diagnostic landscape. Let us imagine toddlers who struggle to describe the intensity of their abdominal pain. However, they non-verbally express their agony by tightly gripping their abdomen, thus indicating the severity. The inability to express pain verbally due to cognitive limitations, language discrepancies, or emotional distress can be compensated for by analyzing non-verbal cues effectively.
However, decoding non-verbal cues in a clinical setting is not a straightforward task. It relies heavily on a high degree of inference. It requires healthcare professionals to be astute observers, picking up on nuances that may be subtle yet critical. For instance, a slight furrow of the brow could signify discomfort or concern, and a patient's posture may reveal signs of tension or distress. The challenge lies in accurately interpreting these cues, as they can vary greatly from one individual to another.
To address this challenge, automated systems capable of detecting non-verbal behaviors and their corresponding meanings can assist healthcare providers. Such technology acts as a supportive tool for medical experts, enhancing patient assessments and ensuring that critical information is not overlooked or misunderstood. In essence, recognizing and interpreting non-verbal signs is essential for holistic patient care, and advanced technology can augment its accuracy and effectiveness, leading to improved diagnosis and treatment outcomes.
Mission confiée
The primary objective of this research is to lead the development of an advanced AI model for Human Behavior Understanding [2] to identify non-verbal cues expressed by patients, and then interpret the cues to derive critical insights about their health. Traditionally, computer vision methodologies encompassing skin color analysis, shape analysis, pixel intensity examination, and anisotropic diffusion were used to identify body parts and trace their activities. However, these algorithms provided limited flexibility because of their domain-specific nature. Deep learning methods can be used to deal with this issue as they offer more training flexibility and better performance results. The overarching goal is to provide a real-time, data-driven analysis
of non-verbal cues exhibited by patients during clinical interactions, thereby delivering invaluable insights to healthcare practitioners.
Approach:
In this work, we aim to develop a novel data-driven, deep learning model to analyze the behaviors of patients during clinical interactions. These interactions can be in the form of single-view videos, which would contain comprehensive information about the overall behavior of a patient being examined. Traditional object detection-based approaches are centered on the two-shot object detection methodology. This method deals with identifying the regions that contain the object, and then refining the region proposal information to perform multiclass classification [3]. A better approach to this could be using a single-shot detection algorithm, as was proposed through YOLO [4]. However, while creating bounding boxes around the desired objects and predicting class probabilities is useful, it is not enough to capture the dynamics of the non-verbal cues and their clinical interpretation.
To deal with this problem, novel foundation models [5] could be leveraged to encode visual features in the videos and perform resilient behavior tracking and understanding. We propose a self-supervised transformer model for semantically segmenting the body parts, tracking their real-time location, obtaining critical positional and behavioral information, and decoding them to perform thorough clinical analysis through a linear classification backbone [8]. The self-supervision would enable the model to learn independently despite having scarce data, and provide optimum prediction results in a computationally efficient wireframe.
By the culmination of this work, the aspiration is to contribute substantially to the advancement of an AI system that augments clinical communication by offering a technically refined analysis of non-verbal cues. This undertaking not only bears the potential to enhance medical diagnostics but also extends its applicability to diverse domains necessitating comprehensive non-verbal behavior analysis, including human-computer interaction paradigms and scholarly research in the realm of psychology [10]. This work will be conducted within the CoBTek team from Nice Hospital, which is specialized in clinical trials for autistic children.
The project is based on filmed assessments recorded as part of current clinical practice at the Centre Ressources Autisme des Hôpitaux Pédiatriques Nice CHU-Lenval, linked to the CoBTek laboratory. The research is part of the ANR ACTIVIS project. The project brings together multimodal skills in AI, body movement analysis, and linguistics.
Code d'emploi : Thésard (h/f)
Niveau de formation : Bac+5
Temps partiel / Temps plein : Plein temps
Type de contrat : Contrat à durée indéterminée (CDI)
Compétences : Intelligence Artificielle, Vision par Ordinateur, C ++ (Langage de Programmation), Encodages, Programmation Informatique, Linux, Interaction Homme Machine, Python (Langage de Programmation), Machine Learning, Object Detection, OpenCV, Tensorflow, Tests Basés sur les Données, Website Wireframe, Pytorch, Deep Learning, Technologies Informatiques, Anglais, Sens de la Communication, Leadership, Gestion du Stress, Enthousiasme, Stabilité Émmotionnelle, Esprit d'Équipe, Implication et Investissement, Innovation, Algorithmes, Autisme (Connaissance Professionnelle), Systèmes Automatisés, Travaux Cliniques, Diagnostic (Médecine), Pratiques Cliniques, Recherche Clinique, Soins de Santé, Psychologie, Linguistique, Mathématiques, Paradigmes, Recherche Scientifique, Réalisation d'Évaluations, Détresse Émotionnelle
Courriel :
Francois.Bremond@inria.fr
Téléphone :
0139635511
Type d'annonceur : Employeur direct
