Modeling Interpersonal Interactions in Multi-Party

There is a growing interest for automatically analyzing, modeling and synthetizing non-verbal behaviors at the group level (i.e. several participants). Structure of the group could be static or dynamic with at least three participants. All participants adhere to specific social norms governing, for example, their distance and body orientation in order to coordinate, to make it easier to interact with each other and to mark their social attitude toward each other.
Turn-taking models have been proposed to manage speaking turns and interruptions among partners (Ravenet et al, 2015). Within a group, participants adapt their spatial position and orientation, their turn-taking and communicative behaviors in function of their intentions and social attitudes.
Interpersonal interaction can be qualified through automatic analysis of social signals exchanged between partners (Delaherche et al. 2012). The concept of interpersonal synchrony has been employed to model dyadic and group interactions. A number of quantifiers has been proposed to characterize different aspects of synchronization between partners within an interaction as well as of the interaction itself. Synchronization measures can serve to quantify interpersonal interaction.
Most of works in human-agent interaction focus on dyadic interaction (Leite et al, 2012). However, few addresses group interaction either involving several agents or humans (Traum et al, 2012). Among the existing works we can note models for turn-taking management (Ravenet et al. 2015), gaze models (Oertel and Salvi, 2013), dialog management (Traum et al., 2012), social skills (Prada and Paiva, 2009). Very few works look at modeling behaviors for social agents while interacting with other agents or humans (Skantze et al. 2015 ; Klotz et al. 2011). This training is set in this research topic. It will focus on group of agents without adding humans within the interaction loop to ensure achieving first result in the time of the internship. This internship would lay the first foundation for future research.

The main objective of this internship is to develop group interaction models for social agents (Greta agents). This will be performed in several steps: (1) analyzing and learning interpersonal patterns emerging from the interaction in multiparty interaction of humans, (2) proposing new metrics to measure interpersonal synchrony, (3) implementing the learned model of interpersonal synchrony in multiparty interaction of virtual agents.
The evaluation of the models will be performed by (1) adequate questionnaires and (2) measurement of interpersonal patterns.


Delaherche, E.; Chetouani, M.; Mahdhaoui, A.; Saint-Georges, C.; Viaux, S.; Cohen, D., "Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines," in Affective Computing, IEEE Transactions on , vol.3, no.3, pp.349-365, July-September 2012

Klotz, D., Wienke, J., Peltason, J., Wrede, B., Wrede, S., Khalidov, V., & Odobez, J. - M. (2011). Engagement-based Multi-party Dialog with a Humanoid Robot. Proceedings of the SIGDIAL 2011 Conference, 341–343.

I Leite, C Martinho, A Paiva, Social Robots for Long-Term Interaction: A Survey, International Journal of Social Robotics, 5(2), 2013, 291-308.

Catharine Oertel and Giampiero Salvi. 2013. A gaze-based method for relating group involvement to individual engagement in multimodal multiparty dialogue. In Proceedings of the 15th ACM on International conference on multimodal interaction (ICMI '13). ACM, New York, NY, USA, 99-106.

Rui Prada, Ana Paiva, Teaming up humans with autonomous synthetic characters, Artificial Intelligence 173 (2009) 80–103

Brian Ravenet, Angelo Cafaro, Beatrice Biancardi, Magalie Ochs, Catherine Pelachaud, Conversational Behavior Reflecting Interpersonal Attitudes in Small Group Interactions, International Conference on Intelligent Virtual Agent (IVA2015), Delft, Netherlands, August 2015.

Skantze, G., Johansson, M., & Beskow, J. (2015). Exploring Turn-taking Cues in Multi-party Human-robot Discussions about Objects. In Proceedings of ICMI. Seattle, Washington, USA.

D Traum, D DeVault, J Lee, Z Wang, S Marsella, Incremental dialogue understanding and feedback for multiparty, multimodal conversation, International Conference on Intelligent Virtual Agents, 275-288

ISIR - Sorbonne Université
Catherine Pelachaud
Mohamed Chetouani
Référent Universitaire: 
2 019

User login