Immersive Sound based on Head Related Transfer Functions

Description

PhD Thesis

One of the big pillars of the current industrial revolution is eXtended Reality (XR). This topic has attracted a lot of interest as much in academia as in the industry (Meta, Apple, Google, Nvidia, etc.), due to its potential socio-cultural and economical impact.

In the context of the collaborative Wal4XR project, which gathers 5 universities in Wallonia and Brussels around XR, this PhD thesis will focus on creating a system for generating personalized HRTFs (Head Related Transfer Functions) that is as simple, accurate, faithful and efficient as possible. The innovations targeted by the thesis will help to improve the automatic creation (without manual retouching) of high-quality datasets, models using photogrammetric data to create accurate 3D models of the ear, and the detection of anthropometric data from these 3D models. With all these elements, this thesis will attempt to clarify the understanding of the relationship between HRTFs and the morphology of a given individual.

The creation of personalized HRTFs is an operation that enables the sound of a virtual scene to be adapted to the morphology of any listener, thereby improving the sense of immersion. Automatic 3D modelling of the ear based on photographs (or photogrammetry [1]) and automatic detection of anthropometric data [2] or morphing of a high-resolution 3D ear mesh with anthropomorphic data [3] are some of the methods used to create these custom function sets. On the other hand, as far as HRTF dataset formats are concerned, although the SOFA standard [4] was defined in 2015, it does not include anthropomorphic data, the structure of which is left up to the individual.

[1] Dellepiane M, Pietroni N, Tsingos N, Asselot M, Scopigno R. Reconstructing head models from photographs for individualized 3d-audio processing. In: Computer Graphics Forum. Vol. 27. Hoboken, New Jersey, United States: Wiley Online Library; 2008. pp. 1779-1727. DOI:10’1111/ j.7467 -8659.20 0 8. 01316.x [2] D. Fantini, F. Avanzini, S. Ntalampiras and G. Presti, “HRTF Individualization Based on Anthropometric Measurements Extracted from 3D Head Meshes,” 2021 Immersive and 3D Audio: from Architecture to Automotive (I3DA), Bologna, Italy, 2021, pp. 1-10, doi: 10.1109/I3DA48870.2021.9610904. [3] K. Pollack and P. Majdak, “Evaluation of a parametric pinna model for the calculation of head-related transfer functions,” 2021 Immersive and 3D Audio: from Architecture to Automotive (I3DA), Bologna, Italy, 2021, pp. 1-5, doi: 10.1109/I3DA48870.2021.9610885. [4] P. Majdak, Y. Iwaya, T. Carpentier, R. Nicol, M. Parmentier, A. Roginska, Y. Suzuki, K. Watanabe, H. Wierstorf, H. Ziegelwanger, and M. Noisternig, “Spatially Oriented Format for Acoustics: A Data Exchange Format Representing Head-Related Transfer Functions,” Paper 8880, (2013 May.)

Your Mission

• Carry on research towards the above mentioned objectives
• Generate resources displaying your research work and make it available to the community for reproducibility purposes
• Participate in various meetings and events in the context of the Wal4XR project

Your Profile

You hold a Master in Computer Science, Electrical Engineering or equivalent, ideally focusing on one of the domains of interest here.

Required Skills

• Comfortable working both autonomously as well as in a team
• Able to adapt and learn new skills quickly
• Good oral and written communication skills
• Background or some expertise in more than one of the following domains and are interested in learning the others: machine learning (deep learning specifically), statistics, 3D modeling, acoustics
• Good programming skills in Python

Ideal Skills

• Good programming skills in C# and C++
• Proven record in research, software engineering, software development, machine learning, 3D modeling, acoustics, or XR related fields in general

What we offer

• A motivating research project in an internationally recognized lab at UMONS, in the framework of an inter-university project gathering 10 PhD students on XR issues.
• 4 years of PhD salary (Bruto around 3950 EUR/month; Netto around 2500 EUR/month depending on your status)
• Starting on Sept 1st, 2024.
• A comfortable holiday package (30 days off/year, including official holidays, plus winter break)

Interested? Send email with CV and motivation letter to Prof. T. Dutoit (thierry.dutoit @ umons.ac.be)