DC9: Physics-based generative modeling of surface appearance from multimodal imaging

Doctoral candidate

To be recruited

Hiring institution

University of Burgundy Europe (UBE)

PhD enrolment institution

University of Burgundy Europe (UBE)

Main supervisor

Alamin Mansouri (UBE)

Co-supervisor

Laura Brambilla (HES-SO)

Co-supervisor

Clotilde Boust (CNRS)

Objectives

Develop a new physics-inspired generative models trained on multimodal appearance data to create digital renderings that remain faithful to the underlying optical and structural properties of real objects, useful for HDTs:

• Create and curate multimodal datasets (RTI, multispectral and 3D imaging) of surface appearance, enriched with metadata describing illumination geometry, and contextual conditions.
• Design a hybrid generative architecture combining Transformers for image synthesis (TransGAN) with a Mixture of Experts (MoE) strategy, where expert specializes in a specific dimension of appearance.
• Develop physics-inspired training protocols integrating expert supervision, enabling iterative refinement and control of generated images with on perceptual and physical validation criteria.

Goals

• Multimodal reference datasets with standardized acquisition and annotation protocols (geometry, spectral data, lighting parameters).
• A novel generative model trained from physically grounded data.
• Proof-of-concept applications demonstrating physically faithful appearance generation for virtual restoration, digital replicas, and museum visualization environments.