Lumen Media Mapping

Camera Mapping in UE5

Tool
x
Electric Noodles

Lumen Media Mapping

Camera Tracking

Real Time Compositing

Camera Mapping

Real Time Relight

Lumen Media Mapping

Lumen Media Mapping is a technic that the Electric Noodles team developped using the new Lumen technology available in Unreal Engine 5.

We map the video on a 3D reconstruction of the environment and relight the 3D integrated objects with Lumen based on video colors. This gives us a 1:1 lighting and reflections fidelity with the backplate footage that is used.

Real Time Compositing

We used compositing in unreal engine to add 3D elements that are correclty placed in the perspective and respecting the lighting of the original footage.

We were able to extract the depth of the footage using the tracking data and use that depth layer for integrating the holograms and the fog.

Given the short timeline of the project, we didn't have time to rig and skin the character for real time use, so we used the USD scene technology offered by NVidia to directly transport our Blender scene to Unreal Engine seamlessly with all of the informations (shaders, animations, scale, etc...)