Connecting ideas at FMX 2024

While it was fun to meet old and new friends, groundbreaking technologies and their applications were the highlight at FMX 2024. Artificial intelligence is rapidly transforming the media landscape. FMX featured advancements in AI-powered tools for animation, character rigging, content creation and project management. While ethical considerations surrounding AI use were addressed, it is very likely that it will play a major role in the industry and might even revolutionise many processes.

Founder and CEO of AvatarOS Isaac Bratzel presented his innovative and visionary ideas in the fields of AI and digital humans. He spoke about how he and his team refine blendshape animations using machine learning and 4D data to match the movements of a specific character. He also gave an overview about cutting edge AI generated avatars and voices by Synthesia, HeyGen, Soul Machines and Ready Player Me. Of course Meta’s Codec avatars, Apple’s Persona avatars and Google’s Starline avatars were mentioned as well.

From Bilawal Sidhu (Billyfx) we learned how AI tools can be integrated into creative workflows. Visual AI like Runway ML, Pika Labs and Midjourney open up new possibilities for everyone to create their own films with advanced text-to-video tools that help realise any creative vision in any style. Even new visual styles were generated by AI such as ‘Forestpunk’. Text-to-audio and lip sync AI tools help bring characters to life. Users can define specific movements with the motion brush in Runway ML. There are image-to-image and image-to-video tools as well, basically anything-to-anything is possible.
Popular web-based tools for spatial AI like NeRFs and 3D Gaussian Splats are Luma AI and Polycam. Postshot Jawset runs locally, can handle large data sets and can import alignment components from Metashape and Reality Capture. We can edit 3D Gaussian Splats with the GPU-based Irrealix plugins in After Effects.

A growing trend 2024 is the focus on open source software and open standards to support collaboration and interoperability across the media industry. The power of open source development was a key point in Shehzan Mohammed’s talk, Director of 3D Engineering and Ecosystems at Cesium. He gave valuable insights into open standards for virtual worlds and Cesium’s unique approach to advance 3D geospatial technologies. The Cesium ion platform is using the open standard 3DTiles and hosts ready to stream global 3D content. Integrations for the Unreal Engine and Unity are supported and the open source 3D world map can be combined with own 3D data. Shehzan Mohammed presented dynamic real time applications such as our virtual scouting application VS Scout.

VS Scout provides a realistic and dynamic digital environment. It supports 3DTiles and custom 3D scans. Filmmakers are empowered to virtually scout locations, saving time, money and resources. Guesswork gets replaced with precision in pre-production. This innovative technology fosters informed decision-making, efficient collaboration and opens doors to a wider range of scouting possibilities, ultimately leading to the selection of the perfect setting for a film.

Virtual Production received significant attention at FMX. Paul Debevec presented his latest work that combined 4D capture with Virtual Production at Eyeline Studios.
Optitrack, Holobay and ARRI collaborated in combining Virtual Production with 3D Gaussian Splats in Unreal Engine. The light reflections and material properties of the 3D Gaussian Splat scene on the LED wall changed physically accurate and in real time depending on the movements of the physical camera. There is huge potential in merging the best of these new technologies for future productions.

Vfx Supervisor Charley Henley gave a brilliant talk about the invisible Vfx of Ridley Scott’s Napoleon. Thank you for the shout out, Charley.

Innovative research projects were presented, such as the MAX-R, the Mixed Augmented and eXtended Reality media pipeline at the Animation Institute in Ludwigsburg. During this program researchers developed a pipeline of tools to create high quality XR content. This includes interactive tools for virtual production and machine learning based character animation generation. Open APIs and interoperability are a main topic here.

Other presentations explored the use of virtual and augmented reality for storytelling, immersive experiences and live events. Esteban Chacin from Moment Factory showcased the immersive nighttime experience in the Dome des Invalides in Paris. The video mapping and light projection show was masterfully done, from a technical perspective as well as artistically. 3D scans of the historical building supported the planning and production.

The lines between traditional filmmaking and interactive media are getting more and more blurred. FMX featured discussions and demos on technologies that create interactive experiences within films and explore the narrative possibilities of video games. This convergence promises to redefine how audiences interact with stories. Erasmus Brosdau, Director of Black Amber, gave an overview about probably one of the largest cinematic productions fully created in Unreal Engine called ‘Gundam’.

Some presentations explored ways to reduce the environmental impact of large-scale film and animation projects, highlighting the industry’s commitment to a greener future.

For the latest updates, behind-the-scenes insights, and a deeper dive into the world of visual effects and 3D scanning, we invite you to connect with us on our social media platforms.

Follow us and be part of our community where we regularly share our experiences, challenges, and triumphs in the ever-evolving realm of film and VFX production. Don’t miss out on the opportunity to see how technology and creativity come together to create cinematic magic.

👉 Follow us on Instagram
👉 Like our Page on Facebook
👉 Subscribe to our YouTube Channel
👉 Connect with us on LinkedIn