Behind the Scenes of VFX Workflows

🧠✨ Blending AI with VFX (02) – Real-world Applications in Flame + ComfyUI + SynthEyes: Extracting 3D Geometry from Image How many times in VFX do we wish we had access to the actual geometry of a scene element… and there’s no one to ask? no 3D team, no scan, no model, just the plate. Well, that’s changing, check some quick examples I made. We can now combine AI-based img to 3D extraction in ComfyUI, camera tracking with Syntheyes from Boris FX , and compositing in Autodesk Media & Entertainment Flame. The result? A smarter, faster, more creative pipeline that makes a huge difference when we deal with product replacement, lighting, texture projection or particle interactions. Modeling a complex object is time-consuming, but now we can generate a 3D geometry from a single image using AI tools. This not only speeds up tasks like screen replacement, glow projection, and interactive FX... it also makes things possible that might’ve been skipped due to time constraints. 🔧 The process: 1️⃣ The shot is tracked in SynthEyes, a classic that’s now embracing AI beautifully: with its own machine learning matte generator, there's no need to roto for matchmoving. You can focus entirely on the track — as it should be. 2️⃣ Grab a carefully selected still from the shot and generate a 3D model using AI-based geometry estimation. I’m currently using the Hunyuan workflow, and after testing, around 80% of the meshes were either good enough or straight-up amazing. The sheer amount of usable geometry you can generate in a single day is… dazing. 3️⃣ Mesh and tracking data are imported into Flame, where compositing begins, now with actual geometry positioned in 3D space using the same camera from the shot. This gives you the power to relight, project textures, and drive particle systems with total control of the scene. ⏱️ Time saved is real, especially when the subject isn’t a cube — but a custom product, a car, or even a human body. And the best part? Quality improves too. Faster and better? That’s a rare win-win in post. 🛠️ What can we do with this geometry? 🔹 Realistic product relighting or replacement 🔹 Texture and reflection mapping 🔹 Particle interaction 🔹 Scene reconstruction for set extensions 🔹 Projection-based cleanup & paintwork 🔹 Custom glows, shadows & light wraps that stick The pipeline is smoother and faster than you'd expect. The results are production-ready. And the creative potential? Only growing... Hunyuan shows amazing understanding of shape and occlusion but there’s always a next gen coming: Amodal3R (code soon to come) looks very smart at predicting occluded elements... and this just keeps evolving. Just another reminder: AI isn’t a buzzword — it’s a tool. When paired with the right software and real VFX experience, it becomes a true force multiplier. More to come! 😉 🐵🚀🪐 #VFX #AI #Compositing #AutodeskFlame #SynthEyes #ImageTo3D #Relighting #VirtualProduction #3DGeometry #VFXWorkflow #Hunyuan #Nuke #ComfyUI

VFX Budgets in 2026: Beyond the AI Hype and the Reality : As a VFX Supervisor, I’m constantly being told that 2026 is the year AI and Virtual Production make everything "faster and cheaper." But when I look at our current shot loads and the files my team is actually dealing with, the reality is a massive Budget Paradox. I’ve spoken with peers across the globe, and the sentiment is the same: technology is moving at light speed, but the manual effort required to hit "Final Pixel" hasn't decreased. In many cases, it’s actually made our jobs more complex. Here is what my team and I are experiencing on the ground: 1. The "100% Effort" Trap: Fixing the AI Producers see a "one-click" AI roto preview and think the shot is 80% done. But as a Supervisor, I cannot final a shot with "temporal chatter" or vibrating edges. The Reality: Most AI tools give us pixels (raster) instead of points (vectors). If the machine makes a mistake, my artists can’t just "nudge a spline." The Result: We often end up throwing away the AI mask and doing it traditionally from scratch. We are paying for the AI license, but I am still paying my senior artists for 100% of the manual effort to make it usable. That is the "Cleanup Tax." 2. Virtual Production and the "Double-Comp" We were promised that LED Volumes would give us "Final Pixel" on set. In reality, "Final Pixel" is often a myth that creates more work for the Compositing department. The Reality: If the LED wrap isn't perfect or the color shift is off, we are stuck rotoscoping actors off complex digital backgrounds instead of clean green screens. The Paradox: We spend a fortune on the LED stage, only to spend it again in post to fix the very background that was supposed to be finished. 3. The Scope Trap: Because the "paper" cost of VFX looks lower due to AI, shot counts are exploding. A show that used to have 600 shots now has 1,500. Even if a tool saves 10% of the time, a 200% increase in volume means my team is under more pressure, and total budgets are still climbing. Strategies for the 2026 Landscape: To the Juniors: Don’t let the software make you lazy. Master the manual fundamentals. Your value in 2026 is being the one who can finish what the AI breaks. To the Seniors: We have to be the voice of reason. If a shot is a "trap," we need to call it out early. Stop letting the hype dictate the bid. Conclusion: Let’s Hear Your Side I know my team and I aren't the only ones feeling this. I want to open this up to the global VFX community: Are you seeing "80% AI results" that actually take 100% manual effort to fix? How many of your "Final Pixel" Virtual Production shots ended up being full-blown Comp jobs anyway? Is your studio actually seeing budget savings, or just shifting the debt from one department to another? Let’s be honest about the state of the craft. Share your experiences below—I’m curious to see if this is as universal as it feels. www.balafx.com

Part 8 of my Q & A Series Q. How Do You Keep AI Images Consistent Across an Animated Short Film Series? I’ve been developing a series of AI-assisted animated short films that rely on visual continuity as much as storytelling. Every scene needs to belong to the same world—same mood, same light, same story realism. AI IS MERCURIAL. PHOTOSHOP IS PREDICTIVE Consistency is easy to lose once you start mixing tools. One image emerges glowing and cinematic, the next looks like it wandered in from a Saturday-morning cartoon. My reference point for cohesion is Ciro Marchetti—his worlds are rich, luminous, and believable. Reaching that level of cohesion across multiple AI frames takes more than a strong prompt. It takes direction. What follows is the workflow I use to keep an AI-generated film series visually consistent: using MidJourney to generate, and Photoshop to refine, balance, and ground the results in one cinematic universe. I. MidJourney is a generator. Photoshop is a director Generative tools are great for “let’s see what happens.” You type in a prompt, cross your fingers, and sometimes you get gold. Diffusion models reinterpret your prompt every time—tiny changes shift the entire look. If you’re maintaining consistency across characters and scenes, that unpredictability works against you. II. Photoshop doesn’t guess. It lets you decide. When you composite in Photoshop: • You control scale, lighting, and placement. • You paint realistic shadows and reflections instead of hoping the algorithm gets it right. • You match tone and texture with a few smart adjustments instead of wrestling with “style strength.” III. The best workflow blends both worlds. Here’s what worked for this cinematic AI project: 1. Design characters and backgrounds in AI. Use #Midjourney, #OpenArt, #Seedream, etc. to generate assets at high resolution with clean edges. 2. Composite in #Photoshop. Drop every element into its own layer. Resize, relight, and blend. • Use Match Color to unify lighting. • Add a light Gaussian Blur to push distant figures into depth. • Paint shadows manually—AI sometimes misses those. 3. Run a cinematic regrade. Once your composition looks natural, export it and regrade (Seedream 4.0) for tone, light, and atmosphere. 4. Animate in #LumaDreamMachine. Luma handles camera motion with cinematic depth. 5. Finish in #CapCut or your preferred editor. Add timing, music, and sound. IV. Why this matters AI is mercurial. Photoshop is predictive. If you’re building a narrative world, you need both. V. Final Thoughts I’m not anti-AI. I’m anti-automatic. AI tools are brilliant, but they still need human direction. The best results come when you treat them as collaborators—AI for ideas, Photoshop for intention. So when your next scene looks too soft, too plastic, or too “AI,” don’t scrap it. Open Photoshop. Adjust the light. Fix the edges. That’s where your film starts to feel real. #lyndacathcart #motiondesigner #aianimation #curio

I just released a new podcast episode with Lynette Duensing, and it’s one of those conversations that really pulls back the curtain on what happens before a film is finished. Lynette is a senior colorist working behind the scenes where creative intent meets studio marketing LONG before final grading session. We talk about the reality of grading trailers when VFX are incomplete, timelines are unforgiving, and every second has to communicate tone, story, and scale. We dig into her real workflow: • Building looks around show LUTs, CDLs, and ACES so filmmakers stay oriented • Grading temp VFX live in the room to ground notes in final intent • Handling missing mattes, unfinished comps, and constant revisions without locking in mistakes • Why organization, version control, and trust are critical to running a successful session What stood out most to me is how much of this job is about communication and decision-making under pressure. Trailer shots often never exist next to each other in the feature, yet they still have to feel cohesive, intentional, and believable. If you’re a colorist, filmmaker, or post-production creative, this episode offers a rare look at the space between editorial, final DI, and the craft it takes to make chaos feel intentional. 🎧 Listen and Watch here: https://lnkd.in/gGfX-53r Would love to hear what part of the color or post process you’d want unpacked next.

Here’s how those insane body morphing shots in ‘Together’ were made --> In Michael Shanks’ 'Together', one of the independent film’s most striking kinds of visual effects centers around ‘body morphing’. The story’s central conceit—characters physically merging and transforming—demanded a highly technical approach that blended on-set planning with intricate digital artistry. That becomes most apparent when on-screen (and real life) couple Millie (Alison Brie) and Tim (Dave Franco), having experienced this body morphing phenomenon by becoming literally stuck onto each other, eventually accept their fate and fully fuse. The shots would involve Brie and Franco performing as much of the scenes as possible, with CG take-overs then required to show the body parts ‘morphing’ together. Overseeing that work and other VFX in the film was visual effects supervisor Genevieve Camilleri. The principal vendors were Framestore (visual effects supervisor Josh Simmonds), RISE | Visual Effects Studios (Camilleri was the VFX supervisor here), Basilic Fly Studio, Avatar Factory (Mark Ruff) for scanning and Kyle Ashley (previs). Director Michael Shanks and assistant editor Oliver Dear also personally contributed a significant number of visual effects shots, mostly involving clean-up. Here at befores & afters, go behind the scenes with lots of fun details, and behind the scenes images and videos! https://lnkd.in/eT2BbTYc

Snow Wall VFX Breakdown | UE5 This entire sequence was rendered inside UE5 using a combination of Alembic geometry caches and VDB volumes. The fractured snow wall was driven by Alembic, while the powder and atmospheric breakup were handled through VDBs. Lighting, camera work and final rendering were all completed in engine. The real challenge was not just the destruction itself, but making the interaction feel believable. The weight of the truck, the timing of the impact and the density of the volumetric snow all had to work together. Being able to iterate in real time made a huge difference. Adjusting lighting, tweaking volume density and refining the camera could all happen instantly, which allowed the shot to evolve naturally. #UnrealEngine #UE5 #VFX #BehindTheScenes #AutomotiveCGI #VirtualProduction

What happens when a physical model-maker builds locations for virtual production? You get workflows like this. This is an example of how photogrammetry can be used inside of the art department; capturing hand-built models and developing them further in Unreal Engine for visualization and design. In Ahsoka, Jason Mahakian built this beautiful foam model (bottom left). Giovanni Shull captured it via photogrammetry (bottom right), and the result was a 3D version that we art-directed directly in Unreal Engine (top right). While this didn’t end up on the LED stage, it was used in VFX, we passed off the Unreal Engine and Maya textured versions. They virtually scouted it together, to inform layout, and drive the creative. The Art Department doesn’t stop at physical anymore - it's evolving into fully realized 3D spaces that guide action, inform shots, and serve the team all the way through post. Learn more about this process in our Ahsoka article: https://lnkd.in/g2GMFkRv #VirtualProduction #UnrealEngine #Photogrammetry #SetDesign #ArtDepartment #photogrammetry #filmmaking

I had the honor of collaborating with Erik Weaver and the ETC team on a white paper for Europa, a short film exploring advanced virtual production workflows. The paper is finally out and ready to download! Some highlights: 💫 Europa explored OSVP 2.0: metadata/ontology workflows, AI environments, remote collaboration, IBL, and camera/lens metadata optimization. 🎬 Chosen by ETC - Entertainment Technology Center @ USC for educational value and testing black levels on LED panels for realistic storytelling. 💫 Pre-production involved AI tools like Stable Diffusion AI, collaborating with the VAD to create digital/physical environment integration. 🎬 Metadata and ontology systems via Konsol enabled real-time data integration between physical and digital elements. 💫 Collaboration with Sony’s Innovation Studios Stage 7, using Pixomondo’s LED volume and Torchlight for previsualization. 🎬 Cinematography tackled anamorphic lens distortion, IBL, and integrating physical/virtual lighting via DMX and RGB-HSV conversions. 💫 Cloud-based workflows enabled 24-hour production cycles with remote New Zealand and India teams. 🎬 AI tools (Runway ML, Cuebric, Topaz Labs) used for generative content, compositing, upscaling, and editorial enhancement. 💫 AI-driven metadata systems enhanced asset management and scene identification with Avid and Universal Studios. Interviewees include: Erik Weaver, Tom Thudiyanplackal, Christina Lee Storm, Jacqueline Elyse Rosenthal, Erik "Wolfie" Wolford, Vanessa kao, Angran Li, Kristin Wang, Eric B. Roth, Dane Brehm, MC D., Tim Kang, Daniel De La Rosa, Spencer Stephens, Pete Harrow, Devon Mathis, Jason Fotter, Jeremy Stapleton, ASA BAILEY, Gary Palmer, Tony Martinez, Ian Sansavera, Steve Mangurten. Companies mentioned: Sony, PIXOMONDO, Torchlight, CG Pro , ROTU, Qube Cinema, Davant Systems, The Rebel Fleet, Konsol, Aputure, Bind Studio, Cintegral, Runway, Electric Sheep, Universal Studios Hollywood, Avid. This publication is 100% free, so please grab your copy here: https://lnkd.in/gdcH-CFP #virtualproduction #ledwalls #cinematography #ai #usc #education #filmmaking