■1: Gaussian splats
Gaussian splats: much ado about nothing?
I have seen a lot of gaussian splats demonstrations, and very few impress me. There are so many weaknesses.
You can’t edit it.
You can’t get a mesh from it. (Not cleanly)
Reflections and lighting are baked-in. There are re-lighting solutions, but I don’t believe they’re very robust.
There’s no refraction.
No AOVs.
No shadow casting.
You can’t render them from an angle you don’t have a photo from. So… why not just use the photo?
Well, because you can use it as 3D on web or in VR! But web 3D seems still only to be a clunky gimmick and VR hasn’t taken off. (Is it just not that cool?)
Volumetric video is fun, but I can’t see a great use for it yet. If you know any, let me know on Substack.
That said, I think there are good uses for splatting, and I’m collecting them as I see them.
Maybe the most useful part is being able to animate a camera through a space unlike how you could move it for real. For example, I don’t have a dolly. It’s very hard to get a nice smooth camera move through a space. But if I splat it, I can animate that smooth camera in Blender.
Another benefit is that you can capture reflective things. For an upcoming video, I wanted to photoscan my camera, but that’s impossible. So I made a splat instead. I wish it would react to CG lights, but it’s better than nothing.
Finally, the files are legit tiny and they run on anything. I still think 3D on web is a gimmick (almost always, photos are better), but I can imagine uses that aren’t. Museums, for example, could benefit a lot. Until now, a lot of them have been making photogrammetry, which just doesn’t look like the original objects.
By the way, I found this article very helpful: Gaussian Splatting for VFX: The Honest Guide | CG Lounge
Tutorial: Rig a gaussian splat
This tutorial is for Houdini, but I know a lot of it will work in Blender. I’ve done some tests with rigging gaussian splat point clouds with geometry nodes, and it absolutely works.
Resource: 3D data
Open Heritage 3D has free-to-use 3D data of a lot of cultural heritage sites. Usable for photogrammetry and gaussian splatting.
Tip: Save renders as splats
Archiving 3D scenes is a nightmare. Have you ever tried opening a two-year-old project?
I think you could easily save your scene as a gaussian splat. Animate a camera through the entire scene and render over-night. Then train a splat on it. The result is a very small file that looks like the finished render from every angle. a really nice way to archive old projects in a way that you know will look correct and will take up very little space.
I once encounter an eery video using splats, I think I still have it on my disk, it was one of the very early experimental artistic use of splats when the tech wasn't very good, feel like those early AI experimental video. It's a video of a person walking toward a building with glass sliding doors, as they approaches it the glasses reflection start to get weird, then they finally walk into the reflection and look back outside, seeing the 'back' of the reflections splats, the ambience sound and reverberation changes when they poke the camera through the glass too so it feels like those backroom videos. That was 2-3 years ago iirc.
I work in Augmented Reality (AR) and I am very excited about Gaussian Splats because it means you can have photo-realistic 3D video in realtime.
Currently, I use 3D models (GLB type) and greenscreen videos. 3D models don’t look as good in AR because it’s realtime rendering (although it’s getting better!) and greenscreen videos either have to physically rotate to always face the viewer (kinda weird) or they end up looking like Flat Stanley if you walk around them. The most common use case for greenscreen videos is people can film themselves on a greencreen and then key out the green to become a hologram (one tool that does the keying for you is an AR publishing platform called Hoverlay).
With Gaussian Splats, it’s the best of both! If someone wants to be a realistic 3D hologram in Augmented Reality, they soon can.
While I won’t bore you with why I’m convinced AR is the future, the use cases ARE endless!