Working with 3D Assets

StereoKit’s core 3D asset format is the GLTF! While there is still support for other formats, like STL, OBJ, PLY, and SVG, GLTF is StereoKit’s preferred format of choice. It has a well defined modern specification, and a large collection of quality tooling available in the ecosystem.

GLB is still just the “binary” format of GLTF, typically with all related textures bundled inside it.

I’ve found Blender to have a very nice GLTF exporter! So if your tool of choice doesn’t support GLTF yet, exporting to Blender for a final pass can be a good approach! However, most tools will at least have a plugin for GLTF export.

Most 3D asset creation tools have material systems more geared towards cinematic rendering, rather than realtime rendering, so it can be important to understand how materials are exported in GLTF format (via Blender), as well as how StereoKit interprets them.

Materials

StereoKit’s rules for interpreting GLTF materials are mostly straightforward!

If the GLTF material is PBR (the normal case), StereoKit will use a PBR shader. It uses the metallic roughness definition of PBR, and does not currently support the specular/glossiness GLTF extension.

If the GLTF material is Unlit (In Blender, this is a material using a Background or Emission surface) StereoKit will use an Unlit shader.

If the GLTF material has alpha mask on, StereoKit will use a “clip” variant of the PBR or Unlit shaders that discards transparent pixels.

It’s good to note that GLTF supports vertex color data, and almost all StereoKit default shaders support vertex coloring! Vertex colors don’t often show in 3D asset tools out-of-the-box, but this can be a nice way to add cheap baked lighting to Unlit materials, or add color variation to looping textures, or other creative uses!

Lightmaps

StereoKit also supports a Ligtmapped material from GLTF files! This is a somewhat non-standard setup not found in the original spec, but is a very useful one for performant applications. It is not unusual to find bad GLTFs where lighting data is baked down into the diffuse texture, expanding a “looped” texture into something much much larger. This consumes much more RAM than preserving the original looped texture and combining it with a separate lightmap texture!

Instead, if StereoKit encounters a material that has:

An emissive texture (the base color, like an Unlit material)
An occlusion texture using the second UV channel (the lightmap)
No texture for PBR base color
No texture for PBR metal/roughness

This will trigger StereoKit to switch to a high-performance Unlit + Lightmap shader.

Best Practices

Optimize with GLTFPack

StereoKit supports all the GLTF extensions required for GLTFPack to work properly! Mesh compression, KTX textures, quantization and transform all work. If you can optimize your GLTF files in advance with a tool like this, you can get 50-80% smaller files (even on GPU!), much faster load times, and improved render performance!

gltfpack.exe -cc -tc -i modelName.glb -o modelName.opt.glb

This is effective with just about any model, but especially for photogrammetry-like assets this type of optimization is absolutely critical!

Backface Culling

GLTF supports a setting for enabling or disabling backface culling, an important optimization that will skip rendering triangles that face away from the viewer. Having backface culling ON can improve performance significantly, and in most normal cases, is completely unnoticeable! For whatever reason, many 3D tools seem to disable backface culling by default, and this mistake is very common to find in many GLTF files!

Always make sure your materials have backface culling ON whenever possible!

Settings for turning on backface culling in Blender

PBR Shaders vs. Unlit

Physically Based shaders look great! But the accuracy they provide costs a lot of computation. If you can use an Unlit material instead, do it!

For more information about shaders and material, check out StereoKit’s Material guide!

References

A quick note about how Asset types (Model/Material/Tex, etc.) work in StereoKit!

When you work with an Asset, you’ll want to keep in mind that you’re actually working with a reference to the Asset! Sometimes the handle you have is just one of many references to the Asset, and so changing the Asset will affect all other references to it as well.

If you want to change an Asset’s property without affecting other references to that Asset, then you should Copy() that Asset, and modify that new, unique instance instead! This is why you’ll frequently see the example code doing something like Material.Default.Copy() before changing any of the properties.

GLTF extension support

On a more technical note, here’s a specific list of what GLTF extensions StereoKit supports.

KHR_materials_unlit
KHR_mesh_quantization
KHR_texture_basisu
KHR_texture_transform (not rotation)
EXT_meshopt_compression

Vector Art with SVG

SVG files load as flat, vertex colored geometry, which makes them a good fit for logos, icons, charts, and diagrams that might look bad up close when rasterized to pixels! Mesh.FromFile gives you the geometry on its own, and Model.FromFile wraps the same Mesh in a node with an unlit Material.

Mesh  logo  = Mesh .FromFile("logo.svg" );
Model chart = Model.FromFile("chart.svg");

logo .Draw(Material.Unlit, Matrix.T(0, 1.5f, -1));
chart.Draw(                Matrix.T(1, 1,    -1));

All colors, gradients and opacity land in the vertex colors, so pick a Material that reads them. Material.Unlit is usually right for artwork, and Material.Default will light it like any other surface!

SVG supported functionality

This is deliberately a small, sane subset of SVG. Paths and basic shapes, groups and transforms, solid fills and strokes with joins and caps, linear and radial gradients, use references, and CSS classes in a style block all work. Clipping, masks, filters, images, dashed strokes, and text do not.

Text is the one you’ll most likely run into! Most tools keep text as text elements, which SK will skip with a warning in the log. Convert the text to paths before exporting, and it’ll load fine! A quick way to convert a file from a tool that can’t is rsvg-convert -f svg in.svg > out.svg.

SVG size and orientation

If the file’s root declares a real world size in mm, cm, or in, the Mesh uses that size! Otherwise the image’s height is normalized to 1 meter high, with the width following the aspect ratio, exactly like Sprites. The Mesh is centered on the image and faces -Z, matching Text and UI.

SVG overlap and transparency

Everything in the file shares one depth, so shapes can’t sort against each other the way they do in a browser. Shapes are sorted in reverse document order, so that the last shape in the document is the first drawn shape, which survives Z-testing. This does mean that transparent shapes won’t work well, a translucent shape blends with the scene behind the whole mesh but still hides the shapes behind it in the same file. Changing depth testing will also change sorting behavior!

Notes About Alternative Formats

Why no FBX you might ask? FBX is an old and poorly defined format. Early versions of StereoKit supported it, but it ended up being far too much pain. Among other things, FBX has no reliable concept of “units” or even “which direction is forward”.

Why not USD? USD unfortunately has no format specification! It’s less of a file format, and more of a library. As such, it has a very poor tooling ecosystem making the “format” difficult to support well. If you want to support USD correctly, you must use the singular implementation of USD. And unfortunately, that implementation doesn’t pass my smell test for code quality.

Found an issue with these docs, or have some additional questions? Create an Issue on Github!