MIT License NuGet

Home / Local AI / Roadmap

Where the AI layer is going

Four items, in the order they should happen, with the reasoning for the ordering included because it is the part most likely to be wrong. Everything on this page is a plan. None of it is shipped, and nothing here should factor into a purchase decision today.

The question this track has to answer

The engine roadmap asks whether the renderer is genuinely next-generation. This track asks something less comfortable: is the paid layer worth money, and does it help make games? Eight generation modalities already ship, so the answer does not depend on adding a ninth. It depends entirely on the first item below.

1. Generated output into the engine

Absent  Highest return on effort in the entire roadmap

The most under-valued gap in the repository. Eight capabilities generate output; none of that output can become an engine asset. Meanwhile the paid tier's whole premise is that developers want generated assets in their game. The scope is small — that is what makes the gap embarrassing rather than difficult:

ItemWhat it means
Image → material textureApplied to the material of a picked object, from the panel that generated it
Image → spriteStraight into the 2D overlay layer
Music → audio sourcePlayable from the scene rather than from a panel
A generated-assets folderStable on-disk layout so a project can reference outputs directly
The reverse directionPick an object, send its current texture into image-to-image as the reference
The last row is the whole argument

Sending a selected object's texture back into generation is the difference between "an AI panel" and "AI in the engine". It is also the only answer to why buy this instead of using ComfyUI, which is free that survives contact with a sceptic: ComfyUI cannot select an object in your scene. Until this exists, that answer is not available, and the site should not pretend otherwise.

2. Cloud models and runtime C# compilation

Absent  Desktop only, by nature

Two capabilities that are more interesting together than apart.

Cloud model access. An HTTP client — the repository has none at all — plus streaming response parsing, storage for a user-supplied API key, and cancellation. A few hundred lines rather than a pipeline. Commercially it has one attractive property: the user brings their own key, so inference cost never lands on the vendor, and there is no subsidised-token trap to grow out of.

Runtime C# compilation. This is where a pure-C# architecture pays a dividend that is not available to the engines it gets compared against. Roslyn compiles C# at runtime, and the object model is already shaped to receive the result: a behaviour is a Panel or Control subclass, so a generated class only has to be instantiated and added. No serialisation format to satisfy, no component model to register with, no editor metadata to emit.

// what a language model would have to produce
class Windmill : Panel
{
    Model blades;

    public override void Create()
    {
        blades = new Model { Name = "Assets/windmill.glb" };
        AddControl(blades);
        base.Create();
    }

    public override bool Update(float time, float? alpha = null, ...)
    {
        blades.Rotation += time * 0.4f;
        return base.Update(time, alpha: alpha, ...);
    }
}

That is the entire integration surface. An engine whose behaviour is assembled from inspector-configured components has to emit scene files, component references and metadata to achieve the same thing; here the generated artefact is just source.

Three constraints, stated before anyone builds on it

Roslyn runtime compilation is unavailable under full AOT, which rules out iOS and WebAssembly — this feature is desktop-only by nature. Dynamic compilation inside an MSIX package needs verifying early rather than at submission time. And generated code executes with full application trust, so the consent and security story has to be designed in rather than retrofitted.

3. 3D asset generation

Absent  Static props only — and further away than it looks

This is the most requested item and the most misjudged. The generation model is the small part. Four things have to exist, and only the first is about AI:

#WorkCurrent state
1A local image-to-3D or text-to-3D modelReal, and the easiest step
2Mesh post-processing — decimation, UV unwrapping, normal repairNo mesh processing code exists
3glTF exportImport only
4Something actually usable in a sceneOnly reachable after 2 and 3

Generated meshes arrive with poor topology, unconstrained triangle counts and no usable UV layout. That is a mesh pipeline, not a feature. It does converge usefully with mesh LOD on the engine roadmap — both need a decimator, so it gets built once — but the honest scope is static prop generation, clearly labelled, with no rigging attached to the same deliverable.

4. Automatic rigging

Research, not a commitment

Skeleton inference and skin-weight prediction are substantially less mature than mesh generation, and they are sensitive to exactly the topology quality that generated meshes lack. Auto-rigged output usually needs manual correction — which is the worst possible property for something sold as a feature, because the buyer discovers the manual work after paying. It stays in research and off any store listing until output quality has been verified against real characters.

Why 2 comes before 3

The intuitive ordering puts 3D generation near-term and cloud reasoning far off. That ordering is inverted here deliberately, and since it is the one substantive disagreement in the planning documents, the reasoning belongs in public:

  • Effort asymmetry. Item 2 is an HTTP client plus a Roslyn host. Item 3 is a mesh processing pipeline plus a glTF exporter plus a model — several times the work.
  • Output-quality risk. Item 2's quality is supplied by frontier models and improves for free. Item 3's quality is capped by what local 3D models produce today, which is not yet drop-it-in-a-game quality. Selling that as a headline feature invites refunds.
  • Architectural leverage. Item 2 exploits a property this engine has and its competitors do not — behaviour as a single compilable C# class. Item 3 exploits nothing specific to this project; any engine can import a mesh.
  • Convergence. Runtime compilation and the planned world format are the same insight approached from two directions. Built near each other they produce one coherent design; built years apart they produce two that have to be reconciled.

The counter-argument is real and worth acknowledging: 3D assets are the harder bottleneck for a solo developer, and therefore the thing people would more readily pay for. That is why item 3 stays on the roadmap rather than being cut. It is an argument about eventual value, not about sequence.

What is deliberately not planned

Subscriptions. The value is not recurring and the inference cost is the user's own electricity. There is nothing to meter.

A paid base application. The free build with optional add-ons is doing real work: someone arriving from the repository can install and judge it at zero friction.

Model hosting. Weights stay with their publishers. This project is not going to become a redistributor of other people's licences.

Cloud inference on the vendor's account. If cloud models arrive, the key is yours. Anything else eventually forces a price rise or a shutdown.

Read the rest of it

This page is one of five tracks. The engine roadmap covers rendering, the application host and world format, foundations — tests, benchmarks, image-diff regression, all of which are currently absent — and a later VR track. It is also markedly more candid than a roadmap normally is, including about this one.