Incubator

Projects, not products.

Everything starts as a draft. Graduation requires two independent implementations and a conformance suite. InferenceOSS does not ship an engine.

• OIP-Gdraft

  Open Inference Protocol — Generative

  The conversational, tool-using, structured, multimodal protocol engines should actually implement — with capability discovery, not folklore.

  WG · Specs

• CapSpecdraft

  Engine capability manifest

  OpenAPI for runtimes. A JSON document that says, precisely, what this process can do on this hardware today.

  WG · Specs

• KVXdraft

  KV cache interchange

  A portable prefix-cache format so a hit on one engine is a hit on another, and so disaggregated prefill/decode is not a vendor feature.

  WG · Interchange

• InferTracedraft

  Inference telemetry conventions

  OpenTelemetry semantic conventions for the numbers that actually matter: TTFT, ITL, cache, batch, speculation, and abort.

  WG · Observe

• SPEC/Idraft

  Standard Performance Evaluation — Inference

  A vendor-neutral, continuously run inference benchmark. Signed configs. Rented hardware. No vendor-submitted slides.

  WG · Observe

• Gatedraft

  Inference-time policy sidecar

  A sidecar spec for schema enforcement, PII, allow/deny, and output constraints that is not baked into one engine.

  WG · Safety

• EdgeABIdraft

  Portable edge runtime ABI

  A small contract so llama.cpp, MLX, NPU runtimes, and WASM builds look like the same citizen as a datacenter engine.

  WG · Runtime

• Routerdraft

  Reference inference router

  A small, boring scheduler above engines: CapSpec-aware routing, disaggregated prefill/decode, and retry. Reference, not product.

  WG · Runtime