Configuration guide
Cloudflare AI Gateway · Prepared for NetSPI

Control AI spend
at the group level.

Use Cloudflare AI Gateway Dynamic Routing to translate trusted identity context into shared budgets, approved model choices, and predictable fallback behavior.

Model a group policy View implementation
01Identity drives policy

Normalize IdP membership into one trusted budget group.

02Routes hide complexity

Expose task names, not a catalog of provider models.

03Fallbacks are intentional

Degrade to a lower-cost model or stop cleanly.

Recommended architecture

Put identity before inference.

Dynamic Routing evaluates request metadata; it does not query Entra ID, Okta, or another identity provider. A trusted application tier must resolve group membership and attach the canonical policy attributes.

1Identity provider

User + group claims

groups: ["ai-standard"]
2Trusted app / Worker

Normalize policy

budget_group: "standard"
3AI Gateway

Evaluate route

dynamic/netspi-assistant
4Approved provider

Run selected model

fast / balanced / advanced
Interactive reference design

One entry point. Three policy tiers.

Select an illustrative group to see how a single route can branch into a group budget and a deliberately small model set. Group names and dollar values should be replaced during discovery.

Selected policy

General workforce

Active
Shared daily budget
$25
Budget pool value
group.general
Primary route
Fast
At limit
Stop + explain
START
GROUP
GENERAL
BUDGET
$25 / DAY
MODEL
FAST

Low-cost default for summarization, drafting, and knowledge retrieval. Advanced models are not presented.

Shared group pool

Set the rate node key to metadata.budget_group. Every request carrying the same normalized value consumes the same quota.

Per-user guardrail

Add a second rate node keyed on metadata.user_id if one user should not consume the group allocation alone.

Hard or soft threshold

Terminate for a hard stop. A cheaper-model fallback preserves service, but continues spending and is therefore only a soft threshold.

Implementation pattern

Normalize first, route second.

Keep directory-specific identifiers out of route definitions. The application maps raw claims to stable policy tiers, then AI Gateway applies conditional and budget nodes.

01

Define precedence

If a user belongs to multiple groups, resolve exactly one effective tier. Prefer explicit priority such as research > delivery > general.

02

Attach trusted metadata

Send stable identifiers in cf-aig-metadata. Avoid sensitive directory attributes and stay within AI Gateway's five-entry metadata limit.

03

Branch to fixed limits

Each group branch leads to a cost-based rate node with its approved limit, window, primary model, and over-budget behavior.

04

Version and observe

Deploy route changes as versions, monitor spend and fallback rates, and retain an immediately reversible prior version.

Request metadata
const policy = resolvePolicy(validatedClaims.groups);

const metadata = {
  user_id: hash(validatedClaims.subject),
  budget_group: policy.budgetGroup,
  model_tier: policy.modelTier,
  workload: "workforce-assistant"
};

await fetch(AI_GATEWAY_URL, {
  method: "POST",
  headers: {
    "cf-aig-authorization": `Bearer ${GATEWAY_TOKEN}`,
    "cf-aig-metadata": JSON.stringify(metadata)
  },
  body: JSON.stringify({
    model: "dynamic/netspi-assistant",
    messages
  })
});
Group budget node · illustrative values
{
  "id": "budget-general",
  "type": "rate",
  "properties": {
    "limitType": "cost",
    "key": "metadata.budget_group",
    "limit": 25,
    "window": 86400
  },
  "outputs": {
    "success":  { "elementId": "model-fast" },
    "fallback": { "elementId": "END" }
  }
}

Important: the values above illustrate structure, not a final NetSPI budget recommendation. This hard-stop pattern returns an error at the limit; the application should translate it into a clear message. Establish limits from measured token volume, model pricing, concurrency, and business criticality.

Model governance

Show capabilities, not catalogs.

A long provider model list creates decision fatigue and makes cost unpredictable. Give non-technical users a few stable, task-oriented choices while routes own the underlying model selection.

BeforeRaw model catalog
  • Provider A / model-x-2026-08
  • Provider A / model-x-mini
  • Provider B / model-pro-latest
  • Provider C / model-70b-fp8
  • Provider D / reasoning-preview
  • + 37 more models
AfterApproved capabilities

Quick Assist

Draft, summarize, retrieve

Default

Deep Analysis

Complex security reasoning

Eligible groups

Code Review

Technical analysis

Technical roles
UX control

Curate what users see

The application presents approved route aliases such as Quick Assist. It never fetches or displays the provider catalog.

Security control

Prevent bypass

Keep provider and gateway credentials server-side. Accept only allowlisted route aliases at the proxy and reject arbitrary model strings.

Operational control

Change models centrally

Point each route to a reviewed model set. Swap or roll back the implementation without retraining users or changing application code.

Operating model

Launch with evidence, not guesses.

Budgets should evolve from observed demand. Begin with a narrow pilot, instrument decisions, and promote only after the fallback and cost behavior are understood.

Week 0

Baseline

Inventory use cases, groups, provider keys, approved models, and expected daily volume.

Weeks 1–2

Pilot

Run representative users with conservative limits. Capture cost per group, user, route, and workload.

Weeks 3–4

Tune

Adjust budgets, model tiers, timeouts, retries, and fallback messaging from measured behavior.

Ongoing

Govern

Review monthly, test rollback, remove unused models, and investigate unexpected group or direct-model traffic.

Production readiness checklist

Source material

Cloudflare documentation

Dynamic Routing overview↗ JSON configuration↗ Custom metadata↗ Gateway authentication↗