Catalog awareness on AWS Bedrock: pick-your-model, IRSA, and a chat UI

Part 3 built the load-bearing core of catalog-assistant: grounded natural-language Q&A over the Backstage Software Catalog, with retrieval over real entities and entity-ref citations. It was deliberately minimal — one model, no conversation memory, no UI. Just enough to prove the grounding works.

This post is about the part that decides whether anyone actually runs it inside a company: which LLM, authenticated how, at what cost. Most enterprises that would deploy an internal developer portal are on AWS and want their model calls to stay inside their account. So the answer is Amazon Bedrock — but Bedrock the way an operator actually wants it: the model is their choice, auth is an IAM role (no long-lived keys), and the per-question cost is low enough that “call an LLM for every catalog question” isn’t a budget conversation.

The plugin is public at Naga15/platformlog-plugins (@theplatformlog/catalog-assistant-backend). Here’s what changed.

One provider switch, any model

The Q&A pipeline never talks to a model SDK directly — it goes through the Vercel AI SDK’s generateText, and a tiny resolver picks the provider. Adding Bedrock was a single entry in a map plus a branch for how it authenticates:

export const SUPPORTED_PROVIDERS = {
  anthropic: { pkg: '@ai-sdk/anthropic', factory: 'createAnthropic' },
  openai:    { pkg: '@ai-sdk/openai',    factory: 'createOpenAI' },
  google:    { pkg: '@ai-sdk/google',    factory: 'createGoogleGenerativeAI' },
  mistral:   { pkg: '@ai-sdk/mistral',   factory: 'createMistral' },
  bedrock:   { pkg: '@ai-sdk/amazon-bedrock', factory: 'createAmazonBedrock' },
};

The resolver dynamically imports only the provider that’s configured, so a backend that runs Bedrock never pulls in the others. Switching providers is a config change, not a code change — which matters, because the same plugin needs to run on a developer’s laptop (free, local) and in a company’s EKS cluster (Bedrock) without a rebuild.

Auth on EKS: an IAM role, not a key

The thing I care most about for production: no static credentials in config. On EKS that means IRSA (IAM Roles for Service Accounts) or the newer Pod Identity — the pod assumes a role, and the AWS SDK resolves it from the environment. The Bedrock provider doesn’t bundle the AWS credential resolver (deliberately — it keeps the package lean), so the plugin supplies it from the optional @aws-sdk/credential-providers peer dependency when no static keys are present:

// no static keys → resolve the role from the environment (IRSA / Pod Identity)
const { fromNodeProviderChain } = await import('@aws-sdk/credential-providers');
factoryOpts.credentialProvider = fromNodeProviderChain();

The config in the cluster carries no secret at all:

catalogAssistant:
  provider: bedrock
  awsRegion: us-east-1
  model: us.amazon.nova-lite-v1:0

Attach an IAM role allowing bedrock:InvokeModel (and ...WithResponseStream) on the model and inference-profile ARNs to the backend’s ServiceAccount, and you’re done — the key never exists. For quick validation without IAM, a short-lived Bedrock API key (bearer token) also works via apiKey / AWS_BEARER_TOKEN_BEDROCK.

Two gotchas worth your time

• Model IDs are inference profiles, not the bare name. For current Claude/Nova models Bedrock routes through a cross-region inference profile, so the value is us.amazon.nova-lite-v1:0, not amazon.nova-lite-v1:0. Copy the exact id from the console after enabling Model access.
• Pin the AI-SDK generation. The @ai-sdk/* provider packages move in lockstep with the ai core — mixing a provider that implements a newer model spec with an older core throws Unsupported model version. Keep @ai-sdk/amazon-bedrock on the same major line as your other @ai-sdk/* providers and ai.

There was also a bug I only found by running it: the router was reading a token off BackstageCredentials that doesn’t exist, so the catalog read 401’d. The fix is to read the catalog as the plugin’s own service identity — getPluginRequestToken({ onBehalfOf: getOwnServiceCredentials(), targetPluginId: 'catalog' }) — which works for any caller. Unit tests are green and the thing looks done right up until a real backend hands you a real credential.

Let the operator pick the model — and cap the bill

Different questions deserve different models, and operators want to disable the ones they don’t trust after a week of testing. So the model is an allowlist in config: a default plus the set offered to callers.

catalogAssistant:
  provider: bedrock
  awsRegion: us-east-1
  model: us.amazon.nova-lite-v1:0       # default
  models:
    - { id: us.amazon.nova-lite-v1:0, label: Nova Lite (cheap, default) }
    - { id: us.anthropic.claude-haiku-4-5-v1:0, label: Claude Haiku 4.5 (quality) }

That backs a GET /v1/models endpoint (which a UI dropdown reads) and an optional model field on each query, validated against the enabled set — a model that isn’t listed gets a 400, so nobody can quietly route traffic to an expensive model.

What “cheap” means here

The job is grounded Q&A over a small provided context — read the entities, answer, cite. That rewards instruction-following, not raw world knowledge, so a small model is usually plenty. Bedrock has no free tier, but for this workload the cost is rounding error:

A typical query is ~2K grounded input + ~300 output tokens. That’s roughly 5,000 questions per dollar on Nova Lite, ~285 on Haiku. My default is Nova Lite, with Haiku in the dropdown for the hard, compound questions where staying strictly grounded matters. (Prompt caching knocks ~90% off the repeated system prompt; skip Batch — it’s async, wrong fit for interactive Q&A.)

Make it a conversation

Part 3’s biggest limitation was that every question stood alone — ask “who owns it?” and the model had no idea what “it” was. The fix keeps the backend stateless: the client carries the conversation and sends prior turns as history on each request. The backend threads them before the grounded current turn, and — the part that actually makes follow-ups work — folds the recent user turns into the retrieval query, so “who owns it?” still retrieves the entity named a turn ago.

On the frontend that’s a floating chat widget (a separate package — Backstage keeps frontend and backend as separate plugins) pinned bottom-right on every page: a prompt box, the model dropdown wired to /v1/models, citation chips under each answer, and a Clear button. The conversation persists for the browser session and resets when you clear it.

The same widget in context — the bubble sits bottom-right on every page:

Proving it without an AWS bill

The nicest property of the provider abstraction is that you can validate the entire flow for free before pointing it at Bedrock. The openai provider takes a baseURL, so a local Ollama endpoint stands in for the cloud at $0:

catalogAssistant:
  provider: openai
  baseURL: http://localhost:11434/v1
  apiKey: ollama
  model: gemma3:4b

I ran the whole thing this way in a legacy Backstage app against gemma3:4b, and confirmed end to end:

• “Who owns example-website?” → “guests”, cited component:default/example-website.
• “who owns it?” with history → resolved “it” to the prior entity.
• Switching the dropdown to a second model routed there; a disallowed model returned 400.

Flipping that to production is the config block from the top of this post — provider: bedrock, a region, an IRSA role, and the model list. Same plugin, no code change. That’s the whole point: catalog awareness shouldn’t care whether the model lives on your laptop or in your AWS account.

Pricing and model-quality figures are list prices and public benchmarks as of writing — verify current Bedrock pricing in the AWS console. Comparative figures drawn from AWS Bedrock pricing, Artificial Analysis, the Vellum LLM leaderboard, and Amazon’s Nova technical report.