This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Llama Responses bridge

Loopback Responses API function namespace compatibility for llama.cpp

This project adapts Responses API function namespaces for llama.cpp servers that accept only flat function tools. It flattens function schemas, replayed calls, and forced tool choices, then restores namespace/name identities in JSON and streaming replies. The adapter performs no inference or tool execution.

bazel_agent bazel run //projects/llama_responses_bridge -- --listen 127.0.0.1:8080 --upstream http://127.0.0.1:8081

Point the client’s Responses provider at http://127.0.0.1:8080/v1 and run llama.cpp on 127.0.0.1:8081. Both endpoints are configurable, but the command requires literal IPv4 loopback addresses. It proxies only POST /v1/responses, GET /v1/models, and GET /health, without query strings. The bridge has no authentication and is intended for a trusted local host.

Request bodies, JSON replies, and individual SSE events are bounded at 32 MiB. Only function tools and function-only namespaces are supported; other tool types are rejected explicitly. Mappings are request-local, and disconnects cancel the upstream stream. Call arguments and user text are not rewritten.

  • cmd/responses_bridge owns flags, loopback validation, and the HTTP process.
  • internal/responses owns protocol translation and fixed-upstream proxying.
  • test/protocol owns command-level fixtures with a deterministic local backend and an optional real Codex MCP call/result round trip.
  • openspec owns maintained extraction work. Service units and host deployment remain with their infrastructure consumer.

Run the existing acceptance suite:

bazel_agent bazel test //projects/llama_responses_bridge/test/protocol:protocol_test --test_output=all

See protocol acceptance for the optional Codex check. The adapter was verified with Codex 0.166.0 and llama.cpp’s b11960/b12032 tool schema limitation. These fixtures establish protocol behavior, not GPU loading, model quality, or deployment readiness. Remove the compatibility layer when a pinned upstream server passes the namespace round trip directly.

1 - Local Responses protocol acceptance

Run the command-level fixture without a GPU, model, credentials, or deployment:

bazel_agent bazel test //projects/llama_responses_bridge/test/protocol:protocol_test --test_output=all

The fixture starts the built bridge on an ephemeral loopback port and uses a function-only inference backend. It checks duplicate local names in separate namespaces, plain functions, JSON and SSE replies, replayed calls, forced tool selection, and rejection of unsupported schemas. Bazel retains test.log as the repeatable acceptance artifact.

The same target accepts --test_arg=-codex-binary=/absolute/path/to/codex. Use the pinned Codex 0.166.0 executable in an isolated environment with no system Codex requirements. The optional case starts a local MCP fixture, asks Codex to use its tool, checks the model request contains the flattened schema, returns a deterministic tool call, and verifies that Codex executes the tool and submits its result before completing the follow-up response. This tests the protocol and execution path, not model quality.

On a host with shared managed Codex requirements, build first, then run the compiled test in an isolated network/mount namespace. For example, using the host’s existing bwrap installation:

mkdir -p out/llama_responses_bridge/acceptance/tmp
bwrap --unshare-net --die-with-parent --ro-bind / / --dev-bind /dev /dev \
  --proc /proc --tmpfs /etc/codex \
  --bind "$PWD/out/llama_responses_bridge/acceptance" "$PWD/out/llama_responses_bridge/acceptance" \
  --setenv TMPDIR "$PWD/out/llama_responses_bridge/acceptance/tmp" \
  --setenv RUNFILES_MANIFEST_FILE "$PWD/bazel-bin/projects/llama_responses_bridge/test/protocol/protocol_test_/protocol_test.runfiles_manifest" \
  "$PWD/bazel-bin/projects/llama_responses_bridge/test/protocol/protocol_test_/protocol_test" \
  -bridge=_main/projects/llama_responses_bridge/cmd/responses_bridge/responses_bridge_/responses_bridge \
  -codex-binary=/absolute/path/to/codex -test.v -test.timeout=60s

No namespace tool conversion is currently available in the checked llama.cpp b11960 or b12032 releases. The bridge can be removed when a pinned upstream release passes these acceptance cases directly.