The Gateway in the Middle: Why Your AI Proxy Is a Perimeter Device and What a Pre-Auth SQLi Taught Us

Three different AI developer tools made CISA's Known Exploited Vulnerabilities catalog in the first half of 2026. Langflow, an AI workflow builder, was added on March 25. Marimo, a reactive Python notebook, on April 23. LiteLLM, an AI gateway, on May 8. The first two are tools you point at. The third is the box everything else points at, and it is the one I run.

LiteLLM is the first AI gateway on KEV, and the flaw that put it there makes the rest of the quarter's entries look routine. An unauthenticated SQL injection in the proxy's API-key verification path, present from 1.81.16 through 1.83.6, fixed in 1.83.7. NVD scores it Critical, 9.8 on the v3.1 primary and 9.3 on the vendor's v4.0 vector. The first in-the-wild injection landed 36 hours and 7 minutes after the advisory reached the global advisory database.

This is not a fresh-vulnerability alert. The CVE is from April, the patch shipped in April, and the CISA deadline was May 11. What I want to write down is the structural lesson, which is that the AI gateway is the perimeter device of the self-hosted AI stack. It holds the keys to every upstream provider, it sits in front of every agent profile, and a lot of people leave it on the public internet with no network-layer authentication. The LiteLLM incident is the proof that this class of box gets found and read in a day.

The box that holds every key

An AI gateway is a translation layer. You run multiple agent profiles, each with its own virtual key, and the proxy fans out to OpenAI, Anthropic, Azure, Bedrock, or whatever local endpoints you have. The clients see one API. The proxy holds the fan-out: every upstream provider credential, every virtual key, the budgets, the user records, the master key, all of it in one PostgreSQL database behind one port.

That is what makes the gateway a perimeter device in the same sense a router is. It is the single point between untrusted callers and everything behind it. A compromise does not leak one API key. It leaks the key ring. Sysdig put it more bluntly than I can: "Anything reachable on port 4000 is exploitable."

I run the same class of box. A Docker container on the primary host, a Postgres sidecar for its state, port 4000 published to the host, and a handful of agent profiles behind it. When the CVE dropped I did the version check. The result is half the story.

My proxy reports 1.87.1, past the 1.83.7 fix floor, so the running build is not vulnerable. But the image is main-stable, a rolling tag, not a pinned release. Past the floor and floating on a moving tag are two different postures. The version number says I am safe today. The tag says I do not know what I will be running on Monday. So the fix on my side was a pin to a specific stable release, the vendor-recommended v1.83.10-stable or better, not an upgrade: the next patch is a decision I make, not a pull I inherit.

One part I have not closed, and I would rather leave it open than claim a posture I have not measured. The container binds to 0.0.0.0 on the host, so reachability is decided by whatever firewall sits in front, not by the proxy. Whether port 4000 is reachable from the internet on my network is still a question I am answering for myself. The point for your box is the same: the answer should be a firewall rule you can point at, not an assumption.

What the bug actually was

The flaw is in how the proxy checks the token. When a request arrives at any LLM API route, say POST /chat/completions, the proxy verifies its Authorization header. In the affected versions, the verification query was built by concatenating the caller-supplied token into the SQL text of a SELECT against the LiteLLM_VerificationToken table. No parameter binding. The caller's value went into the query, not around it.

The path is the detail that matters. The vulnerable query sits on the error-handling path of the verification, so a request with a malformed or unexpected token can reach it before authentication is decided. Pre-auth, on any LLM route, with no session and no credentials at all. An attacker who can reach port 4000 can send one HTTP request with a crafted Bearer value and run a SELECT of their choosing against the proxy's own database.

Reads are guaranteed; modification is possible, per NVD's "may be able to modify it". The database holds the virtual API keys including the master key, the upstream provider credentials, the budgets, the user records, and the proxy's own environment variables, which on most installs include the Postgres DSN. One successful injection is not a data leak of one field. It is the credential layer of the whole stack, in one query.

The fix is the boring, correct fix. In 1.83.7 the caller-supplied value is passed to the database as a separate parameter, and the query no longer contains input. LiteLLM shipped v1.83.7-stable on April 19, a day before the GitHub advisory, and published a security update on April 29 from CEO Krrish Dholakia and CTO Ishaan Jaffer. The bug came in through their bug bounty program, reported by Tencent YunDing Security Lab. For the unpatched window, the vendor documented a workaround: set disable_error_logs: true under general_settings, which takes the error path that leads to the vulnerable query out of the loop.

Thirty-six hours and seven minutes

The speed is the story, but the details say it was not a scanner.

Sysdig's threat research team anchored the clock to the advisory's appearance in the global GitHub Advisory Database on April 24 at 16:17 UTC. The first injection attempt was observed on April 26 at 04:24 UTC, 36 hours and 7 minutes later, from 65.111.27[.]132 behind a hosting-provider egress range. Sysdig was careful to note the source may be a rented VPS rather than the operator. The identity is secondary. The precision is the point.

The first twenty minutes were schema enumeration: seventeen UNION payloads, three target tables, and only the tables that hold secrets. The virtual-key table, the upstream credential store, and the config table where the proxy keeps its environment variables. No probes against the user or team tables, which would be the generic-scan move. Then the detail that marked it as targeted. PostgreSQL folds unquoted identifiers to lowercase, and LiteLLM's Prisma ORM generates table names in PascalCase. The first payloads used the lowercase form and came back empty. The retry, minutes later, quoted the PascalCase name and worked. That retry is not scanner behavior. It implies someone had read the Prisma schema first, or had a model do it for them.

A second egress IP in the same range, 65.111.25[.]67, came twenty minutes later replaying the payload set and probing the key-management endpoints, /key/generate and /key/info, without authentication. Every request carried a Python/3.12 aiohttp user-agent. The Bearer values were not keys. They were the injection itself: a literal sk-litellm' prefix with a trailing comment, a column-count sweep that climbs the widths until the query returns, and a terminal OR 1=1.

The coverage carried a timing discrepancy, and I will not repeat it, because the number that matters is the one that is internally consistent. What matters more is what Sysdig did not find: no confirmed compromise in the traffic. No authenticated calls with exfiltrated keys, no virtual keys minted through the endpoint that was probed, no chained reuse of provider credentials. CISA's KEV listing still calls it known exploited, because the injection was executed in the wild against live targets. The honest sentence is targeted exploitation, observed and attempted, with the schema-awareness of an operator who had done the homework.

Then the catalog did what it does. CISA added CVE-2026-42208 to KEV on May 8 with a May 11 deadline for federal agencies. The entry is still in the catalog three months later, and it is now one of two KEV-listed LiteLLM flaws. The follow-on CVE-2026-42271, a command injection in the MCP test endpoints of the same version range, was added on June 8 with a June 22 deadline. The vendor patched both in the 1.83.7 line. The operational read: this product draws active exploitation, so pinning and monitoring it is a standing job, not a one-off.

The gateway is the new edge device

Zoom out and the pattern is not about LiteLLM.

The self-hosted AI stack has a perimeter now, and it is not the firewall. It is the box that turns every internal caller into every external provider. It consolidates credentials the way a router consolidates traffic: one interface, everything behind it. The older exposure problem was inference servers on the open internet, which I wrote about when Cyera counted roughly 300,000 internet-exposed Ollama deployments. That exposure leaks model weights and compute. Gateway exposure leaks the keys every other tool on the network spends.

The placement habit is the same as every other homelab default. The proxy needs to be reachable from the agents that use it, so it goes on a published port. The agents move: laptops, other machines, a phone over the VPN. The port that was published for the laptop stays published. And because the proxy does its own API-key authentication, there is a standing argument that network-layer protection is optional. The LiteLLM incident is the rebuttal. When the authentication layer itself is the injection point, the network layer is the only layer left.

Sysdig also flagged that the unparameterized-query pattern is not unique to this product, noting a related configuration-injection class in recent ingress-nginx advisories (CVE-2026-3288 and CVE-2026-24512). The shape recurs: a component that builds queries from caller input, sits at an edge, and gets published before anyone thinks about who the caller is.

Hardening the gateway

Here is the stack, in the order I would apply it, assuming your gateway is the same class of box as mine.

1. Network placement, first, because everything else assumes it. The proxy sits on the trusted network and is not internet-facing. Agents reach it over the LAN or the VPN, and the firewall rule that allows port 4000 lists the clients that need it. If you cannot point at the rule, the port is open.
2. Pin the version. Past the 1.83.7 floor is the minimum, and the vendor's own recommendation for this CVE was v1.83.10-stable. A rolling tag is a bet that the next build is better than this one. Pin a specific stable release, track the upstream advisory feed, and patch on the release, not on the news. With two KEV entries in two months, treat LiteLLM as a product that needs a watch, the way you watch your router firmware.
3. Rotate the keys on a cadence, and rotate the ones the gateway holds, not just the ones your apps hold. The upstream provider credentials behind the proxy are the blast radius. A rotation schedule that covers provider credentials, virtual keys, and the master key, with a documented cutoff for the old values, is the control that makes a compromise survivable instead of permanent.
4. Least-privilege database user. The proxy's database account should not need write access to the credential tables any more than it does, and it should not be the database superuser. The injection read what the account could read. The smaller the account, the smaller the read.
5. Egress restrictions on the proxy. The proxy's legitimate egress is the provider APIs it fans out to, a short list of hostnames. Anything else it phones out to is either a fallback you forgot you configured or a channel you do not have. A fixed egress list also makes the detection below a lot easier, because the baseline is small.
6. Rate limiting on the auth path. The exploit was a sequence of crafted tokens against the verification path. A limiter on the key-check endpoints, stricter than the one on the inference endpoints, turns the enumeration from a twenty-minute session into a 429. It is not a substitute for the patch. It is the thing that makes the pre-patch window survivable and slows the post-patch scanner.

If the port was open

If your gateway was reachable from an untrusted network during the vulnerable window, the question is not whether you were scanned. You were. The question is what the scanner did, and the answer lives in the database's own history.

Postgres query logs are the primary artifact. Search the window for the shapes above: UNION SELECT in the query text, a lowercase table name followed by a quoted PascalCase retry of the same table, a column-count sweep, an OR 1=1. The vendor linked a helper query for this from the security update. The LiteLLM error logs, if you did not have the workaround on, carry the Authorization values, and the values to look for are not keys. They are Bearer tokens that contain quotes, comment markers, and SQL.

Check the data, not just the logs. Unexpected virtual keys in the verification table, especially ones with names or usage patterns you do not recognize. New user records created in the window. The /key/generate endpoint was probed unauthenticated in the observed campaign, so any key that exists that you did not mint is the finding. And check the egress: connections from the proxy to hosts that are not your provider list, your DNS, or your patch feed, inside the window.

Two correlated signals are the compromise. A UNION payload in the logs with no other finding is the attempt, and it is the same attempt CISA cataloged. A UNION payload plus a key you did not create is the thing that changes the response from a patch to a rotation of every credential the proxy touches.

The standing posture

The gateway is the box that makes the rest of the stack work, and a compromise of it is a compromise of everything it fronts. The April incident was not unusual for the class. It was the class, behaving: a credential-holding edge box, published, patched the day before the repo advisory, exploited within 36 hours of the global index.

The posture is the same one I keep re-applying to every box in the lab. Pin the version to a release you chose. Put the port behind a rule that names its clients. Rotate the keys it holds on a schedule. Give its database account the minimum. Make its egress a short list you can recognize. And keep the query logs off the box, because in the incident the database you are auditing is the one the attacker has already read.

A version floor says you are safe from what has been CVE'd. A perimeter says you are safe from what has been published. The gateway needs both.

Sources

• NVD, CVE-2026-42208: https://nvd.nist.gov/vuln/detail/CVE-2026-42208
• CISA Known Exploited Vulnerabilities catalog: https://www.cisa.gov/known-exploited-vulnerabilities-catalog
• GHSA-r75f-5x8p-qvmc: https://github.com/BerriAI/litellm/security/advisories/GHSA-r75f-5x8p-qvmc
• LiteLLM security update (Dholakia, Jaffer, April 29 2026): https://docs.litellm.ai/blog/cve-2026-42208-litellm-proxy-sql-injection
• LiteLLM helper query for Postgres history review: https://gist.github.com/ishaan-berri/6f31e56e878338eb4c01990bd08378ab
• Sysdig threat research, targeted SQL injection against LiteLLM's authentication path (Michael Clark, April 27 2026): https://www.sysdig.com/blog/cve-2026-42208-targeted-sql-injection-against-litellms-authentication-path-discovered-36-hours-following-vulnerability-disclosure
• The Hacker News (April 29 2026, updated May 8): https://thehackernews.com/2026/04/litellm-cve-2026-42208-sql-injection.html