Your suppliers hold copies of your keys. Six admitted leaks in one week.
If you run a small software business, your production keys are not only on your servers. They sit in your hosting platform's settings page, in a managed database's console, in the AI gateway that routes your model calls, and on every laptop and build runner that has installed your tooling. Each of those is a supplier holding a copy of something that can spend your money or read your customers' data.
Between 14 and 21 October, six of them told customers their secrets had leaked.
What happened
Railway, the app hosting platform, said on 21 October that an unauthorised third party may have accessed encrypted environment variables and, for some of them, obtained the decrypted contents. Environment variables are the settings where most apps keep their API keys and database passwords. Railway told every customer to roll all of them, plus any private registry credentials, and paused all deployments while it investigated. It believes the attack is part of a series aimed at platform companies to get at AI provider keys.
Zeabur, a smaller deployment platform, was breached on 14 and 15 October using its own leaked internal credentials. The attacker read around 3.2 million environment variable records belonging to 321,810 accounts. Zeabur's admin tokens had no expiry, had not been changed since January 2024, and were missed when it rotated credentials after an earlier breach in August. The company is shutting down by the end of the year.
TiDB Cloud, PingCAP's managed database, disclosed on 19 October that someone used stolen service account credentials to copy internal data between 9 September and 7 October. PingCAP says an exported table held encrypted customer API-key values. The attacker claims over 42,000 active keys were readable. PingCAP has not verified that.
OpenCode Console, a hosted service that stores a team's OpenAI, Anthropic, or Google key so members can use the models without seeing it, said on 17 October that keys saved by 1.4% of its users may have been exposed through a legacy system. Some users reported bills for usage they did not run, one putting the loss at $1,700. Deleting a key from Console does not cancel it. Only the provider can.
Those four held secrets on your behalf. The other two took them from you directly.
LiteLLM, a widely used Python library for calling AI models, published a poisoned version 1.107.0 on 21 October, through both PyPI and Docker. It runs every time Python starts on the machine, even if nothing imports LiteLLM, and collects cloud credentials, SSH keys, AI provider keys, and browser passwords. LiteLLM says the attacker used stolen GitHub tokens from three of its engineers. Version 1.106.2 is the last clean release.
The OVHcloud CLI, the command-line tool for OVH's European cloud, had the downloads for five existing releases swapped on 19 October for a build that reads secrets from disk. The bad files were live for 94 minutes, from 12:47 to 14:21 UTC. The attacker replaced the published checksums too, so the usual hash check would have passed. The Hacker News found that anyone who installed through Homebrew was safe, because Homebrew had pinned the original file's fingerprint and would have refused the new one.
Since then, a seventh. On 30 October Northflank, another app hosting platform, told every customer to assume their environment variables had been decrypted and to rotate every secret immediately, after a threat alert from the security firm Wiz. Within hours, customers reported that AI keys they had stored there were being used by someone else. Northflank linked those reports to the wave of attacks on AI developer platforms that Wiz calls Black October.
What it means
None of these businesses was breached because of anything its customers did. That is the point. If your keys are sitting with a supplier, your exposure is their worst week, and you find out when they email you.
You cannot vet your way out of that. What you can control is how much a single leak costs you, and that comes down to three things: how many places a key lives, how quickly you can replace it, and how much a stolen one can do before you notice.
Zeabur is the uncomfortable example of the second. Its admin tokens survived one incident response unchanged and opened the door to the next. The same thing happens in small product teams: the database password nobody rotates because nobody remembers every service that uses it.
What to do this quarter
Write down where every secret lives. For each production key and password, note which platforms, build systems, and laptops hold a copy. When Railway sends the "roll everything" email, this list is the difference between an afternoon and a week. It is also a good answer to have ready when a client's security questionnaire asks how you manage credentials.
Practise a rotation. Pick your most important credential and change it, end to end, timed. If it takes more than an hour or breaks something, fix that now, while nothing is on fire. Anything you have never rotated is the next Zeabur token.
One key per tool. Give each platform, service, and AI tool its own key with only the access it needs. When one leaks, you revoke one key, not the one shared by everything. It also tells you which supplier the leak came from.
Cap what a stolen key can spend. Set hard spending limits and usage alerts with OpenAI, Anthropic, Google, and your cloud provider directly. A limit in a middleman's dashboard does not stop someone using a copied key elsewhere, as OpenCode's users found.
Prefer keys that expire. Most cloud providers let your build system sign in with short-lived credentials instead of a long-lived key stored in its settings. A key that lasts an hour is worth far less to whoever copies it.
Pin what you install. Lock dependency versions and verify them by hash, and do not let build jobs run install scripts that fetch "latest" unchecked. That is what kept Homebrew users clear of the OVHcloud swap. If you ran LiteLLM 1.107.0 anywhere, including a build runner or a Docker container, treat every secret that machine could see as stolen and replace it from a clean machine.
If you are not technical yourself, there is one question worth asking whoever builds your software: if our hosting provider told us tomorrow to change every key, how long would it take? A confident answer measured in hours is a good sign.
How Steelwise can help
Mapping where your secrets live, and making sure you can change them quickly when a supplier has a bad week, is the kind of practical review we do for small software teams. Get in touch.
Further reading
- Railway: Security disclosure, 21 October
- PingCAP: Unauthorised access to PingCAP cloud resources
- NCSC: Secure development and deployment guidance
- OWASP Secrets Management Cheat Sheet