Staging isn’t production

Development and staging are useful. I use both. But neither is production, and I don’t expect them to behave like it.

You can test a feature, run through it in staging, and feel pretty good about the release. Then it gets real traffic and you find out there was more going on than your test covered.

That doesn’t automatically mean the testing was bad. There’s only so much you can reproduce. The mistake is treating a successful staging run as the end of the work.

Traffic doesn’t arrive one request at a time

When you’re checking something yourself, you tend to do things in a fairly predictable order. Open a page. Submit a form. Check the result.

Meanwhile, production has people doing all of that at once. Background jobs are running. Partner callbacks are coming in. Somebody clicks the button twice because the first request hasn’t finished yet.

A bit of work that seems cheap on its own can get expensive when it’s happening constantly. And two requests touching the same record can expose a problem you’ll never see clicking through a feature by yourself.

Load testing helps. So does making staging more realistic. But I still need to see what happens after the release. Where requests are slowing down, whether jobs are backing up, which errors are actually occurring.

Otherwise I’m guessing.

A small database can hide a lot

It’s easy to miss a slow query when there isn’t much data for it to work through.

Production has the history. Old records, uneven data, accounts with much more activity than others. The page you tested with ten results might have thousands for a real user.

Sometimes the query itself needs work. Sometimes the application is making far too many queries. Loading something once for every row doesn’t look especially alarming until there are a lot of rows.

And sometimes a query is waiting on another transaction. An index won’t fix every problem just because the symptom is a slow page.

I care about what the application is asking the database to do, how often it asks, and what else is happening at the same time. Looking at one query in isolation only gets you so far.

The other system has its own problems

Integrations are another place where the happy path can give you too much confidence.

The partner’s test environment responds. The payload looks right. Your code handles it. Good.

Now what happens when the response is late? Or the callback arrives twice? Or your request times out, but the other system already processed it?

That last one matters. You can’t assume nothing happened just because you didn’t get an answer. Retrying may repeat an operation that already went through.

Timeouts, sensible retry limits, handling duplicates, keeping enough information to reconcile the two sides—those things need thought before there’s a problem. A successful API call only proves one part of the integration works.

I still have to run it afterward

That’s probably the biggest influence on how I build software. I’m often the person who has to keep it running after it ships.

So I need to be able to tell what a release changed. I need a way back if something goes wrong. And when production exposes an assumption I got wrong, that needs to feed into the next change.

Staging helps me get ready to release. I still have work to do once the release is out there.