Production databases tend to be treated rather well. They are encrypted, access is restricted, changes are controlled, activity is logged, backups are protected, and security teams periodically ask awkward questions about who can see what.
Then somebody needs to reproduce a bug.
A production snapshot gets restored into staging, the bug appears immediately, QA is happy, engineering can finally fix it, and everyone wonders why security is making that face.
The problem is not that testing happened. The problem is that the staging environment may now contain the same customer names, email addresses, phone numbers, transactions, health information, support conversations or other sensitive data as production, except with broader access, more verbose logging, experimental integrations and a retention period best described as “until someone remembers”.
Production data has not disappeared. It has simply moved somewhere less protected.
This is one of those problems where GDPR, ISO 27001, NIST, PCI DSS and basic security engineering all end up pointing in roughly the same direction. But simply saying “do not use production data in testing” is not particularly helpful.
Anyone who has actually tried to reproduce a difficult production defect knows why.
“We need live data”
This sentence has probably added several years to the collective lifespan of security meetings.
It is also sometimes true.
Real systems contain ugly data. Names contain unexpected Unicode. Phone numbers arrive in formats your validation library swears should not exist. Supposedly mandatory database fields are NULL. Records created by version 2.3 are still interacting with records created by version 11.6. Historical migrations left behind combinations nobody would design today.
Events arrive twice. Events arrive late. Events arrive in the wrong order. Sometimes they do all three and then open a support ticket.
A billing calculation may fail only for an old account type with a partially refunded transaction created before a particular migration. A parser may break only when a string combines two scripts, an emoji and an invisible Unicode character. A queue consumer may behave perfectly until a retry collides with an event another worker has already processed.
Your carefully constructed customer called John Smith, born on 1 January 1990 and living at 123 Test Street will probably not help.
So QA says it needs production data. Security says production data should not be in test.
The useful answer is not to decide which department wins.
It is to work out what “we need production data” actually means.
Production data is not one requirement
When someone says a test requires production data, they are usually asking for one or more characteristics of production.
Those characteristics are worth separating:
Shape
Lengths, formats, encodings, missing values, malformed values, Unicode behaviour and payload structure.
Distribution
How often values occur, how common edge cases are, the spread of transaction sizes, account types, languages or countries.
Relationships
Which objects belong together and how many of each exist: customers to orders, claims to documents, tickets to messages.
History
Legacy schema versions, migrated records, deprecated states and data created under old business rules.
Sequence
The order and timing of events, retries, race conditions and state transitions.
Scale
Millions of rows, payload sizes, throughput, concurrency and queue depth.
Identity
The fact that this record belongs to Jane Müller from Hamburg rather than to a generated person.
That last category is important because most tests need some combination of the first six.
Very few genuinely require number seven.
This distinction changes the conversation from:
“Can we use production data?”
to: “Which properties of production do we need to preserve for this test?”
That is a much better engineering question.
Steal the behaviour, not the people
Suppose QA wants to test whether names are handled correctly. What it may actually need is a dataset containing realistic character sets, string lengths, punctuation, transliteration behaviour and locale-specific combinations.
It does not need the names of your actual customers.
Suppose a defect occurs only for a strange sequence of transactions. QA may need the amounts, ordering, timing, status transitions and relationships between those transactions.
It probably does not need the customer's email address.
Suppose performance testing needs to reproduce production load. You may need record counts, payload sizes, cardinality, query patterns and concurrency.
The identity of the 2.3 million people represented by those records adds remarkably little to the load test.
This sounds obvious when written down. It becomes less obvious at 16:47 on Friday when someone has just discovered a production incident and pg_dump is sitting there looking extremely efficient.
Synthetic data should be informed by production
“Use synthetic data” is good advice until you ask someone to implement it.
A synthetic dataset created from the application schema is usually far too clean.
It knows that phone_number is a string with a maximum length of 32. It does not know that 0.4% of production records contain leading spaces, 1.2% have an unexpected country prefix and twelve records somehow contain letters.
The better approach is production-informed synthetic data.
You can profile production without copying the underlying identities. Useful characteristics include:
- field cardinalities
- NULL frequency
- string length distributions
- value ranges
- enum frequencies
- payload sizes
- relationship cardinalities
- timestamp distributions
- common and rare state transitions
- combinations of fields that occur together
- invalid-but-tolerated records
- schema-version distributions
Production may tell you, for example, that 6% of accounts have more than one address, 0.3% of orders contain a legacy status and 2% of phone numbers exceed what the current UI allows users to enter.
Those are excellent test cases.
The underlying customer records are not.
A test-data generator can reproduce those characteristics deliberately. Over time, that generator becomes a model of the weirdness your production system has accumulated.
Which, unlike most technical debt, is weirdness you actually want to preserve.
Every production incident should improve the test corpus
There is a useful principle here:
Every production bug that requires production data to reproduce should make the next production-data copy less necessary.
Imagine a defect occurs for customer 18473921.
Initially, nothing reproduces the problem, so two authorised engineers inspect the original record in a controlled environment.
Eventually they discover that the actual trigger is:
schema_version = 3
account_type = legacy
refund_status = partial
currency = CZK
invoice_count > 3
event_order = [payment_created, refund_created, payment_confirmed]
That is the useful information.
The regression test should preserve those conditions, not customer 18473921.
The lifecycle should look like this:
Production failure
↓
Identify the relevant data characteristics
↓
Create deterministic fixture or generator seed
↓
Verify that it reproduces the defect
↓
Add it to the permanent regression suite
↓
Delete the production-derived test copy
The original record helped you discover the bug.
It does not need to become a permanent museum exhibit.
This is also why deterministic test data matters. Random generation is useful for finding things you did not expect, but if a generated dataset exposes a bug, record the seed or convert the state into a fixture.
You want:
"seed = 928471"
not:
“The fuzz test failed once on Wednesday.”
Masking is not just changing "name" and "email"
Sometimes generated data is not enough.
Migration testing is a classic example. You may need years of accumulated data inconsistencies because the entire purpose of the test is to find out whether the migration survives years of accumulated data inconsistencies.
In that case, a transformed production snapshot may be justified.
But transforming it properly is harder than:
UPDATE customers
SET name = 'John Doe',
email = 'john@example.com';
That may remove two obvious identifiers while leaving phone numbers, addresses, IP addresses, exact timestamps, account identifiers, free-text notes and transaction histories untouched.
Relationships between records may also identify people even when the obvious fields have been replaced.
And then there is free text.
Free text is where elegant masking plans go to die.
A field called "support_note" might contain:
Customer Sarah Müller called from +49...
She asked us to send the medical report to...
No amount of masking "customers.first_name" will fix that.
Support tickets, call transcripts, comments, uploaded documents, CRM notes and error payloads need separate treatment because users have an impressive ability to place personal information in any field capable of accepting characters.
If the text is irrelevant to the test, drop it. If only structure matters, generate replacement text. If linguistic behaviour matters, preserve language, length and shape without preserving the person.
If the original wording genuinely matters to reproduce a defect, that may justify a tightly controlled exception.
There is no single magic masking query.
Sorry.
Good masking preserves what the application needs
Bad masking solves the privacy problem by destroying the test.
If every date becomes 2000-01-01, every phone number becomes 0000000000 and every customer becomes John Doe, the dataset may be beautifully anonymous and almost entirely useless.
Useful transformation needs to preserve relevant properties.
Dates can be shifted consistently while maintaining intervals. Phone numbers can be substituted with valid numbers from the same numbering format. Email addresses can be replaced while preserving uniqueness. Numeric values can be perturbed without flattening the distribution.
And relationships need to survive.
Suppose the same production customer identifier appears in:
customers
orders
invoices
support_tickets
analytics_events
If each table produces a different random replacement, referential integrity disappears.
One useful pattern is deterministic tokenisation:
test_id = HMAC(test_key, production_id)
The same source identifier generates the same test identifier wherever it appears, while the production identifier itself does not need to travel into the test environment.
The key matters, obviously. Store it separately and protect it appropriately.
Hashing customer IDs with no secret key and declaring the result anonymous is one of those things that looks much better in a compliance spreadsheet than in a threat model.
Different tests need different data
Another recurring problem is the magical environment called staging, which somehow serves every testing purpose known to engineering.
It should not.
Different test types have different data requirements.
Unit and component tests should use generated fixtures.
API and contract tests should use generated payloads around schema boundaries and unusual combinations.
Regression tests should preserve deterministic representations of bugs discovered in production.
Performance tests need scale, distributions, payload sizes, concurrency and access patterns. Real identities generally add nothing.
Migration tests may genuinely need transformed production snapshots because historical weirdness is exactly what is being tested.
Production incident reproduction should start with telemetry and the minimum possible dataset. Where original data is genuinely required, keep the scope narrow and the environment temporary.
The point is not to choose one universal test-data strategy.
The point is to stop using one universal copy of production for every problem.
Sometimes production data really is necessary
After all of this, there will still be cases where the conclusion remains:
We need the original data.
Fine.
Using production data as an exceptional debugging tool is very different from using production data as your test-data strategy.
The first can be controlled.
The second is usually architectural laziness with excellent historical precedent.
When the original data genuinely is necessary, reduce the blast radius first.
-
Do you need the whole database or eight related records?
-
Do you need every column?
-
Do you need the actual name, address and phone number?
-
Do twenty engineers need access?
-
Does the dataset need to survive for three months?
-
Can the investigation happen in an isolated environment rather than general staging?
Once production data enters the environment, treat that environment according to what it now contains. That may mean tighter IAM, MFA, encryption, access logging, blocked outbound traffic, disabled third-party integrations, restricted exports and no local copies.
Most importantly, give the environment an owner and an expiry.
“Temporary” is not a retention period.
Make sensitive test environments ephemeral
Infrastructure automation gives us a better model than the permanent staging environment where old datasets slowly accumulate sediment.
For a sensitive production-data investigation, create an isolated environment specifically for the case.
For example:
environment: incident-4821
owner: team-payments
classification: production-derived
created_at: 2026-08-24T13:00:00Z
expires_at: 2026-08-25T13:00:00Z
network_policy: restricted
external_integrations: disabled
Load only the required data, perform the investigation and destroy the environment afterwards.
Better still, make "expires_at" functional rather than decorative.
The environment should disappear automatically unless somebody explicitly extends it. The same principle applies to snapshots and object storage through lifecycle rules.
Humans are excellent at creating infrastructure.
Remembering to delete it later is apparently an optional module.
Test data drifts too
A one-off synthetic dataset is not enough because production changes.
New countries are launched. New integrations appear. Customer behaviour changes. New versions introduce new object states. Input lengths, payload sizes and transaction distributions drift.
A test dataset that represented production accurately in January may be noticeably less representative by September.
So compare the profile of your test data with the profile of production periodically.
You do not need to compare individual customer records. Compare the characteristics.
-
Have NULL frequencies changed?
-
Are payloads getting larger?
-
Are new language combinations appearing?
-
Has the number of objects per customer shifted?
-
Are there new state transitions?
When meaningful drift appears, update the generator or transformed dataset.
If we already accept that production traffic and schemas drift, assuming test fixtures remain permanently representative would be slightly optimistic.
Better observability reduces the need to copy data
If engineers repeatedly need production snapshots to diagnose issues, there is another question worth asking:
Why can we not diagnose this safely in production?
Sometimes the test-data problem is partly an observability problem.
Good correlation IDs, structured logs, trace information, state-transition histories and carefully designed diagnostic metadata can expose the technical conditions behind a failure without exposing the whole customer record.
For example, this may be enough:
{
"state": "refund_pending",
"previous_state": "payment_confirmed",
"schema_version": 3,
"retry_count": 2,
"event_delay_ms": 4821
}
You may not need the customer's name, email, address and full request body.
Obviously, do not solve the staging problem by dumping all production data into logs.
We have already written that article.
The safe path has to compete with "pg_restore"
This is the organisational part that matters most.
If obtaining a compliant test dataset requires three tickets, two approvals and the personal blessing of the CISO, while restoring a production snapshot takes one command, you have designed a system in which the wrong option is faster.
Developers optimise for shipping.
QA optimises for finding defects.
Security needs to make the safe route usable.
That means providing actual tooling:
- realistic data generators
- production profiling
- deterministic seeds
- reusable edge-case libraries
- automated masking and tokenisation
- transformed snapshots
- minimal case extraction
- ephemeral test environments
- automatic expiry
Ideally, asking for a production-like test dataset should be easier than asking for production.
Otherwise your policy is competing against:
pg_restore prod.dump
The shell command has an excellent user experience.
A practical decision tree
When somebody says, “We need production data”, ask:
1. What production characteristic does the test require?
Shape, distribution, relationships, history, sequence, scale or actual identity?
2. Can we generate it?
If yes, generate it and make the test deterministic.
3. Can we derive the necessary characteristics from production without copying records?
Profile distributions, state transitions and correlations, then update the generator.
4. Can we transform a production snapshot?
Mask or tokenise what is unnecessary while preserving the properties and relationships required by the test.
5. Can we reduce the dataset?
Extract the smallest record set and minimum fields necessary.
6. Does the actual identity still matter?
If it genuinely does, document why and use a restricted exception environment.
7. How does the data leave again?
Define an owner, expiry, deletion mechanism and the downstream systems that may receive copies.
That is usually a much more useful conversation than Security saying no and QA saying reality is inconvenient.
Reality usually wins.
We may as well design for it.
Test data is an engineering capability
The broader lesson is that compliant testing is not mainly a policy problem.
It is a tooling problem.
A mature test-data capability lets teams reproduce the properties of production without treating the production database as the world's most convenient fixture library.
It learns from production, preserves difficult edge cases, supports deterministic reproduction, maintains relationships across systems, distinguishes performance characteristics from customer identity, and provides a controlled exception when original data genuinely is necessary.
Most importantly, it makes the compliant route practical.
“We need production data” is usually not a requirement.
It is the beginning of a requirements discussion.
Sometimes that discussion will end with generated data. Sometimes with a masked snapshot. Sometimes with deterministic tokenisation. Sometimes with a tiny production-derived subset in an isolated environment.
And occasionally it may end with the original data because there genuinely is no reasonable alternative.
That is fine.
What should make security teams nervous is not the existence of exceptions.
It is when the exception became the test-data architecture three years ago and nobody remembers approving it.
Yves-Philipp Rentsch
