What auditors actually ask for during SOC 2 fieldwork

The first time I sat in on fieldwork, I expected the auditor to want documents. They wanted arithmetic.

The request was not "send me your access review". It was: how many people had production access during the period, how did you arrive at that number, show me twenty-five of them, and explain why those twenty-five. Every follow-up for the next three days had the same shape.

If you are preparing for a first SOC 2 Type II, this is the single most useful thing to understand in advance, because it changes what you collect and how you store it.

Population, then sample

An auditor tests operating effectiveness by examining a subset of occurrences and inferring across the whole. That only works if two things are true, and they will probe both.

The population is complete. Every occurrence of the thing in the period. Not every occurrence you remembered to log, and not every occurrence in the tool you happen to be showing them.

The sample is representative. Drawn in a way that does not let you pick the good ones.

The second is what people expect. The first is where audits actually go wrong, and it goes wrong in a specific way: you cannot prove a population is complete using the same system that produced it. If your list of production access grants comes from your IdP, and your IdP is also the thing being tested, the auditor needs a second source — an HR roster, a cloud provider's IAM export, a joiners-and-leavers ticket queue — that independently agrees.

Getting asked "how do you know this list is complete?" and having no answer is the most common way a control that genuinely works fails its test.

The sample sizes, roughly

Sizes are the auditor's judgement rather than anything fixed in the Trust Services Criteria, and they vary by firm and by assessed risk. But the shape is stable enough to plan around:

Two things follow from this table that are worth internalising before you design a control.

A control that runs more often does not cost you proportionally more evidence — 250 occurrences and 5,000 occurrences both get sampled at around 25. Automating something from monthly to continuous does not multiply your audit burden.

A control that runs annually gets tested at n=1, which sounds easy and is the opposite. There is no margin. If your one annual penetration test happened three weeks after the period closed, you do not have a small deficiency; you have nothing.

What a request actually looks like

Fieldwork arrives as a list, usually in a spreadsheet, usually called a PBC — "prepared by client". Real entries look like this:

Provide a complete listing of all users with administrative access to the production AWS environment at any point during the period, including grant date and, where applicable, revocation date. Describe the source of the listing.

From the population above, we have selected the 25 users identified on the Sample tab. For each, provide evidence of documented approval prior to access being granted.

Provide a complete listing of all changes deployed to production during the period. We will select 25.

Notice the pattern: complete listing first, then they select. If you provide the sample, the sample is not independent and the test is compromised. Offering to "just send the twenty-five we think are relevant" reads as helpfulness and lands as a red flag.

Where it goes wrong

Four failures, in the order I have seen them most often.

The population and the evidence disagree. You give a list of 340 deploys from your CI system. The auditor picks deploy 211 and asks for its approval. The approval is in a pull request that was squash-merged and the branch deleted, so the reviewer is no longer visible. The control worked; the evidence evaporated. This is a retention problem masquerading as a control problem.

The date is missing. (Freshness is its own subject — see continuous control monitoring vs point-in-time audit.) An artefact with no collection date cannot be placed in the period. A screenshot of a correctly-configured setting proves it is configured now, which is not the assertion — the assertion is that it was configured throughout. Screenshots are weak evidence for exactly this reason, and system-generated exports with timestamps are strong for the same reason.

The scope is wider than the population. You claim the control covers all production systems; the sample includes a system acquired in month eight that was never onboarded. Now the population was incomplete, and every conclusion drawn from it is suspect.

The evidence is the wrong artefact entirely. For "changes are reviewed before deployment", the policy saying changes must be reviewed is not evidence. The branch protection rule requiring a review is better. The pull request showing that a review happened on the sampled deploy is what they asked for.

What to do about it before fieldwork

Do a dry run. Pick three controls, write down the population, generate the list, pick five at random yourself, and try to produce the evidence. You will find at least one gap, and finding it in month four is an inconvenience where finding it in month eleven is a qualified opinion.

The artefact-by-artefact list is in the SOC 2 evidence collection checklist. Then make the population query repeatable. If producing your list of production access grants takes a person two days of joining exports by hand, it will take two days again next year, and it will be slightly different, and the difference will be hard to explain. A query that runs on demand is worth more than the neatest spreadsheet.

And file evidence with its scope attached. Every artefact should carry what control it answers, what population it was drawn from, when it was collected, and by whom. This sounds bureaucratic until the first time someone asks "what is this file and why is it in this folder" about something collected eight months ago by a person who has left.

The bit that is genuinely reassuring

Auditors are not trying to catch you. Almost every finding I have watched land came from a gap in evidence, not a gap in control — the thing was being done, and nobody could show it after the fact.

That is a solvable problem, and it is solvable in advance. Most of what we build is aimed at exactly this gap: an artefact with no population, no date and no owner is a file, and turning files into evidence at fieldwork time is the expensive way to run an audit.