Interview sampling design based on recommendations from @Kseniya_Vasil
340People in report
8Polar groups
16Interviews
All numbers and names are examples
What decision it supports
The business task comes from the report: for which work tasks the current set of AI tools falls short and what to test before buying. The report also says who makes the decision and by which signs. The sample decides whose tasks we will see in the interviews.
First, define which ways of getting tasks done you need to see: through the corporate AI tool, through other AI tools, by hand or through colleagues, and among people who tried the tool and quit. The ChatGPT Enterprise usage report is one of the sources: it finds the first group and those who quit. Users of other tools are found through a company-wide questionnaire and license and purchase lists.
Input
Table of people from the report, a company-wide questionnaire, license and purchase lists
What the method does
Picks 2 people for each way of getting tasks done
Output decision
Who to talk to before buying and what to test, up to the conclusion that no new tool is needed
Why a top-by-activity list is not enough
Three ways to recruit 16 people. Dot color is the usage scenario. The most active people and volunteers are good respondents and belong in any sample, but they are not enough on their own.
partialTop by messages
scenarios: 2 of 8
partialVolunteers
scenarios: 3 of 8
fullPolar groups
scenarios: 8 of 8
Takeaway
In this exampleTop by messages covers 2 of 8 scenarios, volunteers cover 3, and polar groups cover all 8.
In generalThe most active and responsive people know the tool best and are worth inviting. They are similar to each other and together describe one or two scenarios. To hear the rest, the sample is filled with people at the extremes of other attributes.
Next stepTake 2–3 people from the top by messages. Give the remaining slots to people with other scenarios: those who work only through Codex, quit the tool, use other tools, or run many projects.
Eight groups
Six groups come from report thresholds, "quit" from the last activity date, and "other tools" from the questionnaire. Each group has a question it helps test. The answer can be anything, including "no new tool is needed". The thresholds here are a guide: tune them so each group gets 2–5 people.
Most active in code
code > $200 / 3 mo
what is missing in development: a tool, repository access, or nothing
Hana Sato, Luis Moreno
Agent tasks
agent > $60 / 3 mo
where the agent falls short and why: data, the task, or the agent itself
Mira Castellano, Tomás Reid
Intensive research
> 3,000 msgs, chat is primary
how they check answers and whether the current search is enough
Jonas Keller, Aiko Lind
Codex only
0 msgs and > $20 / 3 mo
which tasks they handle without chat and whether they need anything else
Ben Ortiz, Lena Moss
Inactive
0 msgs and $0 for the period
how they get their tasks done and which options they considered
Pavel Novak, Sara Quinn
Tried and quit
had messages, 0 in the last 60 days
why they stopped: result, time, security, or the task went away
Farah Nasser, Yara Selim
Project organizers
≥ 6 active projects
what projects give them and what they lack
Elif Demir, Rafael Costa
Other tools
per questionnaire: main AI tool is not the corporate one
what is better in the other tool and whether the current one can offer it
In this example16 conversations: two in each of the eight groups.
In generalTwo or three people per group show whether a problem repeats: if two out of three say the same thing, that is a signal. People with average values are left out: their answers resemble neighboring groups and add no new scenarios across 16 conversations.
Next stepIn each group, sort candidates by its threshold and take the two with the most extreme values, from different teams where possible. If a person falls into two groups, keep them in the one where their value is more extreme and take the next candidate in the other.
Team coverage and replacements
A separate step after recruiting that does not change the groups: how many selected people fall in each team and who replaces those who decline.
Team
Regular users
Selected
Status
Team A
covered
Team B
covered
Team C
covered
Team D
after replacement
Team E
after replacement
Takeaway
In this exampleAfter recruiting by group, Teams D and E had nobody. Three candidates were replaced with people from D and E with the same behavior, and all five teams are now covered.
In generalSpend and message thresholds catch extreme users. A team where everyone uses the tool a little never crosses a threshold, and its tasks drop out of the research. That is why teams are checked as a separate step.
Next stepLay the selected people out by team. If a team has nobody, replace one candidate in a fitting group with a person from that team with the same behavior. Replace anyone who declines with the next candidate of the same group by its threshold.
What to learn in the interview
The conversation starts from the last work task, not from the tool: what they needed to get, what means they had, what they chose, and how they got to the result. The full script is in the interview article.
Task
The last work taskWhat they needed to get and for whom
Means and choiceWhat they had, what they chose and why
Path
How they got to the resultWhat they used as is, what they finished themselves
Effort and constraintsTime, data, access, security
A successful case
When it went wellWhat was different then
Quitting and switching
Quit or changed toolsWhat they switched to and why
Takeaway
In this exampleSix points: four about one task, a successful case, and quitting or switching.
In generalA story about a past task gives facts: you see where the person chose AI, where another way, and why. The successful case shows what already works; quitting and switching show why tools get replaced. A forecast about the future cannot be checked.
Next stepAsk them to name their last work task in the first minutes and run the whole conversation on it. If the person does not mention AI, ask which options they considered, not why they do not use it.
Workflow
1
Collect candidates into eight groups
Six groups from report thresholds, "quit" from the last activity date, "other tools" from the questionnaire
2
Keep 2 people per group
From different teams if you have a choice; one person in one group only
3
Check team coverage
If a team has nobody, replace one candidate with a person from it with the same behavior
4
Schedule 30-minute interviews
The conversation starts from the last work task: what they needed to get, what means they had, what they chose, and how they got to the result. With those who quit, also discuss why they stopped
Limitation. 16 interviews show which problems occur and what they look like in real work. They cannot tell you how many people in the company face each one. To learn the scale, send the whole company a short questionnaire, written after the interviews. If the problems are not confirmed, that is also a result: no new tool is needed.