Home / Blog / Note

"Airtime is not contribution in a group exercise"

"The candidate who talks most usually wins the group exercise, and that is a measurement error rather than a judgement. A contribution map with four countable moves separates chairing the discussion from filling it."

"Airtime is not contribution in a group exercise"
Representational image.

The group exercise ends and the lead assessor reads her notes aloud. “Candidate 3 chaired it, brought the quiet ones in, kept us on time.” Candidate 3 spoke for eleven of the twenty minutes. Candidate 5 spoke for two, and at minute five said the costing assumption in the brief was wrong in a way that changed the group’s recommendation. Nobody wrote that down.

Both are then rated on the same dimension, usually influencing or impact. Candidate 3 outranks Candidate 5. The client gets a shortlist that looks defensible, and nobody in the room can say what evidence the ranking rests on, because it was never recorded in a form that could be counted.

This is the most common scoring failure in group exercises, and it is not scoring drift. Drift is two assessors applying different standards. This is worse: one standard, correctly applied, to the wrong variable.

Airtime predicts the vote, and the vote becomes the score

The link between how much someone talks and how much of a leader they look like is one of the more robust findings in small-group research. MacLaren and colleagues tested it in 33 heterogeneous groups of four to ten people working through a simulation, controlling for intelligence, personality, gender and role assignment. The zero-order correlation between total speaking time and leader-emergence votes was 0.67. In their strongest model, roughly 39 seconds of additional speaking was worth another vote. Using speaking time alone as the classifier, they identified the top-voted person in 70 per cent of groups.

Read that in assessment-centre language. Rate a group exercise by impression and you are running a speaking-time test with extra steps, the ranking consistent across assessors because the same bias is evenly distributed.

The distortion is not neutral. In a study of 94 discussion groups reported in the American Political Science Review, Karpowitz and Mendelberg found that in most groups women spoke for less than 75 per cent of the time men spoke, and the gap closed when the group was told to decide by unanimous vote rather than majority rule. A group exercise scored on airtime is also, quietly, a test of who takes the floor in a mixed room.

There is a second, blunter mechanism. The exercise rewards behaviour that correlates with extraversion, then we call the result leadership potential. The candidate who reads the brief and spots the flaw in the numbers has no route into the score, because the instrument only records the mouth.

The standards already tell you what to observe

The British Psychological Society’s best practice guidelines ask for assessors trained in the Observe, Record, Classify and Evaluate process, applied to the simulations in use, evaluating independently before the integration session, with each competency measured more than once.

That structure is sound and most centres claim to follow it. What it does not say is what counts as a recordable behaviour in a fluid group discussion. “Chaired the discussion” is not an observation.

So define the countable moves before the day. If your anchors for other exercises are still written as adjectives, fix that first: what an observable anchor looks like covers the test and three rewrites.

Step one: name the four moves worth counting

A discussion has four moves that carry information about the competency, and one that does not. Write them into the observer sheet with a space for the exact words used.

Move What it sounds like What it is evidence of
Opens a line Proposes a frame, sequence or criterion the group then uses Structuring an ambiguous task
Adds evidence Brings a fact from the brief that others had not used Analysis, not opinion
Tests an assumption Names an unstated premise and asks what happens if it is false Judgement under incomplete information
Closes a decision Restates the choice, names who owns the next step, checks agreement Converting talk into commitment
Fills the silence Restates, agrees, narrates the discussion Nothing scoreable on its own

The fifth line does the work. Most of what fills a group exercise is restatement and agreement, reasonable behaviour in a real meeting and evidence of nothing the centre measures. Putting it in the sheet gives an assessor somewhere to put the noise instead of letting it inflate a score.

Two rules keep the coding honest. A move counts once, so repeating a point four times is one move plus three fills. And the observer writes the words, not a summary. If you cannot quote it, you did not hear it clearly enough to score it.

Step two: build the contribution map, not a ranking

The map is a grid: candidates down the side, the four moves across, a quoted fragment and minute marker under each. It takes four minutes after a twenty-minute exercise and replaces the paragraph of impressions assessors currently write.

Candidate Opens a line Adds evidence Tests an assumption Closes a decision
C3 02:10 proposes ranking by payback period; group adopts it 09:40 cites depot capacity figure from Appendix B none 18:20 restates the chosen option, assigns costing to C5
C5 none 05:05 points out the brief’s costing excludes freight; group switches option 12:15 asks whether the deadline is fixed or negotiable none

Names and timings are illustrative. Read the rows without the airtime numbers and the ranking changes. C3 structured and closed, which is real value. C5 supplied the one piece of analysis that changed the answer. For an analyst who has to challenge a plan, C5’s row is the more expensive behaviour to find. For a project lead, C3’s closing move matters more. Either way the decision is visible and arguable, which the paragraph of impressions never was.

The map settles the chairing question too. Chairing is a cluster of moves, not a personality: opened three lines, tested two assumptions, closed the decision. Narrating for eleven minutes is not chairing.

Step three: change the scoring rule in three places

Contribution outweighs volume. Rate influencing on the four moves, not on presence. A candidate with one evidenced assumption test can score above one with three unclosed proposals. Write that into the assessor guide, because it contradicts what assessors have been rewarded for all their working lives.

Silence is not evidence, but it is a question. A candidate with an empty row is not automatically weak. First check whether the exercise gave them a route in. If the group self-selected a chair in the first ninety seconds, a quieter candidate may never have had the floor. Record that as a coverage gap and rate the dimension from another exercise, which the duplicate-measurement rule exists to allow. Do not fill the gap with an assumption about confidence.

The removed-contribution test. After the map is complete, ask each assessor privately which single contribution, if removed, would have changed the group’s answer. If two assessors name the same person, and it is not the loudest, that candidate has evidence the ranking misses. If nobody can name one, the exercise was too easy and the group cruised, which is a design finding rather than a candidate finding.

Step four: run the wash-up off the map

Two mechanics protect the wash-up from dissolving into seniority.

First, every assessor completes their own ratings, including the report form, before the integration session starts. Once discussion opens, independent judgement is gone. If the day runs late, cut an exercise, not the write-up.

Second, a three-minute cap per candidate, starting with the assessor who observed the least airtime. The person with the loudest memory speaks last.

For a centre whose wash-up has already turned into a trade over scores, the sequence in why scoring drift quietly kills a centre comes first. A map read aloud in an uncalibrated room just gives the loudest assessor better material.

An illustrative run

Twenty-four candidates for a team-lead intake, four groups of six, one twenty-minute prioritisation exercise, two observers per group.

Before the change, each group’s highest-airtime candidate finished first or second in 15 of 24 cases. That matches the research: the centre is largely measuring talkativeness.

After the map, five candidates who had ranked in the bottom half turn out to carry an evidenced assumption test or a decisive piece of analysis. Two of them move into the top six once the dimension is rated across the second exercise, an individual analysis. Three airtime leaders drop, because their rows contain openings with no closings and no evidence, which describes someone who starts things and leaves them.

The useful output is not the reshuffled list. It is that the wash-up argument changes character. Instead of “I felt she led the discussion”, the debate is “C5’s freight point changed the group’s answer, do we credit that as influencing or as analysis”. That is a decision the client can sit in on and defend. These numbers are illustrative and constructed, not drawn from a client engagement.

What this cannot do

Four countable moves do not measure judgement. A candidate can supply a well-timed but irrelevant fact and the map records it as evidence. Pair the group exercise with an individual exercise where the quality of the analysis can be checked, and never let it alone carry a decision on a thinking dimension.

The map rewards a particular confidence: naming an assumption out loud. In a cohort where juniors have learned not to challenge a brief, that skill is shaped by hierarchy. If your brief contains a deliberate error, brief the observers that finding it is scored, then check whether anyone from a shopfloor background sat with three managers. The finding may be about the room.

The last limit is the one that matters in a wash-up. The map records twenty minutes of an artificial task, observed by two people who know the organisation and its vocabulary. It is better evidence than an impression, and it is not a year of watching someone run a shift.

Monday

Add four columns to your last group exercise’s observer sheet: opens a line, adds evidence, tests an assumption, closes a decision. Run the next cohort with them and the quoting rule, then count how many candidates change rank once the airtime number comes off the page. If it is more than a couple, you have been buying a leadership shortlist from a talking test. Our assessment centre design work starts from that sheet, and if the wider question is which dimensions your centre should rate at all, begin with cutting the competency framework down to what managers use.

Sources

https://gwern.net/doc/psychology/linguistics/2020-maclaren.pdf https://news.byu.edu/news/study-deciding-consensus-can-compensate-group-gender-imbalances https://www.klug-md.de/downloads/BPS_PTC_AC_Guidelines.pdf

Keep reading
Competency

"Writing behavioural anchors a manager can actually observe"

Most competency anchors describe a conclusion, so two managers watching the same person reach different ratings. Here is the wording test that fixes it, and how to pilot an anchor set in one afternoon.

Coaching

"A coaching contract the line manager signs"

"Coaching fails at the boundary with the manager's own work, not inside the session. Three clauses to sign before session one: what the manager stops doing, what they get told, and what happens if nothing moves."

Want this applied inside your organisation?

Talk to us