I stood in the corner of the testing suite, staring at a small patch of peeling acoustic foam, and for a solid 11 seconds, I couldn’t for the life of me remember why I had walked into the room. It was one of those cognitive short-circuits where the purpose of your movement just evaporates, leaving you standing there like an unrendered character in a video game. As a foley artist by trade, I’m used to analyzing the world through its textures-the way a pilot’s leather jacket crinkles when they reach for the overhead panel or the specific, hollow ‘thwump’ of a stickpit door sealing shut-but here, in the realm of aviation language assessment, the textures are much more jagged. I finally remembered I was there to observe the interaction between an examiner and a candidate, but the momentary lapse felt like a perfect metaphor for the entire ICAO Rating Scale system: we know where we are, but we often forget exactly how we’re supposed to get to the result.

The Rigidity of Silos

The candidate sat across from the examiner, fidgeting with a pen that made a rhythmic, clicking sound that I knew I’d later recreate using a ballpoint and a plastic cup. This candidate-let’s call him Candidate 101 for the sake of my obsessive need for numbers to line up-was currently navigating the treacherous waters of a simulated emergency. He was technically proficient, his verbs were mostly in the right places, and his vocabulary didn’t fail him when describing a hydraulic leak. Yet, there was this invisible tension in the room. The examiner was scribbling notes, looking down at a rubric that promised objective clarity but delivered subjective fog. We are taught that the scale is a scientific instrument, a yardstick for safety, but in the heat of a 31-minute assessment, that yardstick often starts to feel more like a divining rod.

There is a fundamental contradiction in how we train people to use the ICAO descriptors. On one hand, we demand a rigorous adherence to the six pillars: pronunciation, structure, vocabulary, fluency, comprehension, and interactions. We treat these like isolated silos, as if a human being can be neatly partitioned into linguistic compartments. But language isn’t a series of drawers; it’s a soup. When Candidate 101 stumbled over a word but immediately corrected himself with a joke that showed high-level interactional competence, the examiner’s pen hovered. Do you penalize the fluency or reward the interaction? The scale, in its theoretical elegance, suggests there is a right answer. The practice, however, suggests that we are asking examiners to perform a feat of mental gymnastics that the human brain isn’t naturally wired for. We criticize the inconsistency of raters, yet we continue to hand them a tool that requires them to collapse infinite human complexity into a single, sterile digit.

Listening for the Soul

We have created a credentialing system that prizes the ability to mimic a specific type of ‘operational’ speech, but we haven’t quite figured out how to measure the ‘soul’ of communication. This is where the theory-practice chasm becomes a canyon. We train examiners on the theory-the ‘what’ of the scale-but we leave them hanging when it comes to the ‘how’ of the messy, real-world application. I once tried to explain to a lead trainer that the silence between two words carries as much information as the words themselves, but he just looked at me like I was trying to sell him a haunted microphone. He wanted data. He wanted 101% certainty in a field that is, by definition, an art form masquerading as a science.

Take the concept of ‘fluency.’ In the handbook, it’s about tempo and the absence of distracting hesitations. But in reality, a pilot who speaks with a slow, deliberate cadence might be a much safer communicator than one who rattles off checklists at 181 words per minute with perfect syntax. The scale struggles with this. It wants a specific rhythm. My work as Sky R.J. involves creating the illusion of reality, and I see the same thing happening in these booths. Candidates learn the ‘foley’ of English-the right clicks and pops to make it sound like they are at Level 5 or Level 6-without necessarily possessing the underlying linguistic resilience to handle a truly unexpected, non-routine event. We are testing the performance, not the performer. This is a failure of the system that remains largely invisible because, on paper, everyone is checking the right boxes.

The Contrary Candidate

I remember one specific instance where a candidate was describing a bird strike. His pronunciation was, frankly, a bit of a mess. He hit the consonants too hard, and his vowels were stretched like old rubber bands. By the strict definitions of the scale, he was leaning toward a lower score. But his comprehension was lightning-fast. He understood every nuance of the examiner’s prompts, even the ones designed to trip him up. He was a perfect example of the ‘contrary’ candidate-someone who breaks the internal logic of the rubric. I watched the examiner struggle. There was a visible weight on her shoulders, the pressure of trying to fit a square peg into a hexagonal hole. In the end, she gave him the benefit of the doubt, but she couldn’t articulate why. She just ‘felt’ he was safe. This ‘feeling’ is what the system tries to beat out of people, but it’s actually the most valuable tool we have.

So, we cling to the scale like a life raft, even as it drifts further away from the reality of the stickpit. I think about the 151 different ways I can make the sound of footsteps on gravel. Each one tells a different story: is the person running? Are they heavy-set? Are they tired? Language is the same. A hesitation isn’t just a hesitation; it’s a data point. But until our training reflects the complexity of these data points, we will continue to have this gap. We are training people to be recorders when we should be training them to be listeners.

The Sound We Miss

I once spent 21 hours trying to get the sound of a jet engine right for a documentary. I tried vacuum cleaners, blow dryers, and even a heavily processed recording of a localized thunderstorm. Nothing worked until I realized I was focusing on the roar when I should have been focusing on the whistle. Aviation training often focuses on the ‘roar’-the big, obvious markers of language proficiency-and misses the ‘whistle’-the subtle cues of cognitive load and situational awareness that are buried in the way a person speaks. If an examiner isn’t trained to hear the whistle, they aren’t really assessing safety; they’re just assessing grammar. And in a stickpit, grammar never saved anyone’s life, but clear, resilient communication has saved thousands.

There is a certain irony in the fact that we use a standardized scale to measure something as non-standard as human speech. We have this dream of a world where every Level 4 in the world is identical to every other Level 4, but that’s a fantasy. A Level 4 in a high-context culture sounds different from a Level 4 in a low-context one. Our training needs to acknowledge this cultural friction. We can’t just pretend the scale is a universal constant like the speed of light. It’s a social construct, and like all social constructs, it requires constant maintenance and a healthy dose of skepticism. If we don’t allow examiners to question the scale, the scale becomes a dogma rather than a tool.