Readability, Understandability, and ETS

Alex Wong

It doesn’t take a rocket scientist to figure out that the formulas available for estimating readability are less than foolproof. It doesn’t even take a linguist to notice that things are missing from the formulas.

Last week I offered a link to the website Readability Score, where you can take any text and paste it in for an instant estimate by half-a-dozen different formulas, all purporting to determine the grade level of the passage. They count the average length of sentences, average length of syllables in words, average number of letters in words. The longer the averages, the higher the grade level needed to understand the passages. Supposedly.

What’s left out? Grammatical complexity, for one. Vocabulary, for another. Negatives, for another: It’s not impossible to understand multiple negatives, but not to be underestimated either. And so on.

Fortunately, there’s another free estimator of text readability that takes account of many additional factors. It’s known as TextEvaluator, recently developed by the Educational Testing Service. And just like Readability Score, you can paste in a passage for a free evaluation.

Accessing it is a little complicated, but it’s worth the effort. First go to the sign-in page and enter your email address in the “sign in” box. (It invites you to enter a “client code” too, but that’s not necessary.)

Then you’ll get an email right away with a link to the page that does the evaluating. Click on that link, and there you are. You can enter or paste text up to 1,600 words. And you can specify a grade level. Press “Get Analysis,” and you’ll promptly get another email, this one with a link to a page that shows your results.

On Readability Score, Donald Trump’s speech to the American Israel Public Affairs Committee had a grade-level reading of 7.8. By TextEvaluator, 1,600 words of that speech were readable within grade-level 7, and above grade-level 6.

Using Readability Score, Lincoln’s Gettysburg Address was at a 10.7 grade level. TextEvaluator says it’s above level 10 but within level 11.

The impressive thing about this program is not just that it considers more factors, though it does. Furthermore, other web pages on the TextEvaluator site give considerable detail on the whole endeavor.

The results page lists evaluations in four categories (two with subcategories): sentence structure, vocabulary difficulty, connections across ideas, and organization, as well as overall text complexity on a scale of 100 to 2,000, 100 being appropriate for very young readers, 2,000 for college graduates. Lincoln at Gettysburg scored 1,130 by this measure, Trump 800.

In this post there’s not space enough to do justice to the sophistication of the ETS approach. Try it and see for yourself.

Correction (4/22/2016, 11:55 a.m.): An earlier version of this post said incorrectly that TextEvaluator categorized 1,600 words of Trump’s speech as not readable at grade-level 7. Those word were in fact categorized as readable within that level.