Tuesday, August 6, 2013


Lexile Scores Do More Harm Than Good

Employing photographs of three very wholesome looking children on their website at Lexile.com, MetaMetrics promotes the organization’s Lexile Framework by way of a very cute, little girl of elementary school age, a mop-topped middle-school fellow, and broad-shouldered, bright looking young man, who obviously represents the high school crowd.  In front of each young person is a two-tone, green dot that denotes the Lexile level at which each child should be reading: 550L, 1000L, and 1300L respectively. The trouble begins here, and it comes in at least three forms. One: The Lexile scores are very often unreliable and misleading.  Two: Parents, teachers and curriculum administrators are using Lexile scores as a tool for considering text quality, and that is bad for children.  And, three: The implication that by high school, students should be reading books that earn a 1300 Lexile score is pedagogically unsound and pragmatically impossible.

A Look at the Lexile Framework:

The Lexile Framework attempts to “match readers with texts” by way of three “dimensions:” “Reader and Task,” “Qualitative,” and “Quantitative.” 

From Lexile.com “Reader and Task” addresses ideas such as “students' knowledge, motivation and interests.” “Qualitative” dimensions of a text consider “complexity, such as levels of meaning, structure, language conventionality and clarity, and knowledge demands.” “Quantitative” measures include “text complexity, such as word frequency and sentence length, which are typically measured by computer software.”  If all three inter-related dimensions are considered together, the framework seems arguably sound; however, what is being relied upon by too many parents, teachers, school districts—and even the Common Core—is only the quantitative score.  And, Lexile itself, by way of three charming children, is promoting the idea that its product be used that way.  The website even has a cell where a title can be inserted to get a Quantitative Lexile score for a book.  And this would be fine if the scores were in the least meaningful, helpful, or accurate.

To clarify and expand, here is how a quantitative score is crafted according to a Lexile Agent from the Lexile Support Center blog: “In order to Lexile a book or article, text is split into 125-word slices. Each slice is compared to the nearly 600-million word Lexile corpus – taken from a variety of sources and genres – and words in each sentence are counted. The sentence length and difficulty of the vocabulary is examined throughout the book. These calculations are put into the Lexile equation. Then, each slice’s resulting Lexile measure is applied to the Rasch psychometric model to determine the Lexile measure for the entire text. Lexile measures do NOT measure age-appropriateness, the book quality, the book's theme or other such characteristics of the book.”

The Lexile Framework in Action:

Within this methodology, some books earn understandably high scores: Don Quixote scores a 1410L; Moby Dick a 1200L; To the Lighthouse 1030L.  Some scores are more curious.  The Sound and the Fury and Beloved each earn 870, but Farewell to Arms gets a 730L. But, the real absurdity of the Quantitative Lexile score is reveled by making comparisons.

While the novel Beloved earns 870L, the nonfiction book Understanding Toni Morrison’s Beloved and Sula, garners 1370L. Similarly, The Sound and the Fury (870L) cannot stand up against the nonfiction book that attempts to explore the novel, William Faulkner’s the Sound and the Fury, which earns 1420L.  This condition, whereby the nonfiction book that seeks to supplement the novels, is ironically universal in the Lexile Framework.  To be fair, any English major knows that books that include in-depth analysis and criticism can be tough reads.  And, while the two nonfiction books may indeed have longer sentences that lead to higher scores, it is highly unlikely that any reader would find them more challenging than the novels themselves—particularly to the degree suggested by the difference in their scores.  Certainly, if all of the above-mentioned books (fiction and nonfiction) were assessed in terms of all three dimensions of the Lexile Framework, a more comprehensive understanding might be reveled, but that type of analysis is not what is being promoted by Lexile, nor is that how the framework is being employed by schools or parents.  But, the picture gets yet more bizarre.

A parent seeking a good read for his or her child—or a school administrator considering a request by a teacher to teach a certain book—might be interested to learn that the Hunger Games scores 810L while Hemingway’s classic The Sun Also Rises scores a 610L, which leaves it even short of The Outsiders at 750L. The Fellowship of the Ring at 860L is no match for the sixth-grade staple Hatchet, at 1020L. And, Harry Potter books outrank The Catcher in the Rye, which at 790L, seems remarkably close in text complexity to The Giving Tree at 530L.  Again, if this discussion were held in terms of all three dimensions of the framework, there might actually be a cogent discussion.  But, that’s not how it’s happening.  And that is not how it will even happen.  Parents and educators are too busy for a framework that seeks to manage such complex and qualitative discussions, and Lexile’s own website is steering people away from such discussions anyway.

In a perfect world, parents and educators would have the time and resources to use all three dimensions of the Lexile Framework to evaluate every title for every grade level and/or every child.  This can’t happen, and perhaps this understanding is why Lexile promotes misleading application of its own product.


The Lexile Framework and the Common Core:

More worrisome, however, is how the Quantitative Lexile score is being used to meet the expectation of the Common Core, which calls for students, as they rise though the grade levels, to be reading increasingly complex texts.


School curriculum administrators have broad and complex jobs.  And there is no way they can know everything about every grade level and about every discipline for which they are responsible. It makes perfect sense then that they might choose to rely on the Lexile Framework as a tool to help them gauge the validity of texts for each grade level. But, the more the Lexile Framework is explored, the more obvious it becomes that it is not a tool that can in any way help schools choose appropriate books.  Nonetheless, more and more often, teachers are being called upon to use Quantitative Lexile scores to defend their literary choices. And teachers are becoming ever more frustrated and marginalized because a debate over a book poses the professional judgment of the teacher against a score generated by a big computer at an influential company with a fancy website.

This tension over where professional judgment best resides—with the professional or with the giant corporation and artificial intelligence—is a struggle shared in many fields, and it is likely that a balance will be someday struck, but it will always be imperative that, at the point of contact between professional and client (between teacher and student or doctor and patient, etc.) any outside tool must best meet the needs of the client—in this case the student.  Lexile fails by this measure.

Are The Lexile Framework Expectations Even Reasonable?

Even if the Lexile Framework scores were cogent to the group’s own mission, are the goals reasonable? 

It may very well be the case, as the framers of the Common Core have implied, that students’ college readiness has been negatively affected because the books that are read in schools have, over time and perhaps with the growth of the YA market, become too easy—not challenging enough to build sophisticated reading skill.  But, as Lexile.com suggests they should be, should high school students all be expected to read books that score near 1300L?  These books include: Great Expectations, Moby Dick, and War and Peace.  While these are brilliant books, should teachers and curriculum administrators consider such book the benchmark (Exemplars in the Common Core) for all high school readers?

Interestingly, a search on Lexile.com for books that range from 1200 to 1400 also reveal many titles like the Best Ghost Stories Ever, Rambin’ Robots, and YA books like Shrimp. The logic behind the ranking is truly mystifying, but according to Lexile, Moby Dick and Shrimp are of equal complexity.


It’s More About the Teaching Than the Titles:

If there is merit to the notion that schools need to improve students’ reading sophistication by teaching harder books, then there is even more merit to the idea that teaching needs to improve.  Too often, English teachers focus mainly on vocabulary memorization, the memorization of characters and events, themes, and asking students to explore how they feel about characters and their choices.  All of those ideas are important, but they do not require, promote, or nurture sophisticated reading—or its development.  Sophisticated reading is promoted by studying and comparing how books work: the forms, structures, techniques, and author choices used to express meaning.  Also, teachers need to specifically teach and model reading skills. 

Simply forcing kids to read harder books will not help them reach Common Core or other educational goals.  In fact, intense study of texts and one’s self as a reader are better promoted with reasonably easier books. Marathon runners do not improve by running twenty-six miles full tilt every day; one will not improve strength by filling the bar with his or her max weight.  Fitness, strength, and reading skills are all improved by way of complex experiences that require a matrix of tasks of various types, intensities, and durations, that taken together promote growth.

Why Do We Teach Reading and Choose Books?

Certainly, this question is far too broad and complex for a thorough examination here, but the answer can be simplified into three very broad categories: skills building, content understanding, and personal growth. All three ideas are critically important yet none is addressed by any of the three dimensions of the Lexile Framework.  But, perhaps that is not the Framework’s role.  After all, if Lexile actually offered a quality tool for choosing books, and if the Common Core (or other curriculum documents) establishes standards, there would exists a pretty good structure from which teachers could work to choose books and design units that address what really matters: increasing skills, expanding understanding, and advancing personal growth.  And, while those three most important goals are not really the focus of Lexile or the Common Core, they are central to choosing what book kids read.



The Final—for now—Analysis:

There is some merit in the Common Core’s general assertion that, in part, some American schools fail to teach challenging enough books, and this failure negatively affects college readiness and American competitiveness. There is also great validity in the Common Core’s assertion that students, as they rise though the grade levels, should be reading increasingly complex texts.

But the Lexile scores do not thoughtfully address those ideas.  They make the situation worse.

Perhaps, Lexile needs to find a way to quantify into scores the Reader-and-Task and Qualitative dimensions of its framework so all three dimensions might combine to give helpful and accurate scores.  Or maybe, the rising awareness of the issue and an ongoing and intelligent discussion about a solution will change the paradigm—maybe we don’t need a tool that quantifies books.

But, if some among us do feel the need to rely on some tool, then we need a better tool, even a minimally good tool—or at least a tool that does more good than harm.

Until that day, however, the Quantitative Lexile scores themselves, while they get some things right, are overall, misleading, harmful, and by any comprehensive assessment, absurd.  They should play no role in educational decision making.


3 comments:

  1. very interesting article...i'm personally opposed to lexiles as they take the joy out of reading...that's what reading's about: joy and learning. I think lexiles do more harm than good.

    ReplyDelete
  2. Lexiles should be taken as a starting point for matching a reader to the text. However, there are many other tools that need to be used such as student interests. The best way to become a better reader is to read. Students need lots of practice. If the books are too easy, then they need books that challenge them or help them grow. I hope that teachers and parents are taking the time to really find out what students want to read and how they can be challenged. This is the most important part!

    ReplyDelete
  3. Couldn't have said it better myself!

    ReplyDelete