Why Data-Informed Trumps Data-Driven

Plot, characters, and setting, the three primary qualities of a story; but imagine a story with only a setting. Well, you no longer have a story, we call that a painting. Data-driven decision-making runs the risk of turning one element of effective education into the primary element and in the process, turning beautiful stories of educational success into static pictures of moments in time.

Data is the setting in our story of education. Within this setting, the characters (students, teachers, parents, etc.) create the plot in their daily interactions. To call for an end to “data-driven decision-making” is sure to raise a few eyebrows. Data is the constant, the solid ground; data takes the guesswork out of what we do. However, focusing on the data to the exclusion of all other elements of our story does not advance the cause of effective education for our students.


That is why I believe in “data-informed decision-making.” Ignoring data would be foolish, but data alone does not tell a story. I am reminded of a quote from Jack Nicholson’s character, Melvin Udell, in the movie “As Good As It Gets.” While discussing his relationship problems with an acquaintance, Udell interrupts—“I’m drowning here, and you’re describing the water.” For the majority of the last two decades, the drive for greater accountability through standardized testing has done a super job of describing the water without offering any tangible ways to provide a lifeboat.

“Data-driven decision-making” has become a tool for power. Politicians use data-driven to point out the flaws in education instead of directing efforts toward areas needing improvement. Edupreneurs use data-driven to sell the next great solution to our education woes. Too often, the data drives decisions that have already been made.

This is how data-informed works:

1) All data is considered- student attendance, variables in the life of the teacher and/or students affecting outcomes, problems with the assessment, instructional methods, past performance of teacher and student.  In other words, a data set should not be analyzed relying on numbers alone. Testing does not happen in controlled settings, therefore we should not mistakenly treat results as dependent variables.

2) Instructional decisions are made in response to meaningful conclusions drawn from the data- just like a few high performers do not indicate good instruction, a few low performers do not indicate poor instruction. A few years ago our high school was on a fixed schedule, and every Friday afternoon I taught the same class of seniors ready to start the weekend.  On top of that, our school gave seniors a ten minute early release every Friday.  They also found less success than their peers on the AP exam.  Data-driven decision-making would increase my anxiety by trying to figure out where I was going wrong with these students.  Data-informed allows us to look at the big picture and balance the benefits of a senior release with student achievement.  Before anxiety leads to poor decisions, all of the potential reasons for low achievement need to be addressed.

3) Teacher’s need to honestly evaluate the data they collect- we cannot, nor should we avoid accountability. The response of “I just know” to the question of how much your students are learning is not good enough. I continually make an effort to compare and evaluate how well my grading structure compares to student performance on end-of-year testing. I’m not satisfied with where I am, but I have used the data collected to inform decisions about the changes made over the years.

4) Data does not need to be punitive to be informative- Dan Pink gives an excellent summary of the 21st century understanding of motivational theory.  The use of standardized student assessment data to evaluate the performance of schools and tying these results to consequences run counter to this theory. Further, the national movement toward “value-added” or “growth models” which tie these results to individual teachers further undermines the quality of education. This point could stand further explanation, but in the interest of brevity, I will save that explanation for a later time.

“Data-informed decision-making” is a way of holistically approaching education.  We realize that students are people and not data points.  Unlike the pigeon in a Skinner box, the tools to isolate specific behaviors and tie them to discrete consequences are not at our disposal. Until that becomes possible (and I'm not even sure it would be desireable) we must continue to rely on human judgment, reason, and ultimately decision-making.

If nothing else, the paradigm of data-informed instills the idea of collaboratively looking at the facts and deciding the best course of action for moving forward as opposed to the data-driven paradigm of top-down one-size-fits-all approaches to education reform.