If you have attended an educational conference in the past few years, you have heard or become familiar with the term “data driven.” These two words have become a standard mantra in the education reform movement. Books such as “Leverage Leadership” describe the progress that can be made when school leaders strategically use data to inform decision making. As an observer of schools, especially urban schools serving lower income students, I can attest to the positive impact of data and information when used appropriately. But what is not discussed as readily in these books and articles is the tragic outcomes for school teachers, and more importantly students, when data is used incorrectly and strategies, albeit well intentioned, derived from misguided interpretation lead to chaos and set backs for teachers and students.
Private Industry Has Been Data Driven for Decades
As a young MBA graduate student in the 80s, I worked with many large firms mining customer survey and store feedback data for “nuggets” of insight into how to better sell or cajole consumers to buying a product or service. The tools we used were sophisticated for their time, applying various statistical methods to make sense out of the massive amount of numbers reflecting customer perceptions and actions.
Today large firms employ teams of analysts trained in statistical data mining to help them create insights from the massive amount of data they collect on their customers. Data flows to these analysts from web sites, where every click is tracked, from stores where customer purchases can be linked back to their credit cards, from online profiles that we fill out to shop. Each of these sources, when combined and analyzed, help companies identify your gender, race, profession, stage in life and likelihood to purchase a range of products.
To demonstrate the power of these analytics simply go to Amazon and shop around for a specific category of product and on your next shopping adventure with them, they will have matched you to products their analysis (based on your behavior on their site) tells them you might want.
Educators Are Just Learning the Power of Data
Education and educators are just catching up to the private sector in their understanding of the power of data. But education conferences, articles and even district strategic plans are all calling for more “data driven” instruction. What that means seems to be open to interpretation by those given the responsibility to move the district/school in this seemingly new direction.
Data is meaningless without interpretation. In fact data with the wrong interpretation can do more harm than good. A few examples might illustrate. I attended a teacher orientation meeting in one low performing school where a young school leader in her first days in the building was leading a meeting with a group of teachers, many that had been there for years. As this new leader passionately articulated that the school needed drastic change, and that teachers would be held to account for outcomes, she placed a few charts she created up for all to see. One chart was showing the school’s performance on state student performance scores by grade, color coded so that all the grades that did not achieve 70% proficiency in the prior year were highlighted in bright red. Of course the chart was mostly red.
Pointing to the chart, this leader went on to talk about how data would be used to make teachers accountable and that the massive amount of red on the chart was unacceptable. All of this chastising was going on as verses from Leveraged Leadership were read as if at a Sunday sermon. “We will not have another year like this again,” she stated in a manner demeaning to many of the teachers. On the surface, this young, but inexperienced, school leader was using data as she had been told and had read. But it is unfortunate that she was using it incorrectly and in the process de-motivating her best teachers.
Yes she was correct that most grades did not make the proficiency cut point. But what she did not know, because she did not analyze the data correctly or comprehensively, is that the school had just gone through a major year of growth in all but 2 grades. The teachers, in the year prior, recognizing they had to work together to change course, formed groups, used technology and aligned their efforts. During that same year, we introduced them to the use of data for instructional insights, personalized learning plans and differentiation. The results of which delivered solid and notable year to year growth. But not enough to cross the proficiency cut point. You see the students were very far behind grade level at the start of that year.
In essence the teachers were chastised by this school leader for not obtaining a goal that was nearly statistically impossible given the start point for many of the students. Any efforts to explain the near impossible goal to this school leader was met with rolled eyes and comments like, “if the data says we did not do well – then we have to improve.” During the course of the meeting this first week, I could see the faces of the teachers that had achieved near miraculous growth the year before. For them the tone and message helped them understand that their efforts were not understood and more importantly not appreciated. But they also realized that this well intentioned school leader set the bar beyond that which could be achieved in one year. In her mind she was being data driven. To her teachers she was clueless. By the end of the year she was gone. Unfortunately all of the great teachers from the building left too. And the kids, well, they were chastised for not doing better even though their scores had increased significantly. So “data driven” for this example school has probably set their progress back years.
Data Needs Translation to Get to Instructional Insight
In another school, when inquiring about their use of data to advance their teaching and instruction, they proudly pointed to a young girl that sat in a cubicle filled with papers and reports with charts pasted to her walls. They introduced her as the “data girl.” A few conversations with her revealed that she was really concerned about the role given to her by the school leader. It turns out she was the only staff member that knew how to use Microsoft Excel, the popular spreadsheet, so she was given the data role. She also was web savvy enough to get reports from the various software applications the school was using for student math and reading. When asked what her job was, she said “I get reports for them.” Reports that she did not interpret and that many teachers did not know how to read. But the school leader proudly informed me that they were a “data driven” school – even though it was not effectively used in instructional practice.
In another school teachers are required to spend their time in professional development focused on the use of data for classroom instruction. The course used a data model from an technology program that the school does not own. And the course material were so dense and complex that most teachers I spoke with confidentially stated that they could not follow what they were supposed to do. One teacher told me, “I teach math and for the life of me, I can’t figure out how to read these reports to make sense of them. This is taking up too much time, but I can’t say anything because I will look like I am stupid or something.” So this particular teacher spends time out of the classroom doing the required course homework, learning skills that she will never use, analyzing data that she will never encounter and wasting valuable time that could have been better spent perfecting her lesson plans. But here again the school leader proudly claimed they are “data driven” and their teachers are being trained in how to use data.
Are You Really Data Driven?
All this to say, that “data driven,” when correctly implemented is great. But I have seen few examples of an implementation that is really working to enhance classroom instruction and move student performance ahead. In the one school that serves as a best practices model for me, they have centralized their collection and analysis in one team. That team has the requisite skills to assemble, analyze, interpret for instructional practice and coach teachers on the specific needs of their students and ways to improve in the classroom. The coaching discussions are about children rather than charts and tables. The centralized team has developed expertise over time making them more efficient and better able to work with teachers willing to accept recommendations specific to their students and classroom needs.
But there are far too many examples where the term “data driven” has been used because it is popular and not because it is understood. The negative results of this lack of understanding is setting many schools back. Educators need to take a look at the private sector use of data to understand how it should be done. Books like Leveraged Leadership are great because they show the positive impact of a successful implementation. But what these books do not show or provide guidance on is the cultural change required, the understanding needed, the requisite analytic skills and processes schools need to truly become enlightened to data use for instructional insights.
Reading, interpreting and applying data to classroom instruction is a skill that not all teachers will be able to master. In the private sector, companies hire talent with the skills to make the data useful. School leaders that understand this process and can translate it to their schools will be successful and will have found data driven instructional insights to be a powerful tool to use in their school transformation and turn around.
So school leaders beware, data driven can lead you to rapid growth through carefully planned strategies or it can lead you to years of stagnation and decline, as you recover from ill informed policies, actions as a result of the pitfalls and landmines that far too many have already encountered.
