If you had asked me about standardized tests 5 years ago, I would have vehemently dismissed them as the wrong direction for education. While I still resist the Fitbit model of constant quantification of progress and self, this week I heard and read about compelling ways that data can be used to build professional cultures, see and support individuals, and the design of better systems.
One of the sessions at the Carnegie Summit that I attended was a panel on Doctoral programs that embed improvement science into their curriculum, including the program at UCLA with Dr. Louis Gomez, whom we heard from a few weeks ago. He said two things that struck me. First, in working on problems the same way, you build organizational culture. This is echoed in Halverson (2010) “Over time, teacher concerns about teacher evaluation seemed to ease as the principal made a significant time commitment to help teachers make sense of the MAP data reports in terms of math instruction. The Walker principal used MAP data in faculty and staff meetings to create a common vocabulary for Walker teachers to discuss student learning.” (p. 141) To me, this is what data can do for schools when it is approached from a mindset of possibility rather than fear. Further, I heard more than one person at the conference remark that using data was allowing their teachers to have conversations about instruction never possible before. As Halverson quotes of the Malcolm school leaders, “The beauty of data is that we can have these conversations” (p.144). Second, Dr. Gomez stated that improvement leadership is social justice leadership, precisely because it builds common culture focused on improvement for all kids. It changes the system to yield better outcomes rather than treating the symptoms of a system that doesn’t work.
In another panel discussion, focused on the improvement work underway at Cincinnati Children’s Hospital, the presenter Fred Ryckman described a situation where they were able to identify a particular surgeon who had a higher rate of infection for a procedure. Rather than blame or punishment, leadership approached the surgeon with questions about his practice, and how and why certain things were done the way they were. When presented with the data, the surgeon could see indisputably that he needed to change his practice. This was echoed in the story of the teacher intervention at Malcolm based on a data review process that revealed the source of a disproportionate number of referrals. (Halverson 2010) The key points here are that it is about the teaching, not the teacher; it is about learning, not evaluating or punishing; and that the leadership set the tone that the data was used for improvement.
One of the first professors that I interacted with as a freshman in college was a chemical engineering researcher. He said to me that all the interesting questions and problems are at edges, transitions, or boundaries. I would like to add “divides” such as,
Clearly we see this through the data presented by Diamond (2007) to demonstrate the connection between policy and content decisions and decoupling between policy and pedagogical decisions, and it is important to have this sort of evaluative research that documents the intention vs. used divide. Likewise, the NRC report clearly points out the bandwidth and fidelity tradeoffs of large scale and local assessments and the goal of assessments to have comprehensiveness, coherence, and continuity.
But how can you achieve coherence and bridge these divides in a system with such dramatically different practices? To continue on the lessons learned from Cincinnati Children’s Hospital, Ryckman described two ideas that resonated with me. The first was “standardized work, customized to the patient.” Each child is a unique snowflake, but his/her problems are not. When teachers try to meet the needs of every student at each moment of the day, it becomes an insurmountable, herculean task. All teachers use routines to standardize what happens the classroom, and this cognitive offload is essential. Allowing checklists or protocols to remove decision making frees us to focus on what really matters and what humans are best at: attention to the children as individual snowflakes with struggles, needs, and interests. Could standardization of work in fact allow the professionalization of teachers, who are currently exhausted trying to make infinite decisions on their own every moment of the day?
One educator on the panel, David Kauffman of Austin ISD, responded to Ryckman’s take on standardization of work practices with the open question of whether education has sufficient agreement on what actually works? He referred a number of times to Japanese Lesson Study, which reminded me of the Hiebert and Morris article that we read in our technology course last fall. It is the necessary work of educators to build and refine these work practices, and the role of leaders to facilitate the spaces where this can happen. We have models for how to do this, why don’t we?
On this note, the constant refrain from Carnegie is to learn fast and start small, growing through iterative design, paying attention to context and practice. Variation is to be minimized using contextual understanding rather than contextual elimination. I heard many times, “Every system is perfectly designed to achieve exactly the results it gets,” which means there is a true urgency for a systemic redesign, beyond the adoption of a new curriculum or shift in professional development, because we have a system that regularly fails kids.
Data is compelling: it is hard to argue with, it can interrupt long-held patterns, and it makes all kids visible. In his closing keynote, Marshall Ganz said, “It’s that combination of critical eye and hopeful heart that brings change.” Maybe in turning our critical eye, openly and honestly, to the data we are now able to collect, we can redesign the system and change the conversations. This must be done, though, with the hopeful heart that sees the people in the system, hears their stories, solves problems creatively, and builds inclusive cultures. It is a national purpose that must be solved locally, but, as was said in class last week, we have the best tools ever made and have perhaps come through the worst of it, so maybe we’re ready for a little hope and improvement.
Articles referred to in this reflection: