Reaction 11: Data-Driven Instructional Systems

Anarchist Org Chart: https://juliekallio.com/wp-content/uploads/2014/11/a8423-anarchistorgchart.gif

Reading this week:

Halverson, R.; Grigg, J.; Prichett, R.; Thomas, C. (2007). The New Instructional Leadership: Creating Data-Driven Instructional Systems in School. Journal of School Leadership. 17: 159-193.

Thorn, C.A. (2001, November 19). Knowledge Management for Educational Information Systems: What Is the State of the Field?. Education Policy Analysis Archives. 9(47). Retrieved September 5, 2007 from http://epaa.asu.edu/epaa/v9n47/.

Unit of analysis. That seemed to be the thing that kept popping out at me this week. This was stated directly by Thorn (2001) that if the student is the unit of interest, then the data gathered should be attributes about the student. Student Information Systems, however, tend to be designed to produce reports for district-level analysis, not for the classroom. Halverson et al. (2007) found a mismatch or inoperability of data in the district’s high-tech data storage as opposed to the local collection and storage of low-tech data. The logical goal of the proposed data-driven instructional system is thus to link the results of summative data with formative information systems that teachers can use to improve instruction. The goal of practical measurement, as proposed by Yeager et al. (2013) is that “educators need data closely linked to specific work processes and change ideas being introduced in a particular context” (p.12).

In the past, I have associated data-driven decision making as context-blind work whose sole purpose was to improve standardized test scores. The readings this week as well as the networked improvement communities (Bryk et al., 2010) from a few weeks ago has given me a different perspective on what it means to use data to inform instruction, design, and communities. Continue reading “Reaction 11: Data-Driven Instructional Systems” →

Embeddedness: Implications for Leaders & Researchers

Prompt:

Post a paragraph or two addressing embeddedness in your settings. Specifically, how, where, and to what extent are leaders embedded throughout your school/organizational community? Are there certain areas where embedded insights are needed, but missing? What are the regular routines and practices that operationalize/sustain this embeddedness?

Second, please project how you’ll embed yourself within your school community when you are a leader? Where will you spend time? What types of routines will you utilize to ensure that you are sufficiently “close” to all stakeholders?

Ignatius, David. (2010). The Dangers of Embedded Journalism, in War and Politics. The Washington Post. May 2, 2010. http://www.washingtonpost.com/wp-dyn/content/article/2010/04/30/AR2010043001100.html

Act 1: Weeds of Discontent, from The View From In Here. This American Life. July 26, 2013. http://www.thisamericanlife.org/radio-archives/episode/501/the-view-from-in-here

I could speak to embeddedness in the school that I previously taught at, but I think I’ll attempt this from the standpoint of being a researcher. The arguments about embedded journalists and the price of intimately knowing one side but not the other, reminded me of the questions about rigor in qualitative work. Qualitative reports can become so contextualized that you miss the perspectives of other stakeholders, such as students, parents, community, boards, etc. You have to limit the scope of a case study somehow, but you should identify the ecology of the system and make conscious decisions about how it is limited.

For most of the research that I’ve read this fall, very little student voice is present, perhaps ironically given that their performance is usually the outcome being measured. The This American Life interview that gives the perspective of the inmate reminded me how little we asked students their side. Interventions are done to them and their outcomes are measured, without the consideration that they are an independent actor. Some of the most powerful pieces I can remember reading (School Girls, by Peggy Orenstein; Doing School, by Denise Pope Clark; and more recently the veteran teacher turned learning coach who blogged about shadowing students, posted on Grant Wiggins’ blog) were researchers who embedded with students, understanding their perspective in a way that would not have been clear through interviews or surveys.

Here are three thoughts on the challenges of embeddedness in research and school leadership: First, it can be challenging for one person to have strong relationships with all stakeholders. Some people are naturally more comfortable interacting with students or parents or teachers or board members. Thus a research team and advisor can help identify scope and perspective. Second, both researchers and school leaders could routinize being closer to all stakeholders by establishing (and keeping) a regular time to meet or be in the same space or the other’s space. We often think we understand someone’s perspective by their reports, but physically being there is important and draws out observations and questions that wouldn’t have come through otherwise. Finally, I do not think researchers or school leaders should be non-ideological reporters. While they need to maintain an openness and respect for differences of opinion, I think a leader needs to ideological stand (such as the importance of technology integration or cross-curricular coordination) on issues in order to lead towards an organizational vision.

Reaction 10: Questions of behaviorism and identity in ed-tech

Reading this week:

Bailey, J., Carter, S. C., Schneider, C., & Vander Ark, T. (2012). Data Backpacks: Digital learning now!.

U.S. Department of Education. (2012). Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief.

Hill, P. T. (n.d.). Finance in the Digital-Learning Era. Creating Sound Policy for Digital Learning: A Working Paper Series from the Thomas B. Fordham Institute.

What problems are we trying to solve?

“The current way student records and transcripts are managed is insufficient to meet the evolving needs of teachers, students, and parents” (Bailey et al.).
“Our system doesn’t fund schools, and certainly doesn’t fund students” (Hill).
Policymakers and administrators need to understand “how analytics and data mining have been—and can be—applied for educational improvement” (Data Mining and Learning Analytics).

What vision of education are we working towards?

“Each student’s account would, in a sense, constitute a ‘backpack’ of funding that the student would carry with her to any eligible school or instructional programs in which she enrolls. The contents of the backpack would be flexible dollars, not coupons whose use is restricted to a particular course or service” (Hill).
“The most important [adantage] is individualization and rapid adaptation to what a student is learning, leading to the possibility of more rapid and consistent student growth” (Bailey et al.).
“Making visible students’ learning and assessment activities opens up the possibility for students to develop skills in monitoring their own learning and to see directly how their effort improves their success” (Data Mining and Learning Analytics).
“A study contrasting the performance of students randomly assigned to the OLI statistics course with those in conventional classroom instruction found that the former achieved better learning outcomes in half the time (Lovett, Meyer, and Thille 2008)” (Data Mining and Learning Analytics).

What limitations might there be or what questions should we ask?

Who are the designers of the platforms and how are their biases in them? This is not a new question for educational materials, but still one worth considering.
“What children learn would then depend on the quality of their parents’ choices” (Hill). How would kids fare whose parents were not able, for various reasons, to keep up with choices?
Programs could be prevented from use “only after some children had been demonstrably hurt by them” (Hill).
“All such investments are meant to benefit children but they also benefit private parties—the teachers who use new skills to make higher salaries, the vendors who sell professional devel-opment services, etc.” (Bailey et al.).
“Unlike educational data mining, learning analytics generally does not emphasize reducing learning into components but instead seeks to understand entire systems and to support human decision making” (Data Mining and Learning Analytics).

How might the changes in personalization, finance, and data foster a participatory culture of learning?

I recently watched and blogged about Audrey Watters’ keynote from the C-Alt conference, titled Ed-Tech, Frankenstein’s Monsters, and Teacher Machines. She talked about how ed-tech is full of behaviorist technologies, citing notifications, nudges, gamification, which is built in to many of the platforms written about in this week’s readings. She tells the story of Alan Turing asking if a computer could think, but that he really meant asking whether a computer could exhibit behaviors that could fool a human into thinking it was human “enough.” I see learning more from a constructivist point of view, where students build their knowledge on what they already know, as we read about in “How People Learn.” Are behaviorism and constructivism at odds in a learning ecosystem or could they coexist for building different skills? I see a participatory culture as being fundamentally constructivist, where learners’ own interests and experiences drive affiliations and personal expressions. Does a system based on behavioralism preclude authentic participation?

The idea of authenticity also brought me to questions of identity. This infographic compares “team transparency,” led by Mark Zuckerberg, and “team anonymity,” led by Christopher “Moot” Poole, founder of 4chan. How does having your identity (or a notion of a fixed identity) as a learner tracked from birth affect a person’s own identity formation? How does the platform reward or require a certain identity? (Kimmons, 2014) In working with middle schoolers, I saw the importance of kids trying on different identities as a way to understand their world. A learner profile that follows them year to year and school to school may smooth learning opportunties but it may also lock kids into always being “that kid.” Furthermore, if we have take a post-modern understanding of identity as one that is never fixed but is created in relation to others and situations, what impact (good or bad) will this big data mediated, educational enviroment mean? How will the designers be training certain identities? How will this impact divergent thinking? Will these redesigns mean more creative thinkers? These are geniune questions, not meant to be particularly optimistic or pessimistic, though I think people will see them in one way or the other based on their own inclinations.

Kimmons, R. (2014). Social Networking Sites, Literacy, and the Authentic Identity Problem. TechTrends, 58(2), 93–98.

Networked Scholars Course: Week 3 Reflection

Notes from Networked Scholars course this week. My thoughts at the end…

Zeynep Tufekci‘s article, Social Media is a Conversation, Not a Press Release: How @nytkeller and @emmagkeller flunk understanding @adamslisa:

How should it have been written? In one word: When trying to understand social media presence, dear journalists, don’t “peruse.” Engage. Because that’s how the medium works.
To understand these basic points Lisa Adams made, repeatedly, over the years, you’d have to be engaging her feed, and not just reading snippets of it and projecting it onto one’s own anxieties or issues.
If anything, social media has helped move us to a world in which people are no longer passive, silent subjects of journalists (or academics or other gatekeepers of public discourse). We can no longer speak of people at them, without them talking back of their own experience, and articulating their own narrative in their own terms. And how to deal with that reality, not whether cancer patients should tweet that much, is the real ethical question before us.

Sherry Turkle‘s TED talk about Alone Together:

We want to be in spaces together, but we only want to pay attention to the bits that interest us. I definitely see this is graduate school classes. In texting, we get to edit, delete, and retouch our voice, but real relationships are awkward and messy. A little later, “I share there for I am.” The more we connect the more isolated we are because we do not cultivate solitude. We need to teach our children to be alone.

“People who make the most of their lives on the screen, come to it in a spirit of self-reflection.”

Develop a self-aware relationship with our devices and others: make room for solitude.

Audrey Watters‘ keynote for C-Alt conference, Ed-Tech, Frankenstein’s Monsters, and Teacher Machines: Continue reading “Networked Scholars Course: Week 3 Reflection” →

Reaction 8: Computational Thinking

Articles this week:

Grover, S.; Pea, R. (2013). Computational Thinking in K-12: A Review of the State of the Field. Educational Researcher. 42(38): 38-43.

Berland, M., & Lee, V. R. (2011). Collaborative strategic board games as a site for distributed computational thinking. International Journal of Game-Based Learning, 1(2), 65-81.

Resnick, M. (2012). Reviving Papert’s Dream. Educational Technology. 52(4): 42-46.

Since I’m familiar with computational thinking, I also read Berland & Lee’s article about students playing the board game Pandemic, and I went back to a Scratch project I haven’t worked on in awhile and attempted a little debugging.

My favorite quote was this: “None of the groups understood the rules by reading through the guidebooks without attempting to play through the rules” (Berland & Lee, 2011). The idea of “playing through the rules,” I realized, is how I have approached learning with students because it’s how I approach my own learning. If it’s science, I need to see or do something. If it’s Twitter, sign me up and write a few tweets. If it’s Tinkercad, drag and drop a few objects, then ask why or how it works. I learn rules by interacting with them, not by thinking about them.

This low barrier to entry (sign up and start) is the idea of “low floor, high ceiling,” which has been “one of the guiding principles for the creation of programming environments for children … since the days of LOGO.” (Grover & Pea, 2013) Whether it’s Tinkercad or Pixel art, programs or suites of programs have embraced an easy entrance and seemingly unlimited complexity. (As an interesting aside, this might be an interesting antidote to what Sennett negatively describes as our modern passion for consuming incredibly powerful devices that we never use to their full potential. We might Continue reading “Reaction 8: Computational Thinking” →

Reading Notes: Ch. 3 & 4 of Social Network Theory

Social Network Graph. Retrieved from http://www.ifets.info/journals/6_2/6.html

The next two chapters in the book, Social Network Theory, edited by Alan Daly, are examples of social network analysis studies in elementary schools. Chapter 3 examines the dynamics of and changes in tie formation in a districtwide mathematics reform. Chapter 4 examines network characteristics before and after school-based change efforts aimed at implementing the Literacy Collaborative model of instruction. Going back to the typology described in chapter 2, the first study is an example of Type 2 and 4 because the researchers examine at dyad and network levels and make conclusions about the consequences for ties as a result of the change efforts. The second, on the other hand, is both Type 4 and 6, comparing engagement in coaching (an indicator of implementation of the LC model) with network density (network level variable), coach centrality and isolation (node level variable), and school climate. Both drew on data from larger, longitudinal studies.

One brief observation as to my general understanding of social networks is that these methods of analysis gives us a different way to understand our reality and experiences. It’s not about searching for the “perfect” school-based social network, because these are dynamic systems. Continue reading “Reading Notes: Ch. 3 & 4 of Social Network Theory” →

Reaction 7: Blended Learning

Rocketship School, Retrieved from: http://www.onlineschools.com/imagesvr_ce/9968/rocket%20ed_feat.jpg

Articles this week:

Next Generation Learning. (2010). Report: Bill and Melinda Gates Foundation.

Jacob, A. (2011). Benefits and Barriers to the Hybridization of Schools. Journal of Educational Policy, Planning, and Administration. 1(1): 61-82.

Obviously I am generally in favor of technology integration and am usually one who embraces change. In principle, I like the idea of blended learning as a way to provide students with a chance to test out of material they already know, go further ahead that their own pace, and customize content. I like the idea that this method could provide all students the opportunity to develop their skills. I will admit, however, that I objected to two things in this picture of technology integration by Jacob (2011) and in the Next Generation Learning Report (2010). The first is talking about schools in terms of efficiencies to reduce costs. The phrase that particularly stung was “substituting specialized software for expensive college-trained workers.” What happens in a place where children are “managed” by technicians? Isn’t that Sir Ken Robinson’s critique of the model we already have? Maybe one nostalgic ideal that I cling to is the foundational and formative relationship between student and teacher. I suppose better “products” may roll off the Khan Academy line, but are these going to be reflective, creative, self-aware 21st century citizens? When will they learn how to think and organize their own learning instead of following a playlist? I see the opportunity of blended learning as a way to free up time for a teacher so that he/she can spend more time on checking in with students, 1-on-1 instruction, or coaching student-led projects.

The second sticky point for me is that, even though many researchers and educators say that test scores are not an adequate the measure of a child’s learning, in the absence of other measures, this “achievement” number is still used. Jacob uses “value-added anaylsis of test scores” for the analysis of Carpe Diem Collegiate. These personalized learning programs, from what I’ve seen, are almost all math or reading, so it makes sense that they would perform better on math and reading tests. I remember teaching a science lesson that asked my students to do something very basic, like scale drawings. They were adamant they’d never seen anything like it, so I got their math text book and looked it up. Sure enough, they’d studied a whole chapter about it, but they couldn’t apply it outside of the textbook chapter. Are we assuming that students will be able to apply their Khan Academy math and or Read3000 outside of the program (and not just on another standardized test)?

These two points make me wonder what it means to be a learner in these school environments. In my opinion, blended learning is something that comes after, or at least in complement with, student-driven open inquiry within a learning community. Platforms such as Khan Academy or Read3000, that are effective at transferring skills, are part of a student’s toolkit, and I think they hold a lot of potential for freeing up teachers from repetition of the basics or for providing time/location flexibility in learning. I worry, though, about lack of continuity, whether this is the rapid change in platforms or the high turnover of teachers in charter schools, and the related erosion of a community of practice, both for teachers and students. Schools guided by efficiencies and products reminds me of our conversation about consultants: charter school organizations or educational entrepreneurs come in, provide some new software, see a bump in test scores, and leave. This is, in my opinion, a short-sighted vision of 21st century education that uses incredibly powerful and creative tools for basic skills.

Reaction 6: Teaching & Learning Environments

Reading this week:

Bransford et al. (1999). How People Learn, Chapter 6: Learning Environments

Jenkins et al. (2007). Confronting the Challenges of Participatory Culture: Media Education for the 21st Century

Hattie, John. (2003). Teachers Make a Difference: What is the research evidence?

Reading Hattie’s article about expert vs. experienced teachers went straight to my heart. It made me want to be back in my 7th graders in science class, getting a chance to do many things the same and many more things differently. Several of his points made me appreciate the individuals and the communities from whom I learned to teach. Though both my parents were teachers, I went to school in the same building my mother taught in, so I usually knew the students in her class and got to see her as a teacher. To me, she epitomizes the expert teacher who respected students “as learners and people, and demonstrate[d] care and commitment for them.” (Hattie) I’ll never forget asking her how she could teach 4th graders who “didn’t really know much.” She replied that you just start by asking them questions, and you’ll find out they know a lot. To this day, she gets invited to college graduations for kids she taught in elementary school! The communities that I learned and taught in both had a positive impact on how I saw myself as a teacher. Hattie and Bransford et al. basically say the same thing about classroom and school communities: “learning seems to be enhanced by social norms that value the search for understanding and allow students (and teachers) the freedom to make mistakes in order to learn” (Bransford et al., p.133); “[Expert teachers] build climates where error is welcomed, where student questioning is high, where engagement is the norm, and where students can gain reputations as effective learners” (Hattie). These were the teachers I taught with. One of my colleagues shared with me that when a student answers a question wrong or asks a question that reveals a misunderstanding, he replies with, “Thank you for your answer/question – I’m so glad you said that because that helps me know what we need to do next.”

I think assessment is something that teachers in traditional classrooms (this was certainly true of me) struggle with, whether it’s doing enough formative relative to summative, using it to inform their own practice, or actually assessing “higher level thinking and deep understanding.” (Bransford et al.) The result, unfortunately, is assessments that seem more like judgements about students themselves rather than an accounting of what they learned (or didn’t). Participatory cultures, where there is “strong support for creating and sharing one’s creation” (Jenkins et al.), are really good at assessment, because members are constantly sifting, sampling, and sharing. I see this as a hope of the maker movement in schools, where the artifact being criticized is literally and physically separate from the creator. This allows both the creator and assessor to examine it, and the creator almost becomes a witness to the assessment rather than the target.

It would be interesting to apply Hattie’s characterization of expert teachers to participatory cultures to see what characteristics of an individual teacher might also be present and available, though in a distributed way, to learners in an affinity space. It would be ironic if networks of people, with dynamic structures and teachers, designed for innovation and creativity, was the “idiot-proof” solution.