Monthly Archives: November 2021

Research Interest Histories and the Forces that Shape Them

journal reprintsMany of us who read research literature may notice that research is a very “trendy” business. For example, the areas of “personalized learning” and “individualized instruction” became very popular for about 10 years between the late 1990s and the early 2010s. Subsequently this subject has faded away and few papers address the topic. Why?

Generalizing from my personal experience, I doubt that the shift in trend results from loss of interest on the part of the researchers. If one invests 10 years of study on a topic, boredom is unlikely to cause the shift. Perhaps we should explore three other possible influences.

First to come to mind is results. Many research projects focus on very tough nuts to crack. Figuring how to provide a custom curriculum for every student at scale isn’t easy. After chipping away at such a problem for 10 years it may continue to be of interest but an investigator may be daunted by how far in the distance a solution still appears. When significant results are scarce a line of research may be difficult to continue to pursue.

A second force may be the acceleration of technical change. In a field like education, one is concerned not only with the content being taught, the tools used to deliver that content become a subject of study in their own right. As computing devices infiltrated teaching practice, I’ve seen educational researchers struggle to separate the impact of motivation to play with the machine from both learner and investigator focus on the instructional content. Instead of varying one experimental factor at a time, researchers changed the phenomenon being investigated, the experimental methods employed and the situational context from one experiment to the next. No wonder results about the impact of computing in education have been inconclusive and/or unrepeatable.

A third force is inconsistency in funding for research. Funders, like the rest of us, tend to be dazzled by the latest and greatest innovations that are grabbing the headlines. A foundation may invest millions in a line of research such as personalized learning over a period of years and then suddenly switch to a sexier topic. When I follow the individual researchers who were publishing on personalized learning ten years ago, they show up today in searches for data analytics, MOOCs, AI in education, or workforce retraining. Personalization is no longer an effective keyword. Personalization may be an underlying factor in any or all of these new titles but the connection is difficult to track.

What makes interest histories important? By failing to sustain a line of research from conception of a problem to solution, we waste huge amounts of money and slow the impact of research way down. We discourage careers and often fail to recognize brilliant, although preliminary advances. We make it harder for those with an underlying passion for the same interest to find each other and collaborate.

complex sociogramToday we have some wonderful data visualization tools to make interest histories easier to track and thereby encourage valuable collaborations. We can display mesh diagrams showing student-professor relationships as well as co-authorship of papers and patterns of citations. We can see which journals are featuring articles we need to read, what conferences we would benefit from attending and how siloed groups of practitioners can cross-fertilize each other. The migration of keywords that indicate an underlying interest can come to light. Perhaps an even more practical use of such analytical methods would be to alert us when promising research initiatives begin to starve from loss of funding. I doubt that lack of interest is a key force here. It’s time we paid attention to what is driving trends in research.

Leave a Comment

Filed under Educational Computing, Uncategorized

Contemplating Future Scenarios

In the spring of 2020, shortly after it was clear that the COVID-19 pandemic was not just a passing cloud briefly darkening an otherwise sunny global landscape, I was invited to join a group of forward-looking thinkers on the Millennium Project. Over the next 9 months we produced three “stories” about what the near future might look like as the novel virus rampaged through the world’s populations. Our purpose was not to blame or take political action. Rather, we sought to help medical, social services, NGO, and government decision makers prepare for an unpredictable future. (You can see a video report on this work here.)

This last week, Knowledge Management World published a review of the Millennium Project’s report. It begins:

“The American Red Cross is no stranger to crises and disasters. When things go bad, it is often among the first organizations on the front lines lending assistance.

Yet, last year, during the height of the pandemic, Michael Kleeman of the American Red Cross kept thinking a different approach was needed. In a recent report, he is quoted as reflecting at the time, “We’re so focused on what we have to do today to respond to the COVID pandemic that we don’t have time to think twelve-to-eighteen months down the road. But someone has to.” That’s when he turned to The Millennium Project for help.

Jerome Glenn, CEO of The Millennium Project, and one of the world’s top strategists, had to step back for a moment, since the vast majority of his body of work focuses on the long term. In fact, Paul Saffo, his colleague at The Millennium Project, studies time horizons spanning tens of thousands to billions of years. Never ones to pass up an opportunity, especially given the turmoil the world was (and still is) going through, they accepted the challenge.” Click here to read the rest of the KM World article

I posted these notes in the comments section following the article:

“Nice review, thank you Art.

Several points you made are worth emphasizing. First, future scenarios are not predictions. They are intended to bracket possible outcomes so organizations can prepare for a wide spectrum of events.

Second, the “citation ring”, echo chamber, or confirmation bias effect makes ferreting out reliable data difficult for experts and even more challenging for ordinary news consumers. On one hand, receiving the same message from a variety of researchers may indicate that the community of practice has tested a concept thoroughly and reached a convergent conclusion we can rely. On the other hand, multiple secondary reports may result from many writers picking up on the same, possibly fake, extremely preliminary, or mistaken published research. It take considerable sophistication to discern the difference.

A third issue you highlighted is data that did not “distinguish between purely COVID related deaths and deaths involving comorbidities.” This is an example of the logical fallacy, “post hoc ergo propter hoc” that concludes that if B follows A then A caused B. Under the pressure of so many hospitalizations of elderly and frail people, it was impossible to take the time to sort out whether an individual patient with COVID actually died of COVID-induced respiratory failure or something else.

Finally, it’s worth noting that the Red Cross is not the only response agency that could benefit from scenario studies similar to the one we did for COVID-19. Any time there is a large-scale event that plays out over a multi-year period, agency staff tend to focus on methods drawn from their recent training and experience. And, they talk most often to their known partners and colleagues which can enhance the echo chamber effect. But whether the event is a wildfire, a war, slow but inexorable climate change, or the impact of a new, rapidly-adopted technology, the uniqueness of the event calls for more robust and innovative responses. Scenario studies can both drive innovation and confirm the effectiveness of well-known practices. The world would benefit from more Millennium Project style thinking.”

My contribution to the Millennium Projects COVID-19 report was minimal and I was honored to be able to participate. Public health is not my area of expertise although I have been active in disaster preparedness and recovery in my home county, Sonoma, California. But, by joining this working group, I learned a tremendous amount about how to conduct future studies. Learning and education are my wheelhouse and the view from here is dismal.

Over the next two years I would dearly love to stage a Millennium Project study of the future of formal, non-formal and informal education. It’s not too soon to start taking names of those of you who would be willing to participate in the Real-Time Delphi process.

Leave a Comment

by | November 7, 2021 · 12:46 pm