Confirmation of the Obvious

Is the phrase one of my science ed profs used to describe most educational research. This is not as trivial as it might look at first pass. As I've commented before (here for example), we humans tend to see all sorts of patterns, many of which aren't "real." In other words, we think some relationships or recurrent events are "obvious," but careful gathering of data and analysis often shows that these patterns aren't actually there. So in a situation where putting together an analysis that could show a causal relationship is extremely difficult and costly, the logical place to start is to make sure our assumptions, the things that are "obvious," are actually accurate. Education, particularly in a classroom setting, is one such situation. Furthermore, since the vast majority of citizens in industrialized nations have spent a great deal of time in the classroom, the assumptions of what is "obviously" true are deeply ingrained, widespread and numerous. After all, we all have extensive experience in that setting, and therefore feel our opinions have validity due to that experience. Even though most of us have never bothered to gather empirical evidence and test those assumptions and opinions.

This can be somewhat infuriating to those who do try to gather such data and test those opinions.

It can also lead to some serious "Well, duh!" reactions to reports from educational research. For example, this article from The Telegraph is summarized "Reading bedtime stories to children could help to improve their vocabulary, new research suggests." "Well, duh!" was certainly my reaction. Actually, the article went in a different direction than I was expecting... it's more about retaining memories of words learned just before a good sleep (as opposed to several waking hours later). My assumption was that it was going to be a study of kids who were read bedtime stories versus kids who were not. Which does seem like kind of a dumb study, doesn't it? On the other hand, obvious conclusions have often been shown to be incorrect; if that study hasn't been done, it should be. Furthermore, the assumption that "Reading bedtime stories to children could help to improve their vocabulary," while an obvious projection from the study described in the article, is not necessarily accurate. If the kids are drowsy, for example, and nodding off, their retention might not be as good as if they were read to in the midafternoon when they were alert.

Then there's logistics and ethics. It is reasonable to assume that reading to kids is a positive benefit. Yet to establish causality, you would need to randomly assign them to treatment (reading) and control (non-reading) groups. In essence, you might be put in the position of depriving some students of a positive benefit that they would have otherwise received. This is an ethical quandry, which can be resolved through a variety of means, though none totally satisfactory. Payments for participation, offers of remedial followup, resources (books) given after the study, and so on, are options I see off the top of my head for this hypothetical study. Ultimately, the human subjects committee at the institution doing the research would be responsible for deciding whether the palliative actions suggested by the reasearchers would be sufficient to overcome and balance whatever ill effects might arise from participating in the study.

Oh yeah, and there's the whole issue of informed consent: parents/guardians would have to be filled in on what exactly the risks would be (to privacy, to the development of their children, to their health- inconsequential in this case- and so on), and what the study was trying to achieve. The human subjects committee would have to sign off on this as well. The consent form would also spell out the compensation offered to participate in the study, and thus takes the form of a contract: a legally binding agreement between the parties.

In short, any type of educational research ends up being very complicated and involved, and much, much more expensive than most people might guess. It's typically less of a big deal with research on adults who can sign for themselves (this would include most college students), simply because there's fewer parties involved, than it is with kids whose parents/guardians are (rightfully) more suspicious regarding the researchers, and protective of their kids than they might be for their own sakes.

So while confirming the obvious is often seen as a "Well, duh!" sort of achievement, and is frequently held up both in the media and by demagogues as wasteful spending and a squandering of resources, it is neither unimportant nor cheap and easy. The same professor who characterized much of sci ed research with the title of this post also frequently commented, "It's not rocket science!" To which I often responded, "No it's not. In rocket science they at least know what the relevant variables are. In education, we clearly don't, and I'm not even convinced we're getting closer."