Nature of Science, Pseudoscience, Science Practices, Teaching Methods, Uncategorized

Acupuncture Study as a Cure for Pseudoscientific Thinking

This is a new version of an earlier post from January 2015.


First, A Basic Lesson in Pseudoscientific Thinking

As a science teacher, a major focus in my courses is helping students design carefully controlled experiments that give them answers they can trust with a reasonable level of confidence. Regardless of how hard they work at controlled design, the data they generate are often messy (studying life is inherently messy) and require more than the typical descriptive statistics of mean, median, and standard deviation (among others) to make sense of their outcomes. Yet, even with good training in scientific methodology, my students still struggle to recognize the difference between a good scientific study and a pseudoscientific one—a study that pretends to be good science but isn’t.

A study and resulting claim can qualify as pseudoscience in many ways. In its most simple form, pseudoscience includes experiments that fail to control variables other than the manipulated variable (the independent variable) that could influence the response variable (the dependent variable). Thus, a claim made about a result that does not consider the potential effects of the uncontrolled variables is a pseudoscientific claim.

This is an easy mistake to make.

For example, each year my upper level biology students are run experiments that respond to the prompt: Test the effect of an abiotic or biotic factor on seed germination or plant growth. Each year at least one student proposes the hypothesis that glucose is an essential nutrient in seed germination and predicts that increasing concentrations of glucose will increase germination rate. When students run the experiment with their glucose treatments alongside a treatment of distilled water, they find the opposite to be true and conclude that glucose does not promote germination but instead suppresses it, regardless of whether the presence of glucose during germination has any effect at all. This is a pseudoscientific claim—the students fail to control for the fact that while increasing the concentration of a nutrient, they are also increasing the solute concentration of the solution in which their seeds are soaking. Their seeds are being exposed, not just to more glucose, but also to increasingly hypertonic solutions. Their design requires an additional treatment to rule out or exclude the solute concentration variable. They could, for example, run the same test on germination but use a non-nutritive solute like sodium chloride as the variable of increasing solute concentration and increasing hypertonicity.

Exclusion Reasoning

Knowing to add the solute concentration variable to their seed germination experiments requires students to employ exclusion reasoning. Exclusion reasoning for most students, and people in general, is a major obstacle, and failing to include it is a logical flaw. In a 1993 paper titled, “Science as argument: Implications for teaching and learning scientific thinking,” Deanna Kuhn (Kuhn 1993) explains this problem:

Exclusion is essential to effective scientific reasoning because it allows one to eliminate factors from consideration. Exclusion (inferring the absence of a causal effect) poses more of a challenge than inclusion (inferring the presence of a causal relationship) for several reasons. First, and most fundamentally, is the domination of affirmation over negation—the presence of something is more salient than its absence and, for this reason, both scientific and lay theories pertain more often to the presence than the absence of causal relations. Second, the belief that a factor is irrelevant often leads subjects to ignore it in their investigations. In so doing, they forego the possibility of encountering disconfirming evidence and, hence, ever revising this belief.

Foregoing and even purposefully avoiding “the possibility of encountering disconfirming evidence” is doing pseudoscience.

The Future Leaders We are Training Must be Critical Thinkers

Many of the students in my Biology courses are interested in careers in the health sciences, and many of those students are interested in medical science. Pseudoscientific claims are common in studies involving humans and their health and well-being. Recently vox.com published an article titled, “The one chart you need to understand any health study,” and provided this helpful graphic:

Study Types in Health Science

All of the study types in the graphic above are at the risk of making pseudoscientific claims by leaving out necessary components of scientific methodology, but the pseudoscientific risk increases with decreasing strength of conclusions. Pseudoscientific claims are also nearly guaranteed if a study, regardless of where it lands on the list above, fails at exclusionary thinking.

Acupuncture and CAM can Help Cure Pseudoscientific Thinking

In a course on the science of biology, studying about acupuncture in particular and complementary and alternative medicine (CAM) in general as evidence-based medical science has the potential to help cure students of pseudoscientific thinking because both fail at exclusionary thinking.

The evidence is clear that acupuncture outperforms no treatment in many cases (Witt et al. 2006). No contest, really. The anecdotal stories told by people who have experienced acupuncture are salient and persuasive and, if generalizable, can solve your depression, cure your back pain, help you get pregnant, and improve your golf game. But anecdotes, while compelling stories, are not admissible in science.

The evidence is also clear that the effectiveness of acupuncture as a real, science-based medical treatment ends at anecdotes and the acupuncture versus no treatment “studies.” Appealing to stories and only comparing a treatment to no treatment are pseudoscience in the arena of research on humans, especially pain research.

So, what happens when we study acupuncture using the methodology of real and rigorous science?

We have done this.

When acupuncture is compared with sham acupuncture where patients think they are getting real acupuncture but aren’t (they’re getting poked and prodded at non-acupuncture sites, whether or not there is penetration with the needle) there is no statistically detectable difference between the two (Madsen et al. 2009). Indeed, in one study even when a toothpick was used instead of an acupuncture needle there was no difference in perceived benefit (Cherkin et al. 2009). Whenever a sham (the fictitious, artificial treatment) in any medical experiment is indistinguishable from an intended treatment, it is called the placebo effect.

The placebo effect is a real, measurable biological phenomenon and claims and studies that fail to acknowledge or control for the placebo effect are pseudoscience. For example, the placebo effect has been shown to lower respiration rate (Benedetti et al. 1998), lower blood pressure (Pollo et al. 2003), and can even improve motor performance in Parkinson’s patients (Goetz et al. 2000).

However, the placebo effect also confounds medical science, especially when scientists try to make sense of the results of drug trials that are being developed for pain mitigation. For example, researchers have found that anywhere from 27% to as high as 56% of subjects in pain studies responded to placebo treatment when compared to no-treatment controls (Price et al. 2008). The ability to predict which individuals are more or less likely to respond to placebo would be a critical tool for pain studies, but has been impossible… until recently.

In September of last year, the journal Science reported that specific genetic markers are being discovered in patients that respond more strongly than others to placebo treatments like sugar pills (Hall and Kaptchuk 2013, Servick 2014). If the potential for placebo can be reduced in a drug trial, smaller patient sample sizes can be used and drug trial costs will go down.

How to reduce a drug trial

But this discovery doesn’t fix the pseudoscientific problem for acupuncture. Nor does it fix another problem with acupuncture: the nocebo effect.

The nocebo effect is when a treatment actually does harm rather than good, and alarmingly, there is also no difference in the nocebo effect of real acupuncture compared to sham (Koog et al. 2014). Reports of increased pain or patients dropping out of an acupuncture trial because of unbearable discomfort are common, as are reports of injury from acupuncture treatment. Less common is death, yet there have been five confirmed cases in the scientific literature (Ernst et al. 2011).

One response to the placebo dilemma by the CAM community is the argument that “acupuncture cannot be studied using randomized controlled trials,” because CAM, including acupuncture, treats systems that are just too complex for randomized and controlled trials to be an appropriate test of their effectiveness (Langevin et al. 2011). If this is true, then acupuncture cannot yet claim to be medical science at all because it is impossible to do any fair empirical tests of CAM’s hypotheses. The best bet for acupuncture, and CAM in general, to become a legitimate, science- based approach is to climb the ladder of strength for health science studies.

Cosmologist Carl Sagan made famous the saying, “Extraordinary claims require extraordinary evidence,” and acupuncture continues to make extraordinary claims with little, if any, evidence. Indeed, acupuncture can cure your Bursitis, Ulcers, Laryngitis, Leukopenia,Shingles, Hives, Infertility, and Tendonitis. But, if these claims aren’t extraordinary enough, acupuncturists have claimed it to be an alternative to anesthesia during surgery.

I did a Web of Science search for studies that show how acupuncture can in fact be an alternative to anesthesia during surgery. I used the search terms “acupuncture” “surgery” “anesthesia” and “alternative” and got 33 returns. Twelve of the 33 were about using acupuncture to reduce post-operative vomiting, and they are highly cited by other papers, but those other papers are mostly touting the efficacy of using acupressure bracelets to reduce vomiting after anesthesia. The best, published argument I found for using acupuncture in surgery is the post-operative vomiting approach, and the National Institutes of Health (NIH) has a consensus statement that agrees. One paper on the issue of post-operative vomiting mitigation is referred to by the NIH and has been cited at least 139 times in the peer-reviewed scientific literature since 1999 (Lee and Done 1999).

The closest thing I could find to using acupuncture with real surgery is a case study that describes two men who had acupuncture before surgery for a condition called varicocele (an enlargement of the veins within the scrotum) and one who wanted to be circumcised at age 40, but this study has never been cited in its 10-year history (Menardi et al. 2004). I suppose, in the arena of case studies, the fact that I fall asleep every time I get a tattoo is also equally convincing evidence that an acupuncture-like treatment may mimic anesthesia, or at least cure insomnia.

The take home message for me and for my students is that acupuncture does not reliably perform better than a sham treatment, and the nocebo effect in both acupuncture and sham is real and concerning. Another take home message is that there is no acceptable physiologically testable explanation for how acupuncture works, if it indeed actually does work.

If we are to accept acupuncture as medical science and not pseudoscience, then 1) it must perform reliably in double-blind, placebo controlled experiments that exclude all other possible explanations, and 2) its mechanism of action must be transparent, quantitative, and based in known human physiology. The burden of proof is on acupuncture, and anecdotal evidence does not qualify as empirical data.

However, this entire argument against the claims that acupuncture works as a medical science does not falsify the claim that acupuncture has the potential to do something.

Acupuncture and some CAM practices are Better Than Nothing

Could it be that acupuncture and many other CAM treatments are better than nothing? Certainly, if we look at acupuncture like we look at massage, then hands down, literal hands-on approaches do reap huge benefits for the patient in measurable qualitative physiological and psychological ways that traditional medicine cannot. There is also the psychological variable of what a patient hopes and expects the outcome of a therapeutic intervention to be. Indeed, the National Center for Complementary and Integrative Health (NCCIH) at the NIH does warn that “current evidence suggests that many factors—like expectation and belief—that are unrelated to acupuncture needling may play important roles in the beneficial effects of acupuncture on pain.”

CAM promoters and practitioners often cite the NIH NCCIH as evidence for CAM as a real alternative to Western medicine. I have perused the NIH NCCIH site and am fascinated by the Research Spotlights page. It is really well done, and what jumps out at me is how tentative each headline is. For example on the Research Spotlights for 2012 page is the headline, “Meditation or Exercise May Help Acute Respiratory Infections, Study Finds,” (Barrett et al. 2012).

But there are big challenges for studies like this.

How Making Claims in Science Works (Warning: Statistics ahead, continue at your own risk)

The Barrett et al. (2012) study mentioned above lacks the group that, for example, thinks they are exercising or meditating but they are not. It may simply be a reduction in stress that is behind the measured effect, not the exercise or the meditation. But creating the group to test this hypothesis may be impossible. Again, to come to the strongest conclusions, studies must attempt to exclude all other possible explanations for the observed result. Barrett et al. (2012) understand this dilemma and end their paper with this statement:

While not all of the observed benefits were statistically significant, the researchers noted that the magnitude of the observed reductions in illness was clinically significant. They also found that compared to the control group, there were 48 percent fewer days of work missed due to acute respiratory infections in the exercise group, and 76 percent fewer in the meditation group. Researchers stated that these findings are especially noteworthy because apart from hand-washing, no acute respiratory infection prevention strategies have previously been proven. The researchers concluded that future studies are needed to confirm these findings.

Unfortunately, it’s not likely that this exact study will be replicated enough times, or even once, to test their additional hypotheses.

Another very recent study that has received a flurry of positive comments on social media looked at the effect of meditation and exercise on gene expression (Carlson et al. 2014)—a very sexy study, especially in the new age of epigenetics. However, we have known about the effect that stress has on telomere length for at least a decade (Price et al. 2013). It is a provocative study, but one must read the published article to get the whole story.

The authors summarize the results as follows:

Using analyses of covariance on a per-protocol sample, there were no differences noted between the MBCR and SET groups with regard to the telomere/single-copy gene ratio, but a trend effect was observed between the combined intervention group and controls (F [1,84], 3.82; P = 0.054; effect size = 0.043); Telomere Length in the intervention group was maintained whereas it was found to decrease for control participants. There were no associations noted between changes in Telomere Length and changes in mood or stress scores over time.

That’s a lot to digest.

The p-value above from their analysis of covariance (ANCOVA) is 0.054 and indicates that there is a greater than 1 in 20 chance that the observed “trend” is accidental and not real. Therefore, no statistical significance, and we cannot reject the null statistical hypothesis (H0) that there is no trend. But more importantly, the effect size of 0.043 is tiny and a handful of test subjects (literally four or five individuals) could be entirely responsible for the possible trend. In fairness, though, it is entirely possible that by concluding there is no real trend, we take the risk of making what is called a Type II error in inferential statistical hypothesis testing: failing to reject the null hypothesis when it is false—there may indeed be something going on with meditation and genetics. But the authors do admit:

Although the current study is strengthened by randomization and the inclusion of only distressed survivors, it does have several limitations. Chief among these is missing data, which precluded the feasibility of conducting intent-to-treat analyses. The control condition was also therefore small (18 individuals), because twice as many women were randomized to the active intervention groups as to the control condition. Hence, the study would require approximately twice as many participants in each group to detect a change in the T/S ratio of 0.5.

Indeed, we must be tentative with results like these, but most people read and pass along headlines without skepticism. They do not, in fact cannot, dig into the details of the study due to limitations in being able to read technical language and understand statistical results.

All we can conclude from this study is that, taken as a whole, the results neither confirm nor refute the utility of meditation.

However, the fact that meditation is incredibly healthful and is better than nothing is undeniable. But, unlike acupuncture, it’s free!

In summary, we may be able to cure our students (our future voters) of pseudoscience and pseudoscientific thinking by exposing them to the claims of practices like acupuncture that masquerade as medical science and by helping them identify and unpack the pseudoscientific assertions of these practices and understand why the claims are indeed pseudoscientific.

Curing our students of pseudoscience may be one of our most important roles as science educators.

Peer-reviewed Literature Cited

Barrett, B., Hayney, M. S., Muller, D., Rakel, D., Ward, A., Obasi, C. N., … & Coe, C. L. (2012). Meditation or exercise for preventing acute respiratory infection: a randomized controlled trial. The Annals of Family Medicine, 10(4), 337-346.

Benedetti, F., Amanzio, M., Baldi, S., Casadio, C., Cavallo, A., Mancuso, M., … & Maggi, G. (1998). The specific effects of prior opioid exposure on placebo analgesia and placebo respiratory depression. Pain, 75(2), 313-319.

Carlson, L. E., Beattie, T. L., Giese‐Davis, J., Faris, P., Tamagawa, R., Fick, L. J., … & Speca, M. (2014). Mindfulness-based cancer recovery and supportive-expressive therapy maintain telomere length relative to controls in distressed breast cancer survivors. Cancer.

Cherkin, D. C., Sherman, K. J., Avins, A. L., Erro, J. H., Ichikawa, L., Barlow, W. E., … & Deyo, R. A. (2009). A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Archives of Internal Medicine, 169(9), 858-866.

Ernst, E., Lee, M. S., & Choi, T. Y. (2011). Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. Pain, 152(4), 755-764.

Goetz, C. G., Leurgans, S., Raman, R., & Stebbins, G. T. (2000). Objective changes in motor function during placebo treatment in PD. Neurology, 54(3), 710-710.

Hall, K. T., & Kaptchuk, T. J. (2013). Genetic biomarkers of placebo response: what could it mean for future trial design?. Clinical Investigation, 3(4), 311-313.

Koog, Y. H., Lee, J. S., & Wi, H. (2014). Clinically meaningful nocebo effect occurs in acupuncture treatment: a systematic review. Journal of Clinical Epidemiology.

Kuhn, D. (1993). Science as argument: Implications for teaching and learning scientific thinking. Science Education, 77(3), 319-337.

Langevin, H. M., Wayne, P. M., MacPherson, H., Schnyer, R., Milley, R. M., Napadow, V., … & Hammerschlag, R. (2010). Paradoxes in acupuncture research: strategies for moving forward. Evidence-Based Complementary and Alternative Medicine, 2011.

Lee, A., & Done, M. L. (1999). The use of nonpharmacologic techniques to prevent postoperative nausea and vomiting: a meta-analysis. Anesthesia & Analgesia, 88(6), 1362-1369.

Madsen, M. V., Gøtzsche, P. C., & Hróbjartsson, A. (2009). Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. BMJ, 338.

Minardi, D., Ricci, L., & Muzzonigro, G. (2004). Acupunctural reflexotherapy as anaesthesia in day-surgery cases. Our experience in left internal vein ligature for symptomatic varicocele and in circumcision. Arch Ital Urol Androl, 76(4), 173-4.

Pollo, A., Vighetti, S., Rainero, I., & Benedetti, F. (2003). Placebo analgesia and the heart. Pain, 102(1), 125-133.

Price, D. D., Finniss, D. G., & Benedetti, F. (2008). A comprehensive review of the placebo effect: recent advances and current thought. Annu. Rev. Psychol., 59, 565-590.

Price, L. H., Kao, H. T., Burgers, D. E., Carpenter, L. L., & Tyrka, A. R. (2013). Telomeres and early-life stress: an overview. Biological Psychiatry, 73(1), 15-23.

Servick, K. (2014). Outsmarting the placebo effect. Science, 345(6203), 1446-1447.

Witt, C. M., Jena, S., Selim, D., Brinkhaus, B., Reinhold, T., Wruck, K., … & Willich, S. N. (2006). Pragmatic randomized trial evaluating the clinical and economic effectiveness of acupuncture for chronic low back pain. American Journal of Epidemiology, 164(5), 487-496.

Standard
Hypothesis, Nature of Science, Science Practices, Uncategorized

Generalizing vs Explanatory Hypotheses: How do we use them in Practice?

The following post is an extension of a much earlier post of mine called Teaching the Hypothesis. If you haven’t read it yet, you might start there. Here I focus on the distinction between generalizing and explanatory hypotheses and how we use them in the science practices.


In 1991 I had moved to Seattle to begin my high school science teaching career and was also training as a competitive distance runner with the local post-collegiate and semi-pro runners in the area. Over meals out and meals made together after long, hard runs I quickly learned from the others about the recently reported performance benefits of a diet high in omega-3 fatty acids. Being in the Pacific Northwest it was easy to load up on excellent sources of omega-3s like salmon and halibut, but for a hopeful additional edge against the competition, many of us also took daily fish oil supplements.

Even though I had majored in biochemistry in college and had a secondary science teaching certification, I was unfortunately not trained as a scientist, let alone to think like one. So if my running buddies, especially those far faster than me, claimed that fish oil would make me faster, then it was true.

What the endurance athletics community was all a buzz about was a provocative 1988 study published by two medical scientists at the University of California at Irvine, David Leaf and C. R. Rauch. Leaf and Rauch had investigated how nutrition affected aerobic performance. One result they described in their paper was a pattern they discovered: increasing dietary omega-3 fatty acids seemed to cause an increase in aerobic performance in athletes. However, Leaf and Rauch had not proven that the pattern was true, they had simply shown experimental support for the pattern. But as a young science teacher, I had no idea how to carefully communicate this claim to my biology students and the runners I coached on my school’s cross country team. During a nutrition unit or when discussing with my athletes what to eat as a runner I likely said something  like, “And did you know that eating foods high in omega-3 fatty acids like fish will make you faster?”

I’m better now. I continue to learn from my mistakes as a science teacher (you can read more about me as an impostor here), but I also now have more experience doing science myself as well as more experience teaching it. I now use this story, and a study that was published nine years later, as one of many ways to focus my students on the Nature of Science and how science works. But more specifically, my goal is to provide an illustration for my students from the literature on the difference between a generalizing hypothesis and an explanatory hypothesis, and between predictions and experimental hypotheses in general.

Generalizing and Explanatory Hypotheses

It is important to make clear here that there are two distinct categories of experimental hypotheses. First, there are generalizing hypotheses that are descriptions of observed patterns in nature. For example, dietary omega-3 fatty acids increase aerobic performance. A pattern may be real and meaningful and repeatable across broad natural landscapes. A pattern may also be real, but not meaningful. And a pattern may be accidental, where one variable may be associated with another variable but neither causes the other (i.e. correlation does not imply causation). Then there are explanatory hypotheses, possible explanations for observations in nature, that explain why the potentially meaningful patterns are there or how the patterns are generated and maintained.

Results must be Repeatable to Maintain Credibility

Three Scandinavian exercise physiologists were so intrigued by the pattern discovered and reported by Leaf and Rauch that they set out to replicate the Leaf and Rauch study and test their hypothesis, but under more controlled conditions. The scientists, Raastad, Hostmark, and Stromme, published their work in 1997 in the Scandinavian Journal of Medicine and Science in Sport (Figure 1). The link to the article is here. It is behind a paywall, but perhaps your institution has a way around it. You could also email Truls Raastad directly for a copy.


Raastad 2

Figure 1. Image of Raastad et al. (1997).


What I find so interesting and instructive is how Raastad et al. moved so elegantly and logically through scientific methodology and I share this with my students. In fact, I have my students read the paper while following the guide in Figure 2 below.


Article Review for Raastad et al 1997

Figure 2. The guide I use for students to analyze Raastad et al. (1997).


To help my students dial in on the hypothesis component of the Nature of Science, I focus them in on a few areas of the paper, as illustrated by #3 and #4 in Figure 2 above: What are the generalizing and explanatory hypotheses mentioned in the Raastad et al. paper, and which hypothesis is actually being tested?

The Hypothesis on Trial

In Figure 3 below is the first sentence (and first paragraph) of the Raastad et al. Introduction. Notice how they get right to the pattern claimed by Leaf and Rauch. They also make the statement tentative: “may.” However, words like “may” aren’t required of hypothesis statements. Indeed, being tentative is implied by the fact that a hypothesis statement is falsifiable.


Raastad 3

Figure 3. Raastad et al. (1997) first paragraph of the Introduction.


At the end of the Introduction, Raastad et al. then propose a mechanism (explanatory hypothesis) to explain the Leaf and Rauch pattern, if indeed it is real, and restate the generalizing hypothesis (from Leaf and Rauch) that they themselves are retesting (Figure 4).


Raastad 4

Raastad 5

Figure 4. Raastad et al. last paragraph of the Introduction.


Clearly, Raastad et al. propose that a possible physiological explanation (an explanatory hypothesis) for the suspected pattern is that

“[O]mega-3 fatty acid supplementation may improve performance by increasing tissue oxygenation due to reduced blood viscosity, thereby increasing maximal cardiac output and the peripheral blood flow.”

The three exercise physiologists probably had no intention initially of testing their explanatory hypothesis because of its complexity and how invasive it would be for their test athletes. However, if they had found strong confirmation for the Leaf and Rauch pattern, it may have given them a reason to further pursue the underlying mechanism for the pattern; i.e. the explanatory hypothesis.

I use the Raastad et al. article to teach my students much more about the Nature of Science (see again Figure 2), but what I emphasize at this point for my students is the difference between generalizing and explanatory hypotheses, and that for the results of a study to be credible, other researchers must be able to replicate the study and get the same or very similar results.

Relating Generalizing and Explanatory Hypotheses

Using an example familiar to most folks who have taken a biology course, Figure 5 illustrates how generalizing and explanatory hypotheses are related.

In the 1850s, Hungarian monk, Gregor Mendel, formally described some inheritance patterns he and others had noticed: Forms of traits become segregated from each other during sex cell formation and different traits sort independently. Then Mendel tested if the patterns were real. And he tested them and tested them and tested them again. In the end, after the patterns continued to exist, Mendel proposed the Laws of Segregation and Independent Assortment. Alongside the patterns, he proposed a reason for the patterns and hypothesized that the hereditary material was not fluid, but instead particulate in nature. The particulate explanation allowed for the prediction that the traits Mendel was following should not behave like a fluid, but should stay distinct and recombine in later generations—and indeed, in predictable proportions. This hypothesis was further tested by T. H. Morgan who lifted Mendel’s particulate hypothesis to a more sophisticated level and, with his results of his X-linked white-eye mutation experiments, Morgan developed the Chromosomal Theory of Inheritance.


Mendel

Figure 5. The pathways to theories and laws by way of explanatory and generalizing hypotheses.


Figure 5 thus shows how sometimes, but rarely, explanatory hypotheses can mature to the level of scientific theories, and how generalizing hypotheses can sometimes prove universal enough to be called laws. But while theories can sometimes explain why laws work the way they do, they themselves can never become laws. Theories aren’t patterns, they are explanatory frameworks for patterns. A similar version of Figure 5 appears in my 2015 American Biology Teacher paper, Hypothesis Generation in Biology: A Science Teaching Challenge & Potential Solution.

Hypothesis Testing in the Science Classroom

In the Science Classroom, most of what our students do is the testing of patterns, in other words, the testing of generalizing hypotheses. Only when patterns are supported by experimental evidence do we encourage our students to come up with mechanisms to explain the patterns. In some cases, students can test their explanations with additional controlled experiments, but often this can take time and some pretty sophisticated methods and equipment. What is most important, though, is that our students constantly practice their skills at designing and running controlled experiments, analyzing, summarizing, and interpreting data, and coming to logical and arguable conclusions, regardless if the hypotheses under fire are descriptions of patterns or mechanistic explanations.

An Example from Physiology

Let’s take “physical activity increases heart rate” as an example of a testable hypothesis. This statement is a generalizing hypothesis, a pattern that we may observe and take note of in ourselves and in a group of test subjects: “It seems that when this group of people exercises, all of their heart rates increase.” We then formally describe the observation with a generalizing hypothesis.

Hypothesis: Physical activity increases heart rate.

This statement has all the appropriate pieces of a hypothesis: it is tentative (can be found to be false), it is testable, and it is within the boundaries of nature. We can thus do a controlled experiment to test if the pattern is real and repeatable among many subjects from different groups and even across animal species. The particular experimental methods will allow us to make a critical prediction from the hypothesis.

Prediction: Subjects who exercise vigorously for two minutes will have significantly higher heart rates than when they started and will also have significantly higher heart rates than a resting treatment group.

We might even add the hypothesis at the beginning and have what the science education literature calls the Research Hypothesis. Research hypotheses are rarely seen in published experimental studies, but they can function as important organizing tools and take on the form, If X hypothesis is valid, and I perform Y methods, then I can predict Z as a specific, measurable outcome.

Research Hypothesis: If physical activity increases heart rate, and ten subjects exercise vigorously for two minutes while ten other subjects sit quietly, then the subjects that exercised will have significantly higher heart rates than when they started and will also have significantly higher heart rates than the resting treatment group.

If we continue to test the pattern over and over again and in different species, at some point we may even conclude that the positive relationship between exercise and heart rate is simply a biological law and we may even be able to explain the relationship mathematically.

But, what we don’t yet have is a physiological explanation for why the pattern exists or a mechanism for how it is produced. The explanation for why there is a repeatable pattern is our explanatory hypothesis. Explanatory hypotheses are often difficult for students in a lab setting to come up with, let alone test in any meaningful way. However, we can still propose explanatory hypotheses without ever testing them—students could do this in their conclusion and evaluation of their lab paper and scientists do this all the time in their discussion sections.

For example, students in an IB or AP Biology course or a college Anatomy and Physiology course may propose the explanatory hypothesis that increased exercise increases carbon dioxide production and lowers blood pH which in turn stimulates the heart to beat faster, thus removing excess carbon dioxide and bringing blood pH back to normal. This is a testable explanatory hypothesis but we would not expect our students to do the controlled experiment, at least not on their peers, but maybe on mice.

An even Simpler Example from Physical Science

Students are shown three swinging pendulums of the same mass and notice the pattern that the longer the pendulum, the slower the swing.

Hypothesis: There seems to be a positive relationship between the length of a pendulum and the period of its swing.

Prediction: If I increase the length of a pendulum by 0.2 meters at a time, beginning at 0.2 m and ending at 1.0 m, swinging each different pendulum with the same beginning angle, then the number of seconds it takes the pendulum to make one pass through its arc will also increase… and might even be linear.

Notice that the prediction is specific to the planned test/experiment.But also notice that the prediction is not simply a restatement of the hypothesis: There will be a positive relationship between the length of a pendulum and the period of its swing. Instead, the prediction includes a clear, unique, planned test that should produce a specific measurable outcome.

Beginning with a simple experiment like this can lead students to ask additional questions about pendulums and propose additional interesting hypotheses and predictions.

In summary, science tests its hypotheses better than any other human endeavor. Scientists do their best to test (even falsify) their own hypotheses before they will accept them. Helping students understand that there are two types of hypotheses and how to use them in science inquiry can clear up some misunderstandings about hypotheses and predictions and open up a lot of possibilities in the classroom laboratory where students are learning how to think like scientists.

References

Leaf, D. A., & C. R. Rauch. (1988). Omega-3 supplementation and estimated VO2 max: a double blind randomized controlled trial in athletes. Annals of Sports Medicine, 4: 37-40.

Raastad, T., A. T. Hastmark, & S. B. Strømme. (1997). Omega‐3 fatty acid supplementation does not improve maximal aerobic power, anaerobic threshold and running performance in well‐trained soccer players. Scandinavian Journal of Medicine & Science in Sports, 7: 25-31.

Strode, P. K. (2015). Hypothesis Generation in Biology: A Science Teaching Challenge & Potential Solution. The American Biology Teacher77:500-506.

 

 

Standard
Hypothesis, Science Practices, Teaching Methods

“Science is not belief, but the will to find out.” – Anon

Soon after I first met my Father-in-Law, Ted, I learned that one of his greatest pleasures in life was to ask a provocative question during a gathering of a thoughtful group of people and see what ensued. It was in these arenas that Ted challenged the people he loved to reveal and defend their most strongly held convictions. Stir the pot, he did.

In the last few years of his life, Ted’s ability to stir the pot was gradually quieted by the advancing symptoms of Progressive Supranuclear Palsy (PSP), a brain disorder similar to Parkinson’s disease. However, this condition did not stop Ted from occasionally handing me things to read to challenge my own thinking, especially in the realm of science and religion. For example, on an October 2014 Sunday afternoon, soon after we arrived at my In-Laws’ to watch the Denver Broncos on TV, Ted reached over from his brown Lazy Boy recliner and slid me a copy of America Magazine. America is a weekly magazine published by the Jesuits of the United States and “contains news and opinion about Roman Catholicism, and how it relates to American politics and cultural life.” Ted wanted my thoughts on an article titled Justified Reason: The Collaboration of Knowledge, Belief, and Faith, by astrophysicist, Adam Hincks.

In the article, Hincks argues that knowledge (in this case, that which we know from scientific consensus) versus belief and faith need not be mutually opposed. In fact, Hincks maintains that belief “is a crucial element in scientific research.” As an example, Hincks explains that “we (scientists) need to believe in the results that our colleagues produce.” Hincks describes his use of the term belief as aligned with the writings of St. Augustine, who described belief as “nothing other than to think with assent.” Thus, according to Hincks, regardless of whether the context is science or religion, to believe in something, a claim for example, involves nothing more than agreement and compliance (assent) after some sort of internal mental activity (thinking).

The definition of belief Hincks has chosen with which to make his argument is so over-simplified that, in connecting it to scientific thinking, he misrepresents how science itself works. Hincks also misrepresents how complex and contextual the concept of belief is.

Below, I will argue that Hincks, and others, including Presidential candidates, are careless and irresponsible when they lob the word belief into the arena of science such that it dilutes and weakens the scientific process and the immense knowledge we have accumulated by way of science.

What is Belief?

In a strictly religious context, and to zero in on Hincks’s Catholic foundation from which he argues, the online Catholic Encyclopedia defines belief as “that state of the mind by which it assents to propositions, not by reason of their intrinsic evidence, but because of authority.”

Immediately, this definition (“not by reason of their intrinsic evidence”) disqualifies belief as a way of thinking in science and contradicts Hincks’s argument altogether.

Moreover, the Oxford English Dictionary’s definition of belief includes the eight contexts summarized below. But heads up, the link is hard to get to and requires a library card # from your local library, so here is the abbreviated entry:

  1. Theol. (a) The trust that the believer places in God; the Christian virtue of faith.
  2. The mental action, condition, or habit of trusting to or having confidence in a person or thing; trust, dependence, reliance, confidence, faith.
  3. Theol. A formal statement of doctrines believed, a creed.
  4. Something believed; a proposition or set of propositions held to be true. In early usage esp.: a doctrine forming part of a religious system; a set of such doctrines, a religion. (b) Philos. A basic or ultimate principle or presupposition of knowledge; something innately believed, a primary intuition.
  5. (a) With of. Acceptance of the truth of a statement or the words of a speaker; acceptance of a supposed fact. Now rare. (b) With in. Acceptance or conviction of the existence or occurrence of something.
  6. With that. Acceptance that a statement, supposed fact, etc., is true; a religious, philosophical, or personal conviction; an opinion, a persuasion.
  7. Without construction: assent to a proposition, statement, or fact, esp. on the grounds of testimony or authority, or in the absence of proof or conclusive evidence. Also (chiefly Philos.): the way in which pure reason acknowledges objects existing beyond the reach of empirical evidence or logical proof.
  8. Confident anticipation, expectation; an instance of this.

Considering all eight definitions, only numbers 5 and 6 qualify (loosely) as what a scientific thinker likely means when he or she uses belief or believe in the context of a scientific claim: acceptance that a statement is true. But contrary to what Hincks argues, stating belief in a scientific fact, observation, or hypothesis is careless use of language by scientists, and most scientists, especially evolutionary biologists (more on this later), avoid the term altogether. To understand why scientific discourse should avoid the word belief, we can first turn to philosophy.

Belief versus Acceptance

The Stanford Encyclopedia Philosophy Archive has dedicated over 13,000 words to the questions, What does it mean to believe? and When and how are beliefs justified and qualify as knowledge? The entire entry is certainly a fascinating and worthwhile read, but most germane to the discussion here is the Encyclopedia’s passage on belief and acceptance. Here the author, University of California, Riverside, philosophy professor, Eric Schwitzgebel, makes the distinction that “acceptance is held to be more under the voluntary control of the subject than belief.”

Consider this example: You may see that a friend has leaned a ladder against a roof. She states that the ladder is safe to climb and you believe her. Thus, you have established a belief in the safe ladder claim. However, as you approach the ladder, your empirical nature takes over. Upon close inspection (i.e. data gathering) by considering what the ladder is made of, by pushing and pulling on the ladder, and by observing the surface upon which it is placed, you conclude that the ladder is far too wobbly and weak to climb and therefor no longer accept the claim that it is safe. In fact, the evidence available is grounds for rejecting the safe ladder claim outright. In this case, acceptance requires a voluntary investigation into a claim before we accept or reject it.

Science education researchers Louis Nadelson, from Boise State University, and Sherry Southerland, from Florida State University, explain this distinction between belief and acceptance in a recent article in the International Journal of Science Education. While their focus in the paper is to develop and analyze a tool to assess student acceptance of evolution, Nadelson and Southerland first establish a clear distinction between belief and acceptance that further explains the logic in the example above:

“Many educational researchers and in particular those investigating evolution education are careful to draw a distinction between a learners’ belief in a construct and their acceptance of that same construct (Nadelson, 2009; National Academy of Sciences, 1998; Sinatra et al., 2003; Smith, Siegel, & McInerney, 1995). Smith (1994) argues that belief implies the warrant for consideration of a construct is based on personal convictions, faith, feelings, opinions, and the degree of congruence with other belief systems. In contrast, acceptance of a construct is based on an examination of the validity of the knowledge supporting the construct, the plausibility of the construct for explaining phenomenon, persuasiveness of the construct, and fruitfulness or productivity of the empirical support for the construct.” (Nadelson and Southerland, 2012)

Indeed, “the plausibility of the construct for explaining phenomenon” and “productivity of the empirical support for the construct,” are key when employing scientific thinking to assess a claim.

Why Word Choice is Important in Science Discourse

Kevin Padian is a Vertebrate Paleontologist in the Department of Integrative Biology at the University of California at Berkeley (the same university where Eric Schwitzgebel recieved is Doctorate in Philosophy). In a 2013 article in the Journal of Evolution Education and Outreach, Padian argued that the sloppy treatment of scientific subjects by professional writers in science textbooks and other science media, and by teachers in science classrooms may have its origin in large part in the careless use of language by many scientists. This language of course includes–but is not limited to–belief and believe. Says Padian:

“[S]aying that scientists ‘believe’ their results suggests, falsely, that their acceptance is not based on evidence, but is based somehow on faith. Yet again, it is about the quality of the evidence: scientists accept their results as the best explanation of the problem that we have at present, but we recognize that our findings are subject to re-evaluation as new evidence comes to light. This is a problem because scientists themselves often use the word ‘believe’ when discussing their results! It is just sloppy diction: they would not say that their conclusions are a matter of faith, rather than of evidence. Instead of saying ‘many scientists believe’ or ‘some scientists think’, it is more productive to talk about the evidence.” (Padian, 2013)

Thus, a statement by a scientist about a well-tested hypothesis or a known scientific fact that is introduced with the words, “I believe that…,” can be falsely interpreted by a science skeptic or science denier as equivalent to saying, “I believe in magic.”

University of Chicago Evolutionary Geneticist, Jerry Coyne, in his new book Faith vs. Fact: Why Science and Religion are Incompatible, takes Padian’s argument a bit further. In addition to the fact that use of the terms belief and believe in science discourse and by scientists is sloppy and unproductive, Coyne maintains that faith, inferred by belief and believe, is simply dangerous in the context of science:

“[F]aith, as employed in religion (and in most other areas), is a danger to both science and society. The danger to science is in how faith warps the public understanding of science: by arguing, for instance, that science is based just as strongly on faith as religion; by claiming that revelation of the guidance of ancient books is just as reliable a guide to truth about our universe as are the tools of science; by thinking that an adequate explanation can be based on what is personally appealing rather than on what stands the test of empirical study.” (Coyne, 2015, pp. 225-226)

Nowhere is the blurring of science and faith more of a problem than in the scientific study of evolution as it relates to the biochemical origin of life and human origins from earlier forms of nonhuman life. Indeed, the constant attack on evolutionary theory by various versions of creationism, including a literal interpretation of Genesis in the Bible (biblical literalism) and the claims of intelligent design creationists, should be news to no one.

When Religion Rejects Science: Appealing to the Consequences

In the case of biblical literalism and its effect on acceptance of science, Yale University Sociology graduate student, Esther Chan and Rice University Sociologist, Elaine Ecklund report the following in a recent paper published in the Journal for the Scientific Study of Religion:

“Many studies suggest biblical literalism is the root cause of conflict between the authorities of the Bible and science. Scholars find that fundamentalists display less scientific literacy than the average American (Sherkat 2011), less confidence toward scientists (Evans 2013), as well as considerable opposition toward evolution (Berkman, Pacheco, and Plutzer 2008; Lienesch 2007; Scott 2004). Because those with literalist views of the Bible believe the Bible provides foundational knowledge and are less likely to be influenced by other sources (Darnell and Sherkat 1997; Ellison and Musick 1995; Greeley and Hout 2006), their views are often in conflict with scientific claims. Indeed, some scholars suggest that biblical literalism acts as a dominant cognitive schema that facilitates how one interprets the social world, including science (Bartkowski et al. 2012; Hempel and Bartkowski 2008; Hoffman and Bartkowski 2008; Ogland and Bartkowski 2014).” (Chan and Ecklund, 2016)

A biblical literalist can thus easily (and in his or her mind, rationally) reject scientific knowledge and make the statement, “I believe in the Bible’s account of creation as described in Genesis,” (i.e. supernatural causation) while following it with, “I don’t believe in evolution.” But neither the statement, “I don’t believe in evolution,” nor the statement, “I believe in evolution,” are at all useful or meaningful in a scientific context. Biologists and Co-Directors of the New England Center for the Public Understanding of Science at Roger Williams University, Guillermo Paz-y-Mino-C and Avelina Espinosa argue that by way of the Incompatibility Hypothesis, “Scientific rationalism and empiricism are incompatible with belief in supernatural causation. Belief disrupts, distorts, delays, or stops the comprehension and acceptance of scientific evidence.” Thus, neither statement about evolution belief (for OR against) is necessary given the weight of the evidence in favor of undirected, biological evolution. Paz-y-Mino-C and Espinosa, explain in their new book, Measuring the Evolution Controversy, that given the overwhelming evidence that natural, biological evolution has occurred, “the reality of evolution is indisputable… all people in the world should accept it as fact.” However, the common coupling of the term believe with a person’s position on the validity of evolutionary theory makes it too easy for a person to weigh the consequences of accepting the fact of evolution against his or her beliefs. Philosophers call this reaction the appeal to the consequence: rejecting a claim of fact because its consequences are perceived to be undesirable. I’ve written about this phenomenon in my own book with Physicist, Matt Young, Why Evolution Works (and Creationism Fails). Here we explain:

“[I]t is a fallacy to reject evolution because you do not like its consequences (or what you think are its consequences). Evolution may or may not lead some people to disbelieve in God; that has nothing to do with its validity. In the same way, a false doctrine that leads people to believe in God is no more true for that consequence. In short, a claim of fact must be judged solely on its merits, not on your preference for its consequences. More to the point, evolution must be judged on its merits, even though it may lead some people to disbelieve in God.” (Young and Strode, 2009, p. 9)

Who We Are (Belief) vs. What We Know (Science)

When we appeal to the consequences, we are falling victims to a logical fallacy called motivated reasoning: we are basing our conclusions on what we hope to be true about the world (who we are) instead of the knowledge made available to us by way of scientific methodology (what we know). Yale University Law and Psychology Professor, Daniel Kahan, has written extensively about this conflict, especially in the context of national surveys of the public’s general scientific knowledge. The images below show several figures from a 2014 paper by Kahan that illustrate the conflict that motivated reasoning presents.

Screen Shot 2016-07-30 at 9.31.08 AM

Figures (but not surrounding text) from Kahan (2014)

Screen Shot 2016-07-30 at 10.03.50 AM

Figure (but not surrounding text) from Kahan (2014)

The figures show that, for example, when asked if it is true that humans evolved from earlier species, a person’s answer is at the mercy of his or her belief system (who we are), regardless of scientific knowledge. But when asked what can be claimed by way of evolutionary theory (what we know), there is no consequence to a person’s belief system by answering correctly: “Yes, I have learned that evolutionary theory makes that claim.” The same is true with political affiliations (who we are) and what we are told to believe about global warming (or any other hot political topic) by peers, mentors, and political leaders.

Who we are is a powerful motivator. It is so powerful that too many well-informed citizens can comfortably respond, “Well, I don’t believe that,” when presented with, for example, the evidence for a human cause behind global warming, or really any scientific claim that challenges a firmly held belief or overarching belief system.

It is this reality that most likely prompted Hillary Clinton to emphatically pronounce, “I believe in science!” in her acceptance address on July 28th, 2016, during the Democratic National Convention. I cringed at this proclamation. I did not cringe because I think that Clinton literally means that she has faith (belief without evidence) in science. I cringed because by lobbing belief into the brief mention of climate change that followed, she really did not do justice to the massive body of evidence behind the facts of global warming and climate change and its human cause. While it was a commendable attempt to celebrate the power of science and what it can teach us about the natural world, Clinton’s statement failed to help close the door on science denial. I am hopeful that she and her staff will carefully read some of the criticisms of aligning oneself with the power of science and its products by using a statement that begins with, “I believe…” As Kevin Padian argues above, “it is more productive to talk about the evidence.” Indeed, it is.

So, as scientists, do “we need to believe in the results that our colleagues produce[?]” The philosophers would likely say “no.” I would hope that all reasonable scientists would also say “no” and agree with Matt Young and me when we argue in Why Evolution Works (p. 188) that “science tests its hypotheses better than any other human endeavor. Scientists do their best to test (even falsify) their own hypotheses before they will accept them. Any belief, religious or other, that denies known scientific fact is seriously in need of reconsideration.” By “other” we can include even nonreligious beliefs that range from a belief that homeopathy works for treating human health conditions (it doesn’t) to a belief that vaccines cause autism (they don’t).

Is Faith Required for Scientific Inquiry?

Adam Hincks closes his article, Justified Reason: The Collaboration of Knowledge, Belief, and Faith, with a provocative claim that may in fact sit at the foundation of his motivation to marry religious thinking and reasoning with scientific thinking and reasoning:

“For human inquiry was never meant to be a purely human collaboration, but a collaboration with the mind of God. If we expel God from the intellectual life, we may find that reason itself soon withers.”

Hinck’s hypothesis that faith in the existence of the Christian God is required for true human inquiry to proceed and to keep reason from withering makes possible a critical prediction: Our most accomplished and decorated scientists, those that are members of the National Academy of Sciences (NAS) and responsible for generating some of our most important new knowledge and understanding of the natural world, when asked, should claim to be religious and/or believe in a personal God.

The data suggest otherwise.

A survey of NAS scientists and their religious beliefs has been repeated three times since 1914. The latest survey was conducted by Pepperdine University History and Law Professor, Edward Larson, and author and journalist, Larry Witham, and published in the journal Nature in 1998. While the most recent data are now at least 20 years old, the trend is clear, that belief in a personal God by leading scientists has dropped from almost 28% in 1914 to 7% in 1998. The authors summarize their finding by stating that “among the top natural scientists, disbelief is greater than ever — almost total.”

Screen Shot 2016-07-30 at 12.17.46 PM

Table from Larson and Witham (1998)

Clearly, religious faith and belief was not required for our best scientists to use the methods of science and make a majority of our most important scientific discoveries over the last century. If this were true, we would see the opposite trend from what is revealed by Larson and Witham.

Belief in the Science Classroom

In closing, perhaps the most important period in a person’s life to learn about how science works is during his or her years of formal education. As I argue in an earlier post on this site, Getting Students to Think Like Scientists, the most important thing we can teach our students is how to think like scientists. As a science teacher, I encourage my students to leave the words belief and believe at the door and instead come to class with evidence-based arguments, to share what they know and how they know it. When it comes to belief in science and its contributions to knowledge, I explain to my students, for example, that I don’t believe in evolution or climate change or the published results of my colleagues and the scientists that have come before us. Instead, I trust in the process of science and its ability to generate our massive and growing body of credible knowledge. I trust that the necessary skeptical nature of hypothesis testing and the peer review process can and will produce what we know. I also trust that when the process breaks down, through unsubstantiated claims, or unrepeatable results, or fraud and self-serving dishonesty, the checks and balances and required replication of the scientific process will right this ship. I emphasize to my students that, when done correctly science tests its hypotheses better than any other human endeavor. These things I share with my students as I guide them through the fascinating, albeit imperfect, process of inquiry where they can accept or reject claims based on the weight of the evidence. No belief required.

Instead of loosely and carelessly claiming “I believe in science,” perhaps the best option is to simply say what we mean; to explain how we know what we know, because

Science is not belief, but the will to find out.


My Father-in-Law, Ted, died from PSP complications the day after we watched the Broncos win the Super Bowl together and the evening after teaching (yes, he was still teaching!) one more session of his course, Non-Profit Issues and Techniques, at the University of Denver. He also had a 25-year career as President and CEO of the Rocky Mountain Chapter of the Arthritis Foundation and wrote and published the book, Managing and Raising Money that is Not Your Own, because he couldn’t find an already written book good enough for his students. I hope Ted would have been pleased with this argument, regardless of whether or not he agreed. Thank you for always stirring the pot, Ted!

References

Aveling, F. (1907). “Belief”, in The Catholic Encyclopedia. New York: Robert Appleton Company. Retrieved June 29, 2016 from New Advent: http://www.newadvent.org/cathen/02408b.htm

“belief, n.”. OED Online. June 2016. Oxford University Press. Retrieved June 30, 2016 from http://www.oed.com/view/Entry/17368?rskey=hJK39S&result=1

Chan, E., and E. H. Ecklund. (2016). Narrating and navigating authorities: Evangelical and mainline Protestant interpretations of the Bible and science. Journal for the Scientific Study of Religion 55(1):54-69.

Coyne, J. (2015). Faith vs. Fact: Why Science and Religion are Incompatible. Penguin, New York, NY.

Hincks, A. D. (2014). The collaboration of knowledge, belief and faith. America 211(7):15-18.

Kahan, D. M. (2014). “Ordinary Science Intelligence”: A science comprehension measure for use in the study of risk perception and science communication.” The Cultural Cognition Project Working Paper, 112th ed. Yale Law and Economics Research Paper 504.

Larson, E. J., and L. Witham. (1998). Leading scientists still reject God. Nature 394:313.

Nadelson, L. S., and S. Southerland. (2012). A more fine-grained measure of students’ acceptance of evolution: development of the Inventory of Student Evolution Acceptance—I-SEA. International Journal of Science Education 34:1637–1666.

National Academy of Sciences and The Royal Society. (2014). Climate Change: Evidence and Causes. National Academies Press.

Padian, K. (2013). Correcting some common misrepresentations of evolution in textbooks and the media. Evolution Education and Outreach 6:1-13.

Paz-y-Mino-C, G., and A. Espinosa. (2015). Evolution controversy: A phenomenon prompted by the incompatibility between science and religious beliefs. The International Journal of Science in Society 7:1-23.

Paz-y-Mino-C, G., and A. Espinosa. (2016). Measuring the Evolution Controversy: A Numerical Analysis of Acceptance of Evolution at America’s Colleges and Universities. Cambridge Scholars Publishing, New Castle, UK.

Schwitzgebel, E. (2015). “Belief”, in The Stanford Encyclopedia of Philosophy (Summer 2015 Edition), Edward N. Zalta (ed.). Retrieved June 29, 2016 from http://plato.stanford.edu/archives/sum2015/entries/belief/

Young, M., and P. K. Strode. (2009). Why Evolution Works (and Creationism Fails). Rutgers University Press, New Brunswick, NJ.

 

Standard
Science Practices, Teaching Methods

Getting Students to Think Like Scientists

A version of this post first appeared earlier this year in the Unity and Diversity Writing Project. Unity and Diversity exists “as a venue for teachers to talk about teaching in personal, meaningful ways.” Nearly all of the contributors are science teachers. The guiding question for 2015 was “What is the most important thing we can teach our students?” The original essay can be found here.

 

2015-09-24 10.48.47

Students setting drop traps for a study on edge effect and arthropod species richness and diversity.

The most important thing we can teach our students is how to think like scientists.

In a typical K12 science class, the nature of science lesson — often with a misleading title like “The Scientific Method” — sits distinct and isolated at the beginning. Teachers, some with no formal scientific training, may fumble through the lesson simply repeating what is in the textbook. Most teachers get some of the nature of science wrong, promoting, for example, the misunderstanding that a hypothesis is an educated guess and no different from a prediction. (For those interested, I published a paper in The American Biology Teacher in September of 2015 on what hypotheses are (and are not), how big the problem is among students, textbooks, and teachers, and strategies for fixing the problem. I have also written an earlier post titled Teaching the Hypothesis.)

After an early attempt at teaching the nature of science, any efforts to further expose students to how science works is then abandoned, left behind in late August or early September as the focus shifts almost entirely to content and canned ‘labs’ where students simply follow written procedures.

This was how I was taught in my science classes and thus how I taught biology for my first eight years: mostly content, little science.

I do love teaching the content of biology. I love teaching students about the conversion of light energy to chemical energy and trophic cascades and the mechanisms of evolution, but the content of biology is not the most important thing I teach my students. Indeed, science content in general, while important, is not the most important thing we teach students in our science courses.

The most important thing we can teach our students is how to think like scientists. What this means for our classrooms is doing science every day.

The National Academy of Sciences (NAS) has defined science in various publications. For example, in the NAS publication, Science, Evolution, and Creationism, science is:

“the use of evidence to construct testable explanations and predictions of natural phenomena as well as the knowledge generated through this process.” (p. 10)

However, my favorite informal definition is from “Faith vs. Fact” by Jerry Coyne. Coyne describes that science is:

the set of methods we cite when we’re asked ‘How do you know that?’ after making claims such as, ‘Birds evolved from dinosaurs,’ or ‘The genetic material is not a protein, but a nucleic acid.’” (p. 28)

Therefore, science is literally all of the tasks, including the trial and error, unique to the problem at hand, that we must carefully complete as we investigate patterns, test claims, and ultimately generate new knowledge.

Getting students to think like scientists and successfully perform these tasks is not easy; it means designing curriculum around content that gives students opportunities to:

  • Know and embrace failure (my colleague, Helen Snodgrass has written about teaching students about failure here in Unity and Diversity and discussed it recently on Science Friday);
  • Understand how to use statistics to quantify uncertainty and make inferences about populations from sample measurements (here is a free Stats Guide for Biology teachers!);
  • Know that science isn’t just content — a body of facts about the natural world — but also the methods we use to generate and confirm new knowledge;
  • Creatively invent methods to test both new and old patterns and explanations; and
  • Describe and test patterns in nature and generate testable explanations for those patterns.

This list is a tall order for any teacher. But one method of moving students in the direction of thinking like scientists is to develop inquiry experiences that incorporate one or more of these components because the inquiry is designed to generate messy data. The classroom data my students often generate can be messy enough that one class may find statistically significant support for a hypothesis while another class will not.

I provide my students many open-ended inquiry experiences where they design their own methods around some limited set of materials and equipment — and failure is common. As inexperienced scientists, their methods are often faulty due to a lack of controlling variables — indeed even our best scientists can be haunted by uncontrolled variables coupled with false assumptions (auxiliary hypotheses about the world that are not necessarily true). With each experience, students become more thoughtful about their own methodology and control, and move a little closer to what scientists actually do.

But training in experimental design is not enough.

We live today with overwhelming access to online information. Any citizen with a Web-connected device and a question about the world can ‘just Google it’ and be flooded with possible answers. But few citizens have the training, and more specifically, the scientific literacy to distinguish evidence-based answers from opinions, or legitimate scientific facts from pseudoscience and nonsense. An example of how convincing pseudoscience can be is a misleading biochemistry of glyphosate (the active ingredient in Roundup) paper that I wrote about in an earlier post. No amount of scientific course content or training in experimental design can protect people with little knowledge of the nature of science and scientific reasoning from the lure of pseudoscience and nonsense. This mistake is especially likely if the answer they get is the one they were hoping to find: glyphosate causes cancer and other horrible human health problems — classic confirmation bias.

To that end, we must encourage our students to be thoughtfully skeptical of both their own and others’ claims, evidence, and reasoning; use logic to assess those claims; and be willing to have their own claims, evidence, and reasoning critiqued by their peers.

There are two strategies for critically assessing claims that I have found useful as a science teacher over the years. Both strategies are based on the claim-evidence-reasoning (CER) model that the Biological Sciences Curriculum Study (BSCS) has formalized for teachers.

The first strategy I use comes from the textbook we use at my school in our first-year advanced biology courses, “Biology: The Unity and Diversity of Life,” 12th Ed. by Staar, et al.

When confronted with a claim, the strategy prompts the students to take the following steps when asking themselves about the claim:

Step 1: What claim am I being asked to accept?

Step 2: What evidence supports the claim? Is the evidence valid?

Step 3: Is there another way to interpret the evidence?

Step 4: What other evidence would help me evaluate the alternatives?

Step 5: Is the claim the most reasonable one based on the evidence?

A more formal version of this kind of strategy is provided in the book, “Understanding Scientific Reasoning,” by University of Minnesota Philosophy of Science professor, Ronald Giere, and his colleagues. Giere and colleagues have described what they call a “program” for evaluating theoretical hypotheses (claims/explanations). I use this approach with my Science Research Seminar students and my 2nd year IB/AP Biology students. The strategy goes roughly as follows (Giere, et al. 2006):

Step 1: Identify the aspect of the real world that is the focus of the claim.

Step 2: Identify and describe a theoretical model that should fit with the real world if the claim is valid.

Step 3: Identify a prediction, based on the model and experimental setup identified, that says what data should be obtained as evidence if the model actually provides a good fit to the real world.

Step 4: Identify the data that have actually been obtained by observation or experimentation involving the real-world objects of study.

Step 5: Reasoning: Ask if the data agree with the prediction.

  • If No: Conclude that the data provide good evidence that the model does not fit the real world.
  • If Yes: First ask, if there are there other plausible models that would yield the same prediction about the data. If the answer is ‘No,’ conclude that the data do provide good evidence that the model does fit the real world. If the answer is “Yes,” conclude that the data are inconclusive regarding a fit to the real world.

Getting students to think like scientists by using logical strategies like the ones outlined above can go a long way toward preparing many students for careers in  science, or to simply provide all students with guidance on how to think like a scientist. This is especially true when students are faced with scientific claims that don’t fit comfortably with their own worldviews. Indeed, all of our students, regardless of whether or not they will pursue careers in science, are and will continue to be faced with issues, questions, and claims that may conflict with their worldviews, or are suspect and require careful and methodical analysis.

Getting students to think like scientists is critical. If for no other reason, the public must understand how science works because the most important goal of science education in a democracy is to produce a future consensus of public policy makers and an informed electorate who have a scientific understanding of the natural world. Thus, the most important thing we can teach our students is how to think like scientists.

 

References

Coyne, J. A. (2015). Faith Versus Fact: Why Science and Religion and Incompatible. New York, Viking.

Giere, R. N., J. Bickle, and R. F. Mauldin. (2006). Understanding Scientific Reasoning, 5th Ed. Wadsworth, Cengage Learning.

National Academy of Sciences. (2008). Science, Evolution and Creationism. National Academies Press.

Starr, C., R. Taggart, C. Evers, & L. Starr. (2009). Biology: The unity and diversity of life, 12th Ed. Cengage Learning.

Strode, P. K. (2015). Hypothesis generation in biology: A science teaching challenge & potential solution. The American Biology Teacher 77:17-23.

Standard
Gradeless, Teaching Methods

What I Learned from a Year of Going Gradeless

At the beginning of the 2015/2016 school year, two colleagues and I decided to try to de-emphasize grades and focus on learning and understanding through feedback and reflection which I wrote about last August. Our approach was somewhat rooted in standards-based grading (we called them Learning Targets) with lots of teacher and student feedback and required student reflections on their learning for each curriculum unit. But also with opportunities to meet with the teacher for what we called meaningful corrections where students showed new understanding of the Learning Targets they missed on their unit exams.

At the beginning of the course the students received a guide (below) that detailed where their grade would come from. We decided that there were four core course standards that all students should be able to show progress and ultimately proficiency in during the semester: Summative Content, Formative Content, Scientific Practices, and Student Practices.

Grade Guide 2

A description of the four core course standards.

In the online gradebook students received progress scores for their efforts on the Major Assignments (80% of their gradebook-reported progress: Unit Exams, Papers, and Unit Reflections) and the Minor Assignments (20% of their gradebook-reported progress: mostly Reading Quizzes and the Lab Notebook). Students were to understand that a “Complete” (100%) in the gradebook for an assignment meant that the student may not have been perfect on the assignment, but that the effort was good enough to move on. A “Partial” (80%) meant that the student had an opportunity to improve through written reflections and meaningful corrections. A “Rework” (50%) meant that the student must redo the work and/or meet with the teacher to show enough understanding to reach at least the level of Partial. A “Missing” came through as 0% (Bob Marzano would not support the 0%). Students (constantly) and parents (not nearly enough) were reminded that, given our gradebook system, the grade the online gradebook automatically calculates is not a real grade, but a progress grade. We put raw exam and quiz scores in the comment boxes for each gradebook item, along with teacher comments about the scores. If a student did meaningful corrections or reworked an assignment, this information also appeared in the comment box. The real grade would be determined at the end of the semester by the student through an evidence-based grade claim letter and by the teacher by looking at the same evidence.

So was the course gradeless?

It wasn’t gradeless. Not even close. The fact that we had to arrive at a grade for each student by the end of the semester was always looming. Students still could be seen calculating their percent scores after exams and announcing to friends, “I got a B,” or “I got an A!” It also became clear that both some students and their parents had trouble looking at the progress score and not seeing it as a course grade. This phenomenon is going to be a hard one to overcome. More on the grade claim later.

Reflections

Reading the student reflections for each unit was sometimes fascinating and enjoyable, sometimes frustrating, but always interesting and revealing. Some students were taking the reflecting seriously and some were not. Some students were really interacting with their knowledge and understanding and some were not. Perhaps some were just better at metacognition.

However, it became clear early on that keeping up with reading and commenting on the students reflections for each unit was unsustainable. We used shared Google Docs for nearly all student writing (except exams), which simplified things a bit, but there was still too much to read. I had two sections (62 students) of this first-year biology course to keep up with plus two sections (63 students) of seniors where I was also using a version of this approach. One of my colleagues was juggling three sections of first-year students (90) but the other colleague was buried under five sections (150+ students). When I shared this burden with another colleague at my  school, a Language Arts teacher and one of The Paper Graders, he had no sympathy (he is a Language Arts teacher!) and explained to me that student reflections do not need tons of teacher feedback, just an acknowledgement to the student that the reflection has been read. The most important thing for the student is that doing any kind of reflecting is better than nothing.

Learning Target Tables. Out of this emphasis on reflections came two critical tools developed by my colleague, Kristy. One was a tool called the Learning Target Table. At the beginning of each unit students were provided a the list of the unit’s learning targets in tabular form with a space for students to show evidence to themselves before the exam that they understood the learning target. After the exam students had the opportunity to reflect on whether they were successful or if they struggled, and why. This was a student tool; we did not traditionally assign or collect these, but they could be used by students as grade claim evidence.

LT Table

Part of a blank Learning Target Table for our unit on Human Evolution.

Summative Review of Learning Targets. The other reflection tool, called Summative Review of Learning Targets, students received right after getting their exam results. Here we showed the students how each exam question aligned with each learning target. We asked the students to again reflect on how they did. This is the document students were instructed to bring when they sat with us for meaningful corrections. And like the Learning Target Tables, students could use these as part of their grade claim evidence at the end of each semester.

This tool was an extension of a backward design tool (channeling a bit of Understanding by Design from McTighe and Wiggins) we used as a teacher team to write the exams.

LT Table 2

Example of part of a student’s Summative Review of Learning Targets sheet for our unit on Biological Molecules.

Many students used work from these tools in their written reflections for each unit.

Feedback

Feedback for students came in a few forms. As any teacher (I hope) would do, I constantly moved around the room during activities and chatted with individual students and groups of students. I looked at their lab notebooks and gave them kudos for great work and suggestions for improvement. I gave them bits of encouragement and cheers on their reflections (after realizing that I just couldn’t keep up with making long comments). I made comments on progress in the online grading program that both students and parents could see. But perhaps the most important feedback I gave was when students sat for meaningful corrections after unit exams. It was here that students, as mentioned above, brought their Summative Review of Learning Targets packets and initiated a conversation with the teacher on what they struggled with on the exam, why they struggled, and what they had learned in the process of working back through the learning targets. Many students arrived fully prepared and confident and came away empowered. However, some students showed up still shaky on some of the concepts. Both types of moments were opportunities for us to get to know each student’s abilities and help students with their progress.

But not surprisingly, this most important type of teacher feedback, one-to-one teacher-student conversations, took the most amount of time. A year earlier, our school had put in place a block of time called Collaboration and Access Time (CAT). In the middle of our Block Days (Wed: Periods 1,3,5,7; Thurs: Periods 2,4,6,8) there is a 45-minute mini block where half of the teachers are available to students (Wed for Science) and half are meeting with colleagues to collaborate (Thurs for Science). On many days, usually the week after exams, I had a line of students wanting to sit for meaningful corrections and I would often work into the lunch period with them. Others would schedule to meet me during my planning periods. Students were also trying to meet with their other teachers. Even with CAT there was just not enough time to do some of the most meaningful work with students. This year we plan to try to find more class time to have these personal conversations with students.

Grade Claim Letter

Regardless of how much we tried to de-emphasize grades during the year, we still had to post a letter grade (A,B,C,D,F; our District doesn’t use + or -) for each student. In an attempt to give students even more control over their final grades, at the end of each semester students had the opportunity to write a letter to the teacher where they claimed what course grade they thought they had earned. In their letters, students provided evidence for their claims from each of the core course standards. We provided students with the guide shown below for what it might look like in each of the standards categories to claim a particular grade.

Screen Shot 2016-06-22 at 9.31.36 AM

Screen Shot 2016-06-22 at 9.31.43 AM

Screen Shot 2016-06-22 at 9.31.48 AM

It isn’t clear whether or not such a detailed and structured guide for determining a final grade was necessary. Knowing the core course standards, would students have been able to make a reasonable claim without so much guidance?

In my two sections of seniors for my IB/AP Biology course, I provided a much less detailed guide:

In your letter, please claim the level of excellence (A, B, C, D, F) at which you can justify you achieved during the Semester. Your evidence should include 1) How you did on the summative content (exams), 2) How you did on the science practices (lab work, writing, presenting, using peer feedback), and 3) How you did on your student practices (class participation, group work, sitting with me for corrections). You should also include as an appendix any Unit Reflections you wrote.

Of course, seniors may be more sophisticated and mature than freshmen and sophomores and they may not need as much scaffolding on such an ominous assignment as making a grade claim. Indeed, even with little guidance most were able to write perfectly acceptable and reasonable letters for claiming their grades.

Student Course Feedback

At the end of 2nd Semester, all students were encouraged to fill out an anonymous survey to provide us with feedback on our approach to the course. Here are some of their more interesting responses (n = 200; N = 306). For the Lickert Scale statements, 1 = “completely disagree”; 5 = “completely agree”.

Screen Shot 2016-06-22 at 10.07.58 AM Screen Shot 2016-06-22 at 10.08.08 AMScreen Shot 2016-06-22 at 10.08.14 AMScreen Shot 2016-06-22 at 10.08.04 AM Screen Shot 2016-06-22 at 10.08.18 AM Screen Shot 2016-06-22 at 10.08.22 AM Screen Shot 2016-06-22 at 10.08.28 AM

Overall, it seems that the 65% (200/306) of students in the course who took the survey, the feedback was more positive than negative for most of the items on which we asked for feedback. In general, students felt less stressed than in other courses, they felt in control of their own success, and genuinely wanted to learn the material over getting the grade. Students also understood how to achieve the grade they desired and agreed that the grade they earned reflected their learning. Of the 200 students that took the survey, 157 sat for meaningful corrections. An overwhelming majority agreed that the experience improved their understanding of the material. However, an equally important and interesting result is that less than 25% of the students agreed that having to reflect on each curriculum unit helped them grow as successful students.

Summary

So what do we do as we move forward?

  1. We continue to move in the direction of de-emphasizing grades.
  2. We continue to experiment with ways for students to feel more in control of their fates in the course.
  3. We leverage more time to interact one-to-one with students during class time.
  4. We make Unit Reflections optional but important evidence for claiming a final grade.
  5. We try to streamline the process of getting to the grade.
  6. We communicate more effectively and more often with students and parents about the difference between course progress and a final course grade.

But perhaps the most critical thing we can do (I can do) is to carefully read your comments and suggestions about what we have attempted to do in this high school science course. So, if you have read this far, please take a moment to leave a meaningful and helpful comment. We cannot do this alone.

Standard
Science Practices, Teaching Methods

Gradeless: Toward Greater Learning and Understanding in Biology

On June 1st of this year, just as second semester closed, I posted a blog entry about an email exchange I had with a student regarding his grade in my biology course. The subject line of his original email and the title of the post is “My final grade for this semester is an 89.41%.” The combination of this email exchange, my reflections as I composed the blog post, and my Language Arts teacher wife’s gradeless model in her classroom over the last three years have finally prompted me to change my approach to learning, understanding, and grades in my biology courses.

My wife, Sarah Zerwin, blogs as one of The Paper Graders and has been challenging me to deemphasize grades for two years now after the success she has had in her Senior Literature, Composition, and Communication course. Sarah’s approach is grounded in Alphie Kohn’s The Case Against Grades. However, I was skeptical. I claimed that in such a content-driven course as biology, there was no way that her feedback and reflections model would work, but I decided to try it last year in a unique course I teach called Science Research Seminar (SRS).

The objective of SRS is for students to learn about the human endeavor of scientific research, to learn how statistics fit into scientific research, and to experience planning, performing, and reporting real, original research. The course is not content-driven but there are several benchmarks for students in the course, including a statistics curriculum, two writing assignments about communicating science, a written, graduate level research proposal, a Power Point talk summarizing the research, and a manuscript-quality scientific paper. Last year, instead of giving students points and therefore grades on these assignments as I had done for years, I just gave them feedback with questions and comments and encouragement and notes like, “I’d love to see this again with some improvements! Here are some resources to help you.” I often reminded them what my coauthor, Matt Young, used to tell me when we were working on our book, Why Evolution Works: “You’re never done writing,” Matt would say kindly as he and our editor waited to review my chapters, “but there are deadlines.”

The year in SRS was a huge success and my students worked harder than ever on their writing, their science practices, and their science communication skills. Indeed, they were honored beyond my course for their hard work with many receiving top recognitions at our Regional Science Fair and the Colorado Science and Engineering Fair, with five students moving on to the Intel International Science and Engineering Fair. However, at the end of each semester, my school district requires teachers—as do nearly all public school districts—to put their students into bins labeled A, B, C, D, and F. These bins are designed to communicate to the students, their parents, future teachers, college admissions offices, and potential employers what the students know and understand about a content area.

In December, and then again in May, I had a conversation with each SRS student about where they thought they belonged given all the feedback and their responses to it. In the end, all students felt successful and worked beyond what they had thought was possible. With their presentations, posters, papers, and participation in our class community, we had the evidence to support their final grades. Indeed, each student left with a portfolio of his or her accomplishments and success.

But of course, SRS is a unique student experience where the content is mostly individualized and driven in large part by individual students. I still had to figure out how this approach would look in biology.

The Design Process

I presented the idea to my two colleagues that I team with in designing and facilitating our ten sections of a first-year biology course called Pre-IB (International Baccalaureate) Biology. We struggled for a couple of hours during a late summer teacher work day in August. We could not figure out how to fit the square peg of Sarah’s gradeless language arts model into the round hole of a science course.

So we asked for Sarah’s help.

After no more than 20 minutes of asking us questions, Sarah created a simple framework on our white board upon which we could build our own gradeless approach.

2015-08-13 12.58.43

Sarah’s first sketch.

My Colleague, Kristy, and I then worked further on the idea that afternoon and our own model began to fall into place.

2015-08-14 14.55.13

The very piece of paper Kristy and I presented to our Administration.

We described the model to our administration team and received a green light.

The Reveal

When the school year began, we presented the idea to our first-year biology students—I also am using this model with seniors in my two sections of International Baccalaureate/Advanced Placement Biology.

The three of us approached our ‘reveal’ to our students differently, but here is how the discussion went in my classes:

I asked my students to discuss for a few minutes with their table groups topics like what grades meant to them and how it felt to earn particular grades on assignments, assessments, and in courses. Several groups independently came to similar conclusions, including that grades didn’t really reflect what they had learned in a course but they were important for getting into other courses and for college. They also shared that grades could make them feel everything from great to awful to successful to frustrated and to helpless. I then steered the conversation to that final number that traditionally determines the grade. I asked my students the question, “What does an 89.41% mean about what you know and understand from a science course in general and a biology course in particular?” I followed this with, “How is that single number generated? How much uncertainty is there in each of the dozens of numbers that are used to calculate that final number that is supposed to accurately define your biology knowledge and experience?”

Next I asked, “What does it mean to get 7 out of 10 questions correct on an assessment?” Several responses ensued:

“A 70%,” said one student.

“A ‘C’,” said another

“It means you’re average,” said a third.

“It means you suck!” said a senior in each of my IB/AP Biology sections.

Yet I challenged them to think differently about a 7 out of 10.

“Why can’t it mean, ‘I nailed seven of the ten targets in this unit and I only have three to go before I have mastered the content!’?” I asked.

Of course, this is classic standards-based grading philosophy, and there are certainly components of standards-based grading in our model, but we are trying to move as far away as possible from the grade bins and toward learning and understanding the science of biology through feedback and self-reflection. Many, and perhaps even most students in our classes will earn the same letter grade they would have in a traditional points and percents approach, but our goal is that they will learn and understand and be able to do more. They will know what they know, we will know what they know, they will be able to do science and we will have evidence beyond just numbers to back it up.

The Model

My colleagues and I have agreed—with Sarah’s original insights—that the final grade each semester that we are required to provide the District will come from four core course standards: summative content, formative content, scientific practices, and student practices. The content in my courses is prescribed. The first year biology course content comes from the Colorado State standards for life science and our School District essential learning objectives. We have turned these objectives into statements called Learning Targets. The second year course content comes from the International Baccalaureate Organization’s Biology Guide’s statements called Understandings and the College Board’s Advanced Placement Biology Course and Exam Description’s statements called Learning Objectives.

Of course, being a science class, showing evidence of content knowledge is indeed important. But we asked the students to help us come up with the evidence that should be required of students in order to show that they were also ‘rockstars’ in the scientific and student practices. All classes generated similar lists that we the teachers compiled and honed. From here, we created a document that summarizes how, through shared self-reflections after each unit and teacher feedback, students can demonstrate their success in each of the four course standards.

The document provided to the students looks like this:

——————————————————————————————————————————-

Self Reflection and Feedback Based Learning and Understanding of Biology

Overview of The Plan

There are four overarching course standards in this Biology course:

  1. Summative Content
  2. Formative Content
  3. Scientific Practices
  4. Student Practices
  • You will receive written feedback from your teacher in various ways: Comments on IC (Infinite Campus – our online grading program), schoology (schoology.com – how we deliver the course), and Google Docs.
  • You will receive verbal feedback from the teacher in one-to-one conversations.
  • You will receive rubrics and check lists for certain assignments.
  • You will do and maintain self-assessments and peer-assessments on your writing.
  • You will receive scores (for example, 4/5 on quiz or 25/30 on an exam) in the comment boxes on IC.

There will be three grade categories in Infinite Campus. The categories are Major Assignments, Minor Assignments, and a Semester Final Grade.  The Major and Minor Assignment categories will be used so that you, the students, your parents, and the school have a record of what each you have completed and what needs some work. This “Progress Report” is not your course grade, but literally your progress in the course. At the end of each semester, these categories will be given a weight of 0% and the Semester Final Grade category will be given a weight of 100%. Of course, the Minor and Major Assignment categories will still be visible and relevant to show your semester progress.

Your final Semester Grade will be negotiated at the end of semester, based on how you have met the four course standards.  You, the student, will make a claim for the grade you have earned in a letter you prepare for your teacher. In the letter, you will defend your grade claim, using multiple lines of evidence (examples of evidence are listed below) and careful, logical reasoning. Below are examples of the evidence you can provide for each course standard.

Summative Content Evidence:

  1. Critically examine your scores on major assessments and reflect on your understanding of the each Unit’s Learning Targets (10-15 for each unit).
  2. How did you do on the multiple choice questions, which require a proficient understanding of the learning targets, but not generally an advanced understanding?
  3. How did you do on the free response questions, which require an advanced understanding of the learning targets?
  4. Make meaningful corrections on learning targets that need work. This work will be accomplished on a scheduled teacher access day.

Formative Content Evidence:

  1. Critically examine your scores on minor assessments, including quizzes, with meaningful corrections where necessary.
  2. Keep up with your Learning Target Reflection Tables (Pre-IB Biology only).
  3. Provide a self-reflection for each Unit.
  4. Provide reflections on other formative assignments as they arise.

Scientific Practices Evidence:

  1. Keep a lab notebook and use it regularly. Your lab notebook must be organized, clear, and labeled with no notes from other classes, and your handwriting should be easy to read.
  2. Show evidence of designing and conducting controlled experiments.
  3. Show evidence of growth in your scientific reading and writing.
  4. Show evidence that you are capable of interpreting the results of investigations. For example, provide examples of using statistics and statistical language appropriately.

Student Practices Evidence:

  1. Describe and show evidence of effective collaboration with other students, including feedback from peers.
  2. Be punctual with formal lab writing. Of course, “you are never done writing, but there are deadlines.”
  3. Submit all quizzes and assignments on schoology on time.
  4. Participate in class activities and discussions.
  5. Treat “minor” assignments as important opportunities to test your learning and understanding.
  6. Follow all safety rules and class expectations.

——————————————————————————————————————————-

The Launch

During the first weekend of school, each student began their evidence gathering by creating a folder in Google Drive that is shared with their teacher. Students then created a document in that folder (that by default is also shared with their teacher) called (Student Name) Semester 1 Course Reflections. After reading through the course expectations document that is provided to students every year, the students wrote a letter to their teacher in their shared documents where they made comments and asked questions about the expectations and shared anything unique about themselves that they wanted the teacher to know (e.g. I’m a dancer, I play soccer, I hate science, I love biology, etc.).

At the end of each content unit (or any time during a unit) each student will write a reflection on each of the four course standards and also provide evidence of their work toward each of the standards.

An On-Going Experiment

Of course, this self-reflection and feedback-based approach to learning and understanding biology will be an ongoing experiment. As we progress through the year, we will tweak the system, check in with students, and consult with colleagues. But I do know that two weeks into the semester, students seem relaxed, motivated, and cheerful, and more hands are raised in a class period than over an entire week of school.

Capture 2

An IB/AP Biology student’s first reflection after our very short Unit 1.

Standard
Teaching Methods

My final grade for this semester is an 89.41%

At the end of this short post is an email exchange over the last few days between me and a student of mine about his final grade in my biology class. I post this, not to make fun of the student, or to celebrate our traditional grading practices, or to showcase my skills at making an evidence-based argument. I post this because we are prisoners of the letter grade model. Both I and the student were stuck talking about the grade. This entire conversation, indeed, the focus for the entire course the student had just completed should have been about learning, not grading.

I have begun moving away from grading and toward learning in my Science Research Seminar course. In the course, students receive no grades, but only written and oral feedback on their science methodology, their understanding of statistics, their class presentations, and their overall science communication and literacy skills until the end of each semester (The District still requires that I post a letter grade of some sort). The letter grade is agreed upon by way of a conversation between each student and me. It is astonishing how hard each student works throughout the two semesters. At the same time, it is equally as astonishing how hard I work at giving feedback. This mostly gradeless model works in this nontraditional course, but I have yet to come to a sophisticated understanding of how to move away from traditional grading in my more traditional courses.

Other teachers are way ahead of me.

Three of my colleagues (not in science, but in my school’s Language Arts Department) blog as The Paper Graders. These teachers have been writing about grades for the last three years. One of The Paper Graders, Mr. S., began the dialogue three years ago when he wrote about changing the conversation with parents from grades to learning. Another Paper Grader, Doc Z, began experimenting with going gradeless in her classroom two years ago. She has written about her gradeless experiment here. Her course is standards based and because ultimately a grade must be submitted for each student, she and the students come to an agreement about what an A, B, C, etc. should look like. Indeed, Doc Z argues

“No more grades on writing. No more grades on any individual tasks. Feedback only. Keep students focused on the work and the learning and the process and not the grade. Enter one grade–the one that’s necessary for the transcript at the end of the semester, but negotiate with each student individually to determine what that will be. And that’s it.”

I need to move in the direction that she is going in my other courses, but the conversation below kept both the student and me, the teacher, from seeing beyond the As, Bs, and Cs. I failed and the student continues to argue for an A, “If nothing else, simply based on the numbers.”

Why did I fail?

We cannot blame the student for focusing on the numbers and these big bins into which we put our students’ entire experiences in our courses. In the case illustrated below, much of the blame is on me. Not only did I fail to steer the conversation away from the grade, I must have failed to make it clear to all of my students in this course that focus was on learning and understanding the content and learning and understanding how the science of biology works. If there was no letter grade, would the student have been satisfied with the still imperfect but a little more precise (yet still burdened with an unknown margin of error) 89.41%?

In the end, the student emerges from the conversation as a winner, not satisfied but perhaps with a smile on his face as he writes, “I wish you a great summer! I really did get a lot out of your class.” Indeed, I commend the student for advocating for himself and for forcing this larger conversation, even though that was not his original intention.

————————-

Dr. Strode,

My final grade for this semester is an 89.41% and I know the science department is usually very strict about rounding up grades, but I feel that I worked extremely hard this year and deserve that extra 0.6%.

Thanks for a great year,

Student

————————-

Hi Student:

I appreciate that this email came from you and not a parent. I also appreciate the role you played in the class and I agree that you did work hard. However, simply saying “[I] deserve that extra 0.6%” is not the kind of evidence based argument that is required in this case.

The Final Exam is a summary of your overall knowledge of the material from the semester. Your score of 48/60 is an 80%–the lowest B you can get–and two points below the mean for both of my Pre-IB Biology sections. Your Unit Exam scores sit at a mean of 86%, and include two As, three Bs, and a C. These exam scores are 65% of your total grade in the course.

The score we put forward for you in any course must be accurate, not inflated, and backed by evidence.

Now, if you did indeed arrive at your final percent in the course with maximum effort, which is what I interpret your email to imply, then a B in the course is accurate. If the B was not our best work, then this should be great motivation for the fall as you move into your next set of courses.

If you would like to discuss this further and in person, I will be at school on Tuesday.

I appreciate the contributions you made in class this year.

Sincerely,

Dr. Strode

————————-

Dr. Strode,

I look forward to chatting with you on Tuesday! What time will you be able to meet? I just want to clarify that I completely understand why you would be hesitant to give me the bump, but also please consider that the grade I have right now will go in on my transcript as a B and in the big picture, it is actually so close to an A. As a B on my transcript, it could appear that I had received as low as an 80% in the class, and that is obviously not true and not an accurate reflection on my hard work in this difficult class.

Thanks,

Student

————————-

Hi Student:

You’re still welcome to come in, but I think we can save us both some time. You’ve got an impossible sell on your hands and there is no way I could justify to my colleagues inflating your second semester grade grade to an A. Here is why:

1) You walked the A-B line at 91% all Semester.

2) You did not pass (earned a 55%) one of the Unit 7 Human Physiology quizzes.

3) You earned a C on the Unit 7 Human Physiology Exam

4) You and your group chose not to rewrite your Final Project Design.

5) You chose not to retake your Unit 5 Multiple Choice Exam, on which you earned an 80%.

6) You earned an 80% (48/60) on the Final Exam, two points below the mean of 50.2 for the 61 students in my two Pre-IB Biology Sections.

7) Your overall exam score (Unit Exams and Final) is 83%. This means that 100% of your summative assessments, the work you did alone with no assistance from others and 65% of your grade, is a B-.

Number 4, 5, and 6 above are probably the three most important learning experiences you can take forward into your Junior Year:

  • Always take advantage of rewriting your work.
  • Always take advantage of retaking exams when the opportunity is provided.
  • Always bring your best effort to final exams. This is your opportunity to show that you have learned the material and that your final grade in a course is accurate.

I think you can be proud of the contributions you made in my class and the excellent work you did in your group.

Sincerely,

Dr. Strode

————————-

Dr. Strode,

I understand what you are saying and I am fully aware that I’m not going to get this bump but I just wanted you to know that my group and I DID rewrite our final project design. We had originally received an 18/20 (an A) and during class work time, I came to your desk multiple times to ask you how to fix the errors that you had commented on. After that class period, we had addressed and revised all of the comments you made that were in need of improvement and I specifically told you we had done so for you to go back and regrade it accordingly but you never did. I just wanted to make sure you knew that I know it is important to seize the opportunities for rewriting work. I would never leave a grade at which I know I can do better.

Just a thought… If you had actually gone back and given us the 20/20 on the final project design since we actually did revise it to where you told us it was good in class, would my grade be an A?

Thanks,

Student

————————-

Hi Student:

I do see that you last modified your Design on May 17th, but only now have you indicated that I never rescored it. I honestly don’t remember you telling me that you had made any improvements. I asked everyone who did a rewrite to either send me an email or re-share the Google Document with me. I have changed your score to a 20/20.

At this point, this conversation needs to end. The end of the semester is just not the time to be arguing with a teacher about a grade in a course when a student has had ample time to have that dialogue. This conversation should have occurred after Unit 7 when keeping your A in the class started looking bleak… and then there was the final exam.

You really need to move on from this and focus on having a good summer.

Sincerely,

Dr. Strode

————————-

Dr. Strode,

This will be the last email I write to you regarding this matter but now that my grade is an 89.55%, wouldn’t it be logically implied that the decimals round up to 90%? I’m not trying to be annoying but this is important to me, and I believe that the accurate representation of my work merits an A. If nothing else, simply based on the numbers. And I would like you to think about at what point you would ever round up to the nearest whole number. You do not need to respond, I wish you a great summer! I really did get a lot out of your class.

Sincerely,

Student

Standard