If college professors spent less time lecturing, would their students do better?
A three-year study examining student performance in a “flipped classroom” — a class in which students watch short lecture videos at home and work on activities during class time — has found statistically significant gains in student performance in “flipped” settings and significant student preference for “flipped” methods.
The study, provided exclusively to The Atlantic, is one of the first to examine a “flipped” classroom in the current state of its technology. Russell Mumper, a Vice Dean at the University of North Carolina’s Eshelman School of Pharmacy, conducted the study, and two separate articles based on its findings are now in press in the journals Academic Medicine and The American Journal of Pharmaceutical Education. The education technology company Echo360, whose technology was used in the classes examined, funded the study with a $10,000 grant.
The study examined three years of a foundational pharmaceutics course, required for all doctor of pharmacy (Pharm.D.) students attending UNC. In 2011, Mumper taught the course in a standard, PowerPoint-aided lecture format. In 2012 and 2013, he taught it using “flipped” methods. Student performance on an identical final exam improved by 2.5 percent between 2011 and 2012—results now in press at Academic Medicine—and by an additional 2.6 percent in 2013. Overall, student performance on an identical final exam improved between 2011 and 2013 by 5.1 percent.
Students also came to prefer the flipped model to the lecture model. While 75 percent of students in 2012 said, before Mumper’s class, that they preferred lectures, almost 90 percent of students said they preferred the flipped model after the class.
“As I always like to say, we flipped their preference,” Mumper told me. “They went from largely wanting and valuing lectures to just the opposite.”
Comments from a student echoed that change in preferences. “It was a little hard to get used to begin with,” Natalie Young, a second-year doctorate of pharmacy student at UNC said, of Mumper’s class. “But then, as I got going with it, I realized that [the “flipped” class] was actually facilitating my learning.”
“Overall, I'd say the study adds a useful contribution to the growing literature on flipped instruction and active learning in higher education,” Justin Reich, a Harvard researcher, said about its findings. The study was “about as good a quasi-experimental study as could be done,” he added on the phone.
* * *
So: In one setting, in one class, over 3 years, student performance improved in a statistically significant way in a flipped classroom model. That’s the news. Educational studies, especially studies of this type, are difficult to place into context. They are quasi-experimental. Their samples are determined by circumstance, not by random assortment. And, as here, they’re often underwritten by private companies, with agendas of their own.
Not that that’s a bad thing. Education rises and falls in specificity, and if digital educational technology ever permeates a classroom in a big way, college or otherwise, it will be a specific technology, likely sold by a specific company. Every use will be used in a specific setting. So it’s worth examining Dean Mumper’s class to see how technology specifically shaped it: how “the Internet,” a large and nebulous thing, specifically changed the experiences of his students.
* * *
In 2011, Mumper was teaching Pharmaceutics to 153 first-year UNC Pharm. D. students. Pharmaceutics is “basically the discipline of dosage forms and pharmaceutical dosage forms and delivery systems, so it’s core to our school,” he says. Students came to his class twice a week, for an hour and a half, heard Mumper’s lecture, and then read an introductory textbook between classes. They took two midterms and a final. Mumper had taught the class for almost a decade, and he could predict — very, very well — how each class would do, on average, on that final: They’d score near 80 percent.
And in his mind, the class was hitting a wall.
“There’s so much content, and so many places to access that content, that if our burden, our challenge, as instructors is to relay that content, there’s never enough time,” he told me. “And if your interaction is solely based on PowerPoint slides,” he added, “[students] are no longer paying attention. They’re distracted.”
So, with the help of Echo360, he devised a flipped model for the classroom.
In 2012, that flipped model looked like this:
At home, before class, students watched brief lecture modules, which introduced them to the day’s content. They also read a textbook — the same, introductory-level book as in 2011 — before they arrived.
When they got to class, Mumper would begin by asking them “audience response” questions. He’d put a multiple-choice question about the previous night’s lectures on a PowerPoint slide and ask all the students to respond via small, cheap clickers. He’d then look at their response, live, as they answered, and address any inconsistencies or incorrect beliefs revealed. Maybe 50 percent of the class got the wrong answer to one of these questions: This gave him an opportunity to lecture just enough so that students could understand what they got wrong.
Then, the class would split up into pairs, and Mumper would ask them a question which required them to apply the previous night’s content, such as: “Given your knowledge of the skin and transdermal delivery, describe how you might treat this patient who had breakthrough cancer pain.” The pairs would discuss an answer, then share their findings with the class. At the end of that section, Mumper would go over any points relevant to the question which he felt the class failed to bring up.
In 2012, in class’s final section, a team of students would give a 10 minute presentation to the rest of the class based on the previous night’s reading, then lead a discussion about it. Mumper would then conduct a quiz.
Student presentations were students’s least favorite part of the class, and in 2013 they was replaced. In 2013, too, Mumper removed the introductory reading and replaced it with a “carefully selected” clincal study about the day’s topic. Instead of presenting (or taking a quiz, which was relegated to after-class online work), students read a clinical study and discussed it together and then as a class.
Then, they’d go home, watch the lecture-modules, do some reading, come back in and do it all again.
“To be honest, upfront, there was a lot of, I don’t know, I guess a little bit of complaining,” Natalie Young, the Pharm.D. student, told me, “because we just were used to just going to class and not having to do so much preparation for the class.”
“And with this,” she said, “you actually have to do reading or watch the [lecture modules], you actually have to prepare for the class.”
She added, too, that Mumper’s course succeeded because of how well he implemented the technology and flipped structure. “In terms of execution, I’ve had professors who were kinda committed to this way of teaching, and so they would halfway do it.”
“So it was kind of awkward,” she said. “You’d sit there and you’d have the first hour or 45 minutes of lecture, then they’d want you to get into small groups, and they couldn’t figure how they wanted the groups. In some classes, it didn’t work so well, because the professor didn’t have a set structure that they wanted to follow and stick to.”
“What I immediately realized was, not only were they getting the content, but they were applying it, and then either bringing new content to the classroom -- content I wasn’t aware of -- or they were asking me questions about the application of the content, questions that provided a richness that we could never explore when I was too busy lecturing,” Mumper said.
He added that the flipped classroom provided a particularly good opportunity for research universities to make good on their latent potential.
“Many people outside the university look at a university and don’t make the connection between its researching mission and teaching missions,” said Mumper. “The flipped classroom, in real time and in a real way, allowed me to realize and reinforce the connection.”
* * *
So what does the study mean? I asked Justin Reich, a Harvard researcher and Berkman Center fellow, and author of the “EdTech Researcher” blog at Education Week magazine.
“The study is well done for a quasi-experimental, year over year study,” he wrote to me. “As the authors note, a randomized trail would be better, because we cannot definitively attribute the differences in outcomes to differences in course experience.” But he added, because Mumper’s Pharmaceutics course was required for all UNC Pharmacy students, the findings were “relatively robust … because everyone has to take the class and the cohorts are so similar year to year.”
While studies about flipped or blended learning have been conducted in the past, Reich said that this was “certainly among the first” studies to examine a “flipped classroom” in its current idiom, in which lectures were condensed into time-shifted, small video modules and students did these kind of “active learning” activities in the classroom. “Flipped learning has come to have a fairly particular meaning for many folks,” he said, and this study fit that meaning.
He wondered, though, whether students learned more simply because they did more work. “Now, by all means, I'm all for making students work more,” he wrote. “And they managed to make students work more, without negatively impacting how much they liked the class.” But he doubted the improvements could scale to the rest of the UNC curriculum, or to any college’s curriculum.
When I asked Dr. Mumper about this, he said that, while he assigned more work, exit surveys showed that students did the same amount of work. Because more of the work was upfront, students crammed less, so they wound up devoting the same amount of time to the class.
Students also had access to full video recordings of the class, thanks to Echo360’s technology. In fact, UNC Pharmacy students have access to video recordings of almost all their classes.
Natalie Young described this technology as instrumental in her education. “I’m a mother, and have a four year-old, and, having a child, there are times when your child’s gonna be sick, and you can’t attend class,” she said. “I truly believe that, without a shadow of a doubt, that if they had not implemented Echo360 at my school, I would not be able to get my doctorate in the first place”
Echo360 is happy to take the credit. “The main event in education is still, and will continue to be, in the classroom,” said Fred Singer, the company’s CEO. “Technology like Echo360 is making a significant impact on today’s classroom experience for the millions of students across the globe.”
“A 5 percent increase in academic performance proves active learning will play a big role in education’s future,” Singer said about the study’s results.
* * *
And perhaps it will. At the end of our conversation, Reich asked whether we were seeing the end of technology-enabled teaching gains. “Should we expect even greater gains in future years as instructors refine their methods?”
He continued, speculating on our current ed-tech ecosystem. We’re surrounded by new digital technology, and companies are leaping in to try to apply it to the classroom. Reich wonders whether they asking the wrong questions. Are they starting from the wrong place?
“Are we seeing the limits of teacher-centered content delivery?” he wrote. “With all of the technologies now available to us, should we be content with a two percent gain on final exams?
