Piotr Mitros, edX’s chief scientist, offered the example of a chemistry problem in which students are asked which factors determine whether a particular chemical solidifies into a glass or a crystal. (The answers are “the rate of cooling” and “the complexity of the molecule.”) After being trained on the various ways in which students actually phrase these ideas, ORA can assign points according to a rubric and even provide feedback or hints to students who missed one component or the other.
Similar processes can be used for short-answer questions in a variety of disciplines. They can even be adapted to grade essays, which requires the computer to look at things like essay length, sentence length, vocabulary level, and punctuation.
Mitros, however, is careful to point out that no one—least of all edX—seriously believes that automated grading can fully replace a live instructor. “Human grading and automated grading aren’t in conflict,” he said. “It’s less a question of ‘Will machines grade instead of humans?’ It’s more a question of ‘When do you use machine grading versus when do you use human grading?’”
He continued: “If you asked me, ‘Do you want a school experience where every single piece of text is graded by machine?’ I’d say, ‘That’s a straw-man. Nobody is proposing that.’” Further, Mitros is happy to concede that there are certain types of assignments—like literary criticism—that machines are very poorly suited to evaluate. “They just aren’t capable of actually weighing the quality of a literary argument,” he said.
In a surprisingly poignant conversational turn, Mitros also emphasized that, however advanced machine learning may become, there is no substitute for real human concern and compassion. “Closeness to teachers,” he said, “really does help student outcomes. If I know somebody’s going to look at it, I’m going to do a better job. Machines are never going to replace the need for the human connection—the idea that I created something and someone cares about it, someone cares about me.”
MOOCs may lack a certain human dimension, but there is a sense in which they are brilliantly democratic. The classes offered through edX (which are—and hopefully always will be—free) are designed to bring content from storied institutions like Harvard and MIT to the masses. Unlike Coursera or Udacity, edX is a non-profit that receives most of its money from its university partners, charging only for verified certificates. The university partners, meanwhile, receive promises of future revenue generated from several sources with which edX is experimenting, including charging fees for verified certificates, licensing course content to other institutions, and offering executive education. For the time being, however, companies like edX are simply making elite-level courses available for free to people all over the world.