As a rule, teachers are enthusiastic about forming young minds and putting their energies toward bettering the next generation.
As another rule, teachers hate marking, and they hate it with the heat of a thousand blazing exam papers.
James Colliander, a professor at the University of Toronto, found himself staring at about 5,000 pages of papers from a national math exam. Traditionally, a cadre of markers would sit around a large table for marathon grading sessions, assembly line style, each one tackling the answer to one question before passing it on to the next marker.
Mr. Colliander hacked together an expedient: He scanned the pages into a software framework and distributed them to markers digitally. He was essentially able to parallelize the marking process.
“The markers didn’t all have to be in the same place, so they could move much faster,” says Lyssa Neel, COO of Crowdmark, the company that, with Mr. Colliander as CEO, has brought the idea to market.
Crowdmark is an online service that takes the idea of distributed marking and scales it to an institutional level.
Before they give a test, instructors sign up from a Crowdmark account. Once on the site, they’ll find templates they can use to build their exams. Each page is marked with a QR code – students only write their names on the front page. After the test has been collected, the papers are sent to Ricoh, which Crowdmark has contracted to do some industrial-scale scanning. (The company says 10,000 pages can be scanned and returned the same day, or by the next day, for an evening exam.)
Once the exams are digitized, the software breaks each exam apart into its constituent questions. The user sees a grid: Each row is a student’s exam, and each column is a question, so each square is one answer to one question. The marking load can be shared between as many team members as they like, who can tackle their share of the marking wherever and whenever they like. The software handles the totalling of the final grades, which can then be exported to learning management systems.
Crowdmark says this can shave 50 per cent to 70 per cent off the time of grading, not by reducing the amount of time that markers spend interacting with students’ work, but by cutting down the logistical overhead of dealing with large piles of papers.
Even for solo markers, there are upsides: The papers are all graded anonymously, reducing the subtle biases that can creep into any classroom setting. And since markers end up giving the same feedback regularly, the software allows teachers to store and re-use common phrases.
Ms. Neel, who holds a doctorate in computer science, was running an IT incubator at Toronto’s MaRS Innovation, and she encouraged Mr. Colliander to apply with his idea, later making a substantial seed investment in the firm. Ultimately, she jumped ship and joined the company as COO.
The idea is, in many ways, a modest proposal: It doesn’t require the re-engineering of classroom structures, or the upending of traditional pedagogy. Nor does it stray into the disquieting territory of software that is learning to grade subjective and fluid work, such as essays.
Ms. Neel says Crowdmark is targeting a market in which big class sizes at bigger universities – and open online courses that can host thousands of students – create an opportunity for software that can help maximize the time humans spend grading student work.
“We have a way to scale the human element,” she says.