Skip to main content

Managing U of T prof brings students into the grading process with new software

Prof. Joordens, professor of psychology at University of Toronto, developed a piece of software called peerScholar, an online service that makes it practical to use peer assessment as an anonymized, averaged grading tool

It's easy to ask students to complete assignments. Getting them to think critically about the work they've done is a different matter entirely.

This is a challenge in small classrooms as well as very big ones. Steve Joordens, a professor of psychology and the director of the Advanced Learning Technologies Lab at the University of Toronto's Scarborough campus faces first-year classes of no less than 1,800 students, and was looking for a way to make the process of assessing papers less of a rote, one-way activity, in which the work was done, a grade is assigned – and, with any luck, glanced at by the student – and that's the end of it.

One way of doing this is by bringing students into the grading process. Peer evaluation is a time-honoured tactic, but it has its logistical drawbacks. Prof. Joordens' solution was a piece of software called peerScholar, an online service that makes it practical to use peer assessment as an anonymized, averaged grading tool.

Story continues below advertisement

The software lets students submit their assignments digitally. Then, their work is anonymously distributed to a number of peers – say, five or six. Each of those peers gives a grade, but also feedback, which is a skill unto itself that universities don't tend to teach.

"People are very good at criticizing, but they're not very good at doing it in a way that suggests a positive approach," says Joordens.

With peerScholar, giving feedback becomes part of the course, and students give each other grades on the quality of feedback they receive. Joordens says that his lab's research indicates that peer grading is a reliable substitute for expert opinion. Students who don't like the mark their peers assign can ask to have their work re-graded by a teacher, though Joordens cautions them that mark will be final. Some students see their marks go up, but others see them drop. In fact, Joordens says that in the end, the average mark change for re-graded papers is nothing at all.

"The average of a peer grade is very comparable to the grade a single graduate-level TA would give," he says.

The software's origins date back through the last decade at U of T, but only now are its creators and owners finalizing a licensing model that would allow them to effectively sell to other institutions. Joordens has licensed the system through Pearson Canada – a large textbook distributor – and, after some years spent working through the challenges of being a small part of a big company, the software is entering the market. It's already in use at schools like Queen's and University of British Columbia in Canada, and at several American colleges. They are also marketing a version for primary and secondary schools under the brand name "Cogneeto."

To Joordens, the software is part of a broader project to teach cognitive skills that go beyond acquiring and retaining knowledge. "The analogy I like to give is, you can go to a one-hour seminar about karate. You can learn a lot about it. But if you want to learn to do karate, you cannot learn to do karate in an hour. You need a framework where you can practice."

Not everyone saw it that way. At peerScholar's own home at U of T, the union representing the university's teaching assistants viewed peer assessment as a form of "cheap labour," and launched complaints against Joordens. When an arbitrator sided with the union, Joordens modified the software so that students would provide feedback prior to getting an actual grade from teaching assistants. So far, the episode has yet to repeat itself at other schools.

Story continues below advertisement

While the software was designed for traditional written assignments, the evolving educational landscape is seeing innovative uses for it. A forthcoming class will ask students to submit audio instead of written assignments – they'll conduct oral presentations by speaking into their cellphones, and the audio will be distributed to peers for evaluation just as text would be. And, in another project, Joordens recently used it to grade a massively open online course with 65,000 students.

"The bolts stayed on," he says.

Follow us @GlobeSmallBiz, on Pinterest and Instagram
Join our Small Business LinkedIn group
Add us to your circles
Sign up for our weekly newsletter

Report an error Editorial code of conduct
Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

Comments that violate our community guidelines will be removed.

Read our community guidelines here

Discussion loading ...

Cannabis pro newsletter