{"id":5015,"date":"2014-10-06T14:14:48","date_gmt":"2014-10-06T12:14:48","guid":{"rendered":"https:\/\/oeb.global\/OEB_Newsportal\/?p=5015"},"modified":"2016-11-14T12:19:00","modified_gmt":"2016-11-14T10:19:00","slug":"online-assessment-tools","status":"publish","type":"post","link":"https:\/\/oeb.global\/oeb-insights\/online-assessment-tools\/","title":{"rendered":"The robo-readers are coming"},"content":{"rendered":"
\"Grading\"<\/a>

Image: Quinn Dombrowski<\/p><\/div>\n

They cannot \u2018know\u2019 students, think outside the box or reward creativity or innovation. Their feedback is pragmatic, standardised and \u2013\u00a0some would say \u2013\u00a0soulless.<\/strong><\/p>\n

 <\/p>\n

Despite this, according to recent studies<\/a>, students would rather submit early drafts of their essays to robo-readers than human teachers.\u00a0<\/strong><\/p>\n

 <\/p>\n

By Steven Blum<\/em><\/p>\n

 <\/p>\n

While a teacher\u2019s feedback to an essay is often viewed as \u2018punitive\u2019, in ways that discourage multiple revisions, professors say that students are willing to revise their essays multiple times when their work is being reviewed by a computer.<\/p>\n

 <\/p>\n

\u201cThey end up writing nearly three times as many words in the course of revising as students who are not offered [automated essay scoring software]\u201d, according to an article in The New Scientist<\/a>. As a result, the quality of their writing improves.<\/p>\n

 <\/p>\n

The studies suggest that the impersonal nature of essay grading software is the software\u2019s greatest strength. The less personal the feedback, the freer students feel to try out new things and submit draft after draft.<\/p>\n

 <\/p>\n

 <\/p>\n

But can robo-readers give more than perfunctory feedback? Can they assess meaning and voice, tone and subjectivity?<\/p>\n

 <\/p>\n

It depends on who you ask.<\/p>\n

 <\/p>\n

Can Software Learn to Read?<\/strong><\/p>\n

 <\/p>\n

Today\u2019s robo-readers use artificial intelligence to \u2018learn\u2019 how to grade essay questions. First, thousands of student essays are scored by hand and loaded into the system; then the software learns what components make an outstanding essay.<\/p>\n

 <\/p>\n

This is helpful, especially for those who teach Massive Open Online Courses (MOOCs) and can\u2019t possibly hope to grade hundreds \u2014 even thousands \u2014 of essays in mere days.<\/p>\n

 <\/p>\n

\u201cThe idea is to create a tireless, automated version of the professor that can give feedback on a much broader amount of work,\u201d explains Piotr Mitros<\/a>, chief scientist at edX, an extensive MOOC platform with backing from Harvard, MIT and UC Berkeley.<\/p>\n

 <\/p>\n

The problem is that the grading software on the market is of varying quality. While some can be helpful, others punish student creativity by favoring a vernacular that closely adheres to a secret sauce of jargon. Meaningless jumbles of words triumph, so long as they contain a few key words.<\/p>\n

 <\/p>\n

Les Perelman, former director of undergraduate writing at the Massachusetts Institute of Technology, is one of the most outspoken critics of automated assessment software.\u00a0 Since last year, his petition against robo-readers \u2014 Professionals Against Machine Scoring Of Student Essays In High-Stakes Assessment \u2014 has received over 4,000 signatures, including Noam Chomsky\u2019s.<\/p>\n

 <\/p>\n

Along with a team of students from MIT and Harvard, Perelman has developed a software program called \u2018Basic Automated B.S. Essay Language Generator\u2019 (or Babel for short) that generates essays from scratch using keywords.<\/p>\n

 <\/p>\n

The software draws attention to the fact that simply arranging a string of keywords in an otherwise meaningless sentence can create what a computer would call an A+ essay.<\/p>\n

 <\/p>\n

Perelman has shown that even Abraham Lincoln would have received a poor grade for the Gettysburg Address.<\/p>\n

 <\/p>\n

Satisfying an Algorithm<\/strong><\/p>\n

 <\/p>\n

From refrigerators that order food to cars that drive themselves, our world is becoming more automated every day. It makes sense that we would demand computers take up the painstaking task of grading essays as a next step.<\/p>\n

 <\/p>\n

But, critics say, before we turn over this process to an algorithm, there are both practical and existential concerns to consider.<\/p>\n

 <\/p>\n

Detractors fear that robo-readers will make writing a soulless enterprise.\u00a0 \u201cHow will it change the way students write when they\u2019re no longer writing to communicate with a human but to satisfy a computer algorithm?\u201d reads one comment below an article on computer assessments.<\/p>\n

 <\/p>\n

Still others believe that the software will diminish the role of the teacher.<\/p>\n

 <\/p>\n

\u201cSurrendering one\u2019s professional responsibilities will [also] be good practice for the day when professors will be entirely replaced by computers,\u201d writes Kathleen Anderson<\/a> in the Chronicle of Higher Education.<\/p>\n

 <\/p>\n

Ann Marcus Quinn, an upcoming speaker at ONLINE EDUCA BERLIN 2014, studies the use of online assessments at the University of Limerick. She believes online assessments can be helpful for giving \u201cconstructive feedback to students in a much more efficient manner\u201d; but doesn\u2019t believe their usefulness extends to analysing essays. \u201cI don\u2019t think, in my experience, that an essay can be comprehensively assessed using automated assessment,\u201d she says.<\/p>\n

 <\/p>\n

Finding the Balance<\/strong><\/p>\n

 <\/p>\n

It\u2019s easy to argue for an ideal educational environment where every student essay is carefully read, graded and commented upon by a highly trained, sensitive and compassionate instructor. And at many liberal arts colleges, this is exactly what students receive. But what about those who cannot afford such lavish attention?<\/p>\n

 <\/p>\n

Advocates for the grading software say it can be useful for getting immediate feedback on essays and turns the revision process into a fun game. \u201cStudents naturally gravitate toward resubmitting the work until they get it right,\u201d according to Daphne Koller<\/a>, a computer scientist and a founder of Coursera.<\/p>\n

 <\/p>\n

The software is well-needed in a MOOC environment where essay grading is costly and near-impossible to coordinate. It seems to reduce inhibitions, and could even help students\u2019 who are shy or afraid of criticism to grow as writers.<\/p>\n

 <\/p>\n

Just don\u2019t let the robots give the final grades.<\/p>\n

 <\/p>\n

You can hear more about the latest online assessment tools at <\/em>ONLINE EDUCA BERLIN 2014<\/em><\/a><\/p>\n

 <\/p>\n

Image attribution:\u00a0Quinn Dombrowski<\/a><\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

They cannot \u2018know\u2019 students, think outside the box or reward creativity or innovation. Their feedback is pragmatic, standardised and \u2013\u00a0some would say \u2013\u00a0soulless.   Despite this, according to recent studies, students would rather submit early drafts of their essays to robo-readers than human teachers.\u00a0   By Steven Blum   While a teacher\u2019s feedback to an […]<\/p>\n

<\/div>
<\/div>
<\/path><\/svg><\/span><\/a><\/svg><\/span><\/a><\/path><\/svg><\/span><\/a><\/path><\/svg><\/span><\/a><\/path><\/path><\/svg><\/span><\/a><\/path><\/svg><\/span><\/a><\/div>
<\/div><\/div>
<\/div>","protected":false},"author":26,"featured_media":5016,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[209],"tags":[504,128,503],"class_list":["post-5015","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","tag-assessment","tag-e-learning","tag-online-assessment-tools"],"_links":{"self":[{"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/posts\/5015","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/users\/26"}],"replies":[{"embeddable":true,"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/comments?post=5015"}],"version-history":[{"count":4,"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/posts\/5015\/revisions"}],"predecessor-version":[{"id":6780,"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/posts\/5015\/revisions\/6780"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/media\/5016"}],"wp:attachment":[{"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/media?parent=5015"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/categories?post=5015"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/oeb.global\/oeb-insights\/wp-json\/wp\/v2\/tags?post=5015"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}