University of Illinois Urbana-Champaign

Tasker: Safely Serving Verifiable Micro-tasks for Researchers

Jeffrey, Monique; Whiting, Mark; Kozolan, Abrere S.; Motorin, Imauri; Samuel, Osei-Afriyie

Content Files
Jeffrey-Monique_20180417_V01.pdf

Permalink

https://hdl.handle.net/2142/100240

Description

Title
Tasker: Safely Serving Verifiable Micro-tasks for Researchers
Author(s)
  • Jeffrey, Monique
  • Whiting, Mark
  • Kozolan, Abrere S.
  • Motorin, Imauri
  • Samuel, Osei-Afriyie
Issue Date
2018
Keyword(s)
  • crowdsourcing
  • Amazon Mechanical Turk
  • research tools
Date of Ingest
2018-07-12T15:28:21Z
Abstract
Paid crowdsourcing removes many traditional boundaries in conducting participant based research, however with this new tool, new instrumentation challenges have arisen for researchers. Three common challenges include: the difficulty in creating large numbers of high quality and novel tasks, verifying results of the tasks without relying on manual cheat mitigation techniques, and ensuring that the tasks adhere to the latest visual and instructional design to get high quality results. These circumstances endanger current and future research on Amazon Mechanical Turk and can result in compromised data. We introduce Tasker, a secure system architecture for serving unique tasks supported by usability principles to workers, and providing verification information concerning their completion and accuracy to researchers. This poster discusses insights from our pilot study and explorations toward methods that demonstrate a marked improvement for speed, security and robustness in developing tasks for research leveraging Amazon Mechanical Turk.
Publisher
iSchools
Series/Report Name or Number
iConference 2018 Proceedings
Type of Resource
text
Genre of Resource
Conference Poster
Language
eng
Permalink
http://hdl.handle.net/2142/100240
Copyright and License Information
Copyright 2018 is held by Monique Jeffrey, Mark Whiting, Abrere Sophie Kozolan, Imauri Motorin, Osei-Afriyie Samuel. Copyright permissions, when appropriate, must be obtained directly from the authors.

Owning Collections

iConference 2018 Poster Descriptions PRIMARY