Crowdsourcing_Workflows_With_Microdiversions-Dai
#notesFromPaper Authors: Dai Tags: crowdsourcing citizen science cairns mturk
- participants get bored after a while
- if they think it’s for a charity, they do better
- strategy advice as well is helpful
- tasks can cause poor performance not by being taxing, but by causing boredom
- that said workers in mturk are allowed to switch tasks, so it’s not a problem for them
Research questions:
- Do cairns increase worker retention?
- workers are free to swap tasks, and do so quickly. Is the cognitive cost of switching a task more than a cairns?
- Performance benefits?
- Are different tasks affected differently?
If someone stays, they’re considered engaged, since they could have just left
Wikipedia article quality rating, Freebase entity merging and image labeling are the tasks people were given
Should diversions be a) different tasks b) forced breaks c) games?
They chose (c), though (a) and (b) could be #futureWork.
One diversion was a dice game where you can bet some of the money you’ve made so far.
Another was a story, split out throughout the mega task
Users could skip over tasks, they came approx. every 10 minutes
Wikipedia rating? Workers had 20% more retention with story and game
Freebase entities? Story had a similar impact, but game did nothing, probably because FE was too cognitive
Image analysis was the most interesting, less retention at first, but more retention over time.
Microdiversions (cairns) don’t affect task accuracy
what’s the interaction between task type (heavy cognitive load or not) and cairn type? Looks like similar task types probably work better
