WinterMilestone 1 RenanCastro

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

Overall I didn't like the experience on Mechanical Turk. It doesn't feel like the platform tries to help me become a better worker. It simply lists HITs in a non-friendly manner and leave all the work to me. Here are some reflections about it:

  • What are the motivations of workers to use the platform? Are they looking exclusively for money or is there any other reasons?
  • Why call it HIT? Would TASK have the same effect and be easier for users to understand?
  • There is no hierarchy of information to drive my attention to what is more important.
  • Is the information being shown what workers want to see? I personally don’t care about who the requester is or the expiration date. It can become relevant once you start knowing requesters.
  • No main action present on group of HITs. It took a while for me to understand the View a HIT in this group action at the top right corner of each task.
  • Hard to communicate with requester once you’re inside the group of HITs. And hard to find it outside either.

Experience the life of a Requester on Mechanical Turk

Same feeling as with the worker interface. Main reflections:

  • Hard to generate a survey. I started by going into the source mode. Took a while for me to double click a field and find out it could be modified. No UI indicators.
  • I couldn’t find how to create multiple HITs inside the same project without looking for help.
  • I got confused with what assignment meant. I didn’t relate it to the amount of users that will be able to execute the HIT.
  • Once you create an individual HIT, the Create another HIT based on the HIT you just created feature is very useful.
  • Different experience for different types of projects. No consistency.
  • Inside the Sentiment category, I couldn’t modify the sentiment scale from strongly positive to something else (e.g. strongly agree).
  • HITs I created individually in a sequence showed up on the Workers platform as a group, but on my Requester platform they show separately. I couldn't find a way to group them.
  • Various UI inconsistencies (e.g. submenu present in some situations and hidden in others).
  • I read about uploading a .csv file when you have variables inside the project template but I couldn’t find it inside the UI.

Explore alternative crowd-labor markets

Overall I think Upwork goes in a different direction than MTurk. Here are a few reflections about Upwork:

  • Upwork does a better job on the UI and tries to present the information in a better way.
  • Differently from MTurk, Upwork seems to be more community based. It tries to work on the freelancer profile while MTurk focus on the execution of the HIT itself.
  • Freelancers have hourly wages.
  • There is the possibility for requesters to invite freelancers to execute a job.
  • Not sure how the rating score works and how trustworthy it is (star ratings instead of acceptance rating as MTurk)

Readings

MobileWorks

  • Awesome idea. Very simple and effective.
  • Good way of validating tasks by sending it to more than 1 worker.
  • Good strategy to give opportunity to a specific target audience
  • Possibility to adapt to all classes of the economic pyramid.
  • UI and amount of tasks could adapt according to the screen size and/or phone model.

Daemo

  • Good approach trying to solve the inflated reputation score and the disconnection between mental models.
  • The prototype phase would be good to educate requesters but it also requires educated workers able to provide good feedback. A worker would probably need a specific qualification to be able to work on prototypes.
  • Gamification might be a good addition to the Boomerang system, where both workers and requesters can evolve in the platform by getting performance badges. The higher the level of the user, the more benefits they got. For example, a requester who always posts good jobs with no prototype alterations might need less (or none) approval during the prototype phase.
  • To help requesters rate their workers, Daemo might duplicate their ratings for other workers who executed the exact same task. So requesters would rate a "group of workers" instead of one by one in some scenarios.

Flash Teams

  • Great idea that can work really well with Daemo.
  • It's a great solution for freelancers who have problems dealing with clients.
  • It also offers a very good solution for spread out teams across the world.
  • Potential to integrate with various APIs for deliverables inside the platform

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @renan