WinterMilestone 1 zrehmani

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

Reflect on your experience as a worker on Mechanical Turk. What did you like? What did you dislike? If you're from outside the USA and unable to access MTurk, try try the worker sandbox, or other sites like CrowdFlower or Microworkers or Clickworker.

Response:

The site itself is not very intuitive. Immediately when you sign in/ register, you are met with a long list of HITs that you can seemingly choose from. However, figuring out how to actually accept a HIT was rather difficult because you must click "View a HIT in this group" before accepting the HIT. Sometimes I was not able to click "View a HIT in this group" because I didn't meet the qualifications, other times I was able to view the HIT even if I did not meet the qualifications. Overall, the experience is rather confusing and takes some time to understand the inner workings of the site.

Experience the life of a Requester on Mechanical Turk

Reflect on your experience as a requester on Mechanical Turk. What did you like? What did you dislike? Also attach the CSV file generated when you download the HIT results. If you're from outside the USA and unable to access MTurk, you can try the MTurk requester sandbox, or CrowdFlower or Microworkers or Clickworker

Response:

Note: It took me multiple days to be accepted by Mechanical Turk; therefore, I have only 1 HIT in my CSV file. Regardless, I think I have gotten the hang of this (mediocre) system. CSV File:HITresults.csv

I experienced a lot of difficulties while trying to create HITs on Mechanical Turk. Creating a HIT is easy enough; the first time I attempted to post my HIT, I completed all the steps and received the confirmation that it was posted on Mechanical Turk. However, I tried to search through the HITs in order to find my listing, and I could not find it. The second time I created a HIT, it required for me to purchase a prepaid HIT before I could post anything. After paying for the prepaid HIT, the site did not take me back to post my HIT, which would have been intuitive. Instead, I had to navigate to my HIT and post it.

Explore alternative crowd-labor markets

Compare and contrast the crowd-labor market you just explored (TaskRabbit/oDesk/GalaxyZoo) to Mechanical Turk.

Used Mechanical Turk, and although it's very unintuitive, it's still better than the listed alternatives.

Research Engineering (Test Flight)

I have completed the setup instructions prescribed in the GitHub.

Setup-Daemo.png

Readings

MobileWorks

  • What do you like about the system / what are its strengths?

The primary benefit regarding MobileWorks is that it taps into the potential of the mobile industry. With the global cost of mobile devices and cellular plans decreasing, cell phones are becoming increasingly ubiquitous, outpacing the growth rate of laptops/desktops. MobileWorks takes this trend into perspective, and attempts to provide a mobile interface similar to Mechanical Turk to appeal to audiences that may rely on "Mechanical Turk"-like platforms to earn their personal income.

  • What do you think can be improved about the system?

Much of the "future work" summarizes the aspets of MobileWorks that can be improved: audio transcription, language translation and subtitling, prioritizing "high-priority" tasks, and maximizing the usability of the system so that the fact that the platform is run on a mobile interface does not impede individuals from completing tasks.

Daemo

  • What do you like about the system / what are its strengths?

The boomerang reputation system helps to ascertain that ratings are more authentic to what the user truly felt about his/her experience. The use of prototype tasks seems to ensure that tasks are clear and the work produced matches what the requester had in mind, ensuring that both parties (worker and requester) are content.

  • What do you think can be improved about the system?

There needs to be more experimentation done; so far, the results of the platform cannot generalize the population's experience. As discussed in the paper, although Boomerang is a system that looks great on the surface, users may choose not to rate, or even if they do, may not rate carefully. Providing a larger incentive for users to use Boomerang may help them achieve their goal of restoring trust between workers and requesters.

Flash Teams

  • What do you like about the system / what are its strengths?

This system uses the idea of a flash team, which is a method for dynamically assembling and managing crowdsourced expert teams. End users can assemble a flash team on-demand simply by providing a desired input and output. Flash teams leverage the scale of paid crowdsourcing for expert work. Additionally, a strength of this system is that the results of the field experiments exemplify that flash temas can accomplish efficient and high-quality work, with flash teams taking half as many hours as teams in a control group.

  • What do you think can be improved about the system?

Not enough experiments have been conducted to figure out what tasks flash teams are best suited for; thus, there is still room to continue gathering evidence and information. Additionally, some flash teams have reported that they still had difficulty regarding coordinating and dealing with conflicts, which flash teams are supposed to solve. Possibly in order to reduce conflicts and increase coordination, like-minded or trusted colleagues should be placed on the same team.

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @zrehmani3