WinterMilestone 1 Atin Mittra

From crowdresearch
Jump to: navigation, search

This page details my introductory analysis of the crowdsource industry through various research papers and observations of existing platforms. Questions? Slack @atinmittra

Experience the life of a Worker on Mechanical Turk

My experience on Mechanical Turk was unpleasant. Upon signing up on the platform I was really struck by the low quality of UI design. It took me a several minutes to become oriented with the format and the processes needed to sign on to a task. I wasn’t onboarded really, just asked to sign a service agreement, which felt cold. The industrial feel of the system combined with the surprisingly low earnings available for my qualifications deterred me from using the site as “real user” vs. for this research context.

Experience the life of a Requester on Mechanical Turk

I was very disoriented requesting a task on the platform. Although Turk has a microtask focus, I played the role of the complete beginner with no concept of it. So, I posted a task of a “Financial Web Site design”, a task that could fetch a couple hundred dollars easily. As I was going through the process of adding the task, I was prompted to add a CSV file of the data, however I do not believe there was sufficient background on what the data should contain, further damaging my experience by working off the assumption that I understood what was happening and what tasks I should input. I added four company names with founder/CEO/state of residence as a CSV file to upload as part of the posting. It continued to create four separate tasks for the data rows I inputted, which didn’t make any sense in the context. I was completely confused and thoroughly disappointed with my experience on Mechanical Turk. Media:Batch_2233286_batch_results_(1).csv

Readings

MobileWorks

  • What do you like about the system / what are its strengths?

MobileWorks is a promising idea given the economic benefits it can provide Indian slum workers. The company leverages affordable technological adoption throughout the country to provide a better system of work and potentially wage to a population that is largely left to menial tasks without much opportunity for advancement. I like the quality assurance system that looks for double confirmation among two workers, addressing a key concern of using unskilled workers. Overall I believe MobileWorks provides a job that trains impoverished people with technology and allows for their skills to help provide social mobility.

  • What do you think can be improved about the system?

I am curious to hear why survey participants didn’t rate the usability experience a 5/5 - it seems there can be some improvements made in that field.

Daemo

  • What do you like about the system / what are its strengths?

The Boomerang system is very promising. It provides a sound solution to the very real problem of reputation tracking and management. On this system, using the ratings allows for requesters and workers to work with “preferred partners” to more effectively accomplish their goal. Using prototype tasks is a very interesting idea that I believe will help improve the task authorship. I wonder if these features are more amenable to those with poor past experiences on crowdsourced websites? I am really drawn to the start feature of having multiple check marks, which safeguards against a poor first experience; I think that is extremely crucial for wide scale adoption.

  • What do you think can be improved about the system?

While the Boomerang and prototype task systems help experienced Daemo users develop relationships, I worry there is not enough focus on the “new uses”. First, if a person is a fairly new requester or worker, what is their value proposition to Daemo? If they haven’t personally had a poor experience on a platform such as this, why do they choose Daemo for its superior safeguards to quality control to other popular and established services? Also, the same task will probably cost more due to the extra step of prototype tasks – what attracts a critical mass of requesters? Is the assumption that this platform will be geared towards more habitual microtask workers/requesters? Also, having three distinct check marks to provide a scoring method can be effective, but I wonder if it is limiting a broader picture of quality. I mean, why only provide three grades? To what degree of specificity will the scores really be ranked? If for example a certain person were excelling unequivocally on the platform compared to all the other competition, would they not benefit greater from a 10-point scale? This system can then provide a greater level of description with a singular number.

Flash Teams

  • What do you like about the system / what are its strengths?

I love this concept and can personally empathize with the problem it aims to solve. Providing scaffolding for remote or crowdsourced teams to collaborate more effectively is a home run in my mind. It provides more information and piece of mind for the client while helping to ensure efficiency and quality, and helps the team accomplish the project swiftly. Also, it works to clarify a very messy workflow of remote work through large organizations. The concept of breaking down a large org into multiple flash teams and integrating the block concept is an interesting look related to the type of workflow this research project has taken.

  • What do you think can be improved about the system?

I think there is a lot of potential in relationship creation through this platform. I found it interesting that users desired to have a better team dynamic and interact more than just work together. This insight may provide an even heftier value proposition of not only connecting talented professionals, but creating talented teams – potentially for the long-haul. The desire to curate these professional realtionships would need to be validated in the larger market place, outside of the small study discussed in the paper, but if validated, layering the ability to facilitate community and partnership is what can turn the product from good to great.


Milestone contributors

@atinmittra