Milestone 1 researchinprogress

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

We signed up as worker for Mechanical Turk on [][]. However, the account demanded 48 hours for activation. We could not work on the hits till that time. Plus, that site was not valid for Indian citizens, it appeared. So we moved to [][]. However, the site made us register at 2 places, plus the interface wasn't very easy to comprehend and work in. Therefore, we decided to work on [][]. Easy to login and good jobs to offer. We decided to work here.

There were jobs with different cash amounts varying from around $0.10 to $0.60 for each task. Bonus jobs gave in cash around $2. Since this was our first such experience, we stuck to very basic jobs which involved signing up for different websites, installing different apps, commenting at various portals/sites. There were certains jobs which expected us to read on topics Eg Bitcoin. The submissions were to be given as the usernames/userids/urls used for the different tasks. We submitted the tasks which are pending for approval presently. We have done them to the best of our knowledge and capabilities. So, I believe approval should not be an issue. We did a total of about 11 jobs each of worth just above or around $0.10. Hence, the approximate earning should be a little more than $1.

While most jobs were simple there were still some basic issues we faced in certain jobs. Some links for the tasks did not appear to exist. We had an experience where the link that would open would ask questions based on the site we opened. However, it would lag and randomly pop up questions a very long time after using that link was opened, because of which it wasn't possible to complete the job. Some very basic tasks had not explained the job procedure very clearly. For example, a job expected us to give the email received after confirmation. However, even after confirming the account more than once, we did not receive any confirmation mail which left me at a hanging point about my submission. There was another job in which our last twitter tweets were not to be retweets and we were expected to have done 20 tweets beforehand which should have particular amount of likes which complicated the job to be done.

It was a great experience, and fun too! We learnt about so many new websites and concepts. Some jobs asked us to search meaning of some terms which were useful to us. We downloaded few very great apps. We were even getting paid for the reviews of the apps. We read about the bitcoins and also a website about two people who help others overcome issues regarding personal injuries in Atlanta, both legally and emotionally and how they have been successful in doing this. There were apps like Flickr and Instagram on which you had to work, which is obviously a must know in today's technology centered world.

Experience the life of a Requester on Mechanical Turk

We could not complete this task. By the time we got the hang of the whole procedure, it was too late to get the money and proceed for the task.

Explore alternative crowd-labor markets

Crowd labor is all about doing different kinds of work mechanically. As we know, Mechanical Turk/ Microworkers involves the use of jobs which computers can not do themselves as such presently. Captcha required during sign ups and logins cannot be read by computer themselves. Hence, we experienced the manual doing of work through the small jobs of working on sites and apps. This can be brought to more specific applications. We can have websites to have people work on specific concepts, which might require contributions of many ideas and labor individually. As we see, [][] appeared to be a website dealing only with Furniture shiftings and issues related to that, whether you want truck deliveries, car deliveries etc. This required crowd-labor too, but the field is much more specific now. There were works related to this like repair works etc. Similarly, was specifically based on tasks related to galaxies, studying about their properties and drawing conclusions which helped astronomers in real life. This helps a worker/requester to easily decide his interests and focus and work on that only. A requester will get better quality results of he posts galaxy related queries on [][] alone as compared to other websites. While taskrabbit starts with knowing your zonal area to decide on the available options, galaxyzoo starts by wanting to know the shapes and other specific of the galaxies it provides. Odesk was another example, which appeared to give space for different, big technical project on site. It further clarifies specifications as you narrow your options to designing, web development, software development etc. as soon as you open the website. Microworkers on the other hand, was more general and you could be expected to read about anything or comment about anything, go to sides dealing with businesses, advertisements etc which you got to know only once you read about the job.



Strengths of the system:

1. The system is easily accessible by any participant, living in any part of the world. Quoting from the paper, “Amazon’s Mechanical Turk (AMT) has about 200,000 workers who reside primarily in the US and India. About 56% of these Mechanical Turk workers (“Turkers”) are from the United States and 36% from India”, thus there was requirement of a solution so that these tasks could reach people of the lower economic background in India and elsewhere. Since, MobileWorks involves working on a mobile phone, it is way easier for backward classes, to understand and earn and a living.

2. “Quality is maintained using multiple entry. Each task is distributed to two workers until two of the answers match”, this ensures that the task given by the requester is completed with full efficiency, this also adds quality to the worker profile, if the task done is delivered perfectly, score is added, else their quality reduces.

3. It is a great source of employment for people to earn a little above their normal income. The tasks assigned are not that difficult, as they are assigned a part of the bigger task, and then the task is reassembled. As told by a user of MobileWorks, “I could do the work and earn money while traveling to my regular job or even while watching television”, this is where the advantage of having it all done on a mobile phone comes, one cannot work so easily and effectively on a PC or laptop while they are travelling for their work, however, mobile phones are a device that can be worked on very easily, hence MobileWorks helps its users in many ways.

Improvement in the system:

1. Although the whole idea of MobileWorks seems flawless, there can be some minor improvements that can be brought in for the betterment for its users. There can be encouraging schemes for the ones performing the best, for example, MobileWorks can start a fundraiser, which could collect money and add to the account of the best performer of a span of three-four months, this will not only encourage the best performer to perform more efficiently, but also improve the performance of many other participants as well.

2. There can be an application for MobileWorks so that it can reach an even higher percentage of audience and there can be higher proficiency in the work done. This will be achieved because, there will a constant notification for the user, if a suitable task is uploaded, and this will also increase the response to a particular.


Strengths of the system:

1. The system reaches a larger mass, as it involves SMS services, where in the user is sent the image of a word that is needed to be decoded, the user then has to reply back as to what the word is likely to be. This is a better way of involving a larger mass in a task, and thus generates more employment.

2. “This paper presents the design and evaluation of mClerk, a new platform for paid crowdsourcing amongst low-income workers in developing countries”, since this does not involve any usage of internet, a larger percentage of lower economic background mass can be targeted. This is one of the biggest strengths of this system.

3. “However, in our case, since the users are sending back transliterations, there could be a large number of cases where similar but distinct transliterations point to the same word. For instance, “Nammaa” and “Namma” are the same word in Kannada. To handle this problem, we modified our algorithm so that two responses are considered equivalent if both of them transliterate back to the same word in the local language. We make use of Google’s transliteration API [6] to convert from English to Kannada characters. After this, verification follows the usual procedure where two equivalent responses for the same word are considered to be correct”, this maintains the efficiency of the work done by the users.

4. Also the quality of the work is ensured by sending the same data to two users, and matching their responses as done in MobileWorks.

Improvement in the system:

1. Since this system generates employment in countries like India, where there are 22 regional languages and hence more should be done in this field so that more and more languages can be incorporated and this will directly increase employment since many people are much more comfortable in recognising written words in their language.

2. There was also a survey conducted, to conclude a fixed word length, but this could obviously vary if there are different languages because the width a single word in Hindi, English or Kannada would vary evidently. They had fixed the word length to be 74 pixels considering a single Kannada letter to be 14 pixels, but this can vary in many different languages where the letter sizes are larger than usual.

3. The usage of SMS as means to share data with the users is both strength to the system and drawback. This is because due to delivering the image to the user on his/her phone, the size of the image as scanned is to be compressed that will automatically reduce the quality of the word and hence reduce the clarity, which will end up in a false derivation of the given word by the user.

Flash Team

Strengths of the system:

1. “To structure expert crowd work, flash teams require small atomic actions called blocks. Each block represents one or more experts performing a task”, it involves a more systematic division of task divided into smaller sections which are handled by a group of experts in that field. This result in an efficient delivery of work required from the task.

2. They are organised with the task assigned to them, because they collaborate as workers and work together on ‘Foundry’, where there is a time line, a deadline assigned with tasks given by the requester. The tasks range from mobile applications to web designing and much more.

3. The work and tasks are extremely co-ordinated and are performed in a pipeline manner, task B will only be initiated when task A is complete, this ensures the completeness at each step and hence a better output of the final task.

4. Due to all of the above reasons, the task is completed in a shorter span of time, than if the task was assigned to a single person to work on. Efficiency is also ensured by having a shared Google Folder and online chat window side by side.

5. Since the experts are paid on hourly basis, they tend to work more to earn more and hence the project is delivered to its full potential.

Improvement in the system:

1. Lower educated masses are not employed with this service at all it only leads to the employment of the educated class. This is one big drawback of this system, because with such a condition, masses of a country like India, and other developing nations might not be able to employee many of their backward class population.


This was our first experience of all the work we have done, including the making of wiki page. As the team leader, I hope we are working in the right direction and have done a decent job!