WinterMilestone 1 Crayons

From crowdresearch
Revision as of 04:08, 16 January 2016 by Sachinsharma (Talk | contribs) (Created page with "== Reviews : Mechanical Turk Sandbox version == === What do you like about the system / what are its strengths? === # Each HIT has been provided with instructions which exp...")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Reviews : Mechanical Turk Sandbox version

What do you like about the system / what are its strengths?

  1. Each HIT has been provided with instructions which explains the task to be followed clearly, making the task to be performed achievable.
  2. Easy to seek tasks according to earnings each tasks provides, which is the main incentive for almost all the workers perusing such platforms.
  3. The worker dashboard is pretty useful for workers to track their progress.
  4. As far as i could work on the tasks, i found that the reward per HIT was fair when compared to the work to be done for that reward.

What do you think can be improved about the system?

  1. The parameters to filter the HITs need to have more broader classification criteria, besides the one already present, i.e., creation date,availability,reward amount, expiration date.
  2. Each HIT should have more informative statistics like the acceptance rate of the task by the requester.
  3. Search feature did not prove to be very helpful, since the HITs are not tagged or classified under specific categories depending on the task to be done.
  4. This platform is not available globally, which is its biggest drawback, since much of the online workforce hail from countries other that US. This means the requesters are not getting best of their investment, as they are not getting the potential workforce they could have at the same cost.
  5. Sandbox worker, didn't get paid in the end.
  6. Cannot comment on the result of HIT, the sandbox is only but a simulation and requester may not be looking at the work closely for accuracy as may be doing so on the alternative.
  7. User interface was utilitarian at its best, and did not provide a very welcoming look and feel, and did not prove to be very navigable. Upgrade to responsive design, even a mobile app/m site could do wonders for the platform.
Screenshot of the Mechanical Turk Sandbox Version.

Readings

MobileWorks

Mobile Works is a mobile based crowd sourcing platform. The aim is to allow the weaker section of the society to take advantage of the growing crowd source industry. Simple Optical Character Recognition (OCR) tasks are sent to different people via a mobile application. Usually the document is divided into smaller pieces which contain one or two characters. These pieces are then sent to crowd source workers through the mobile app. A very simple interface is used so as to allow the people to easily understand and perform tasks.

What do you like about the system / what are its strengths?

  1. It very well captures the fact that in developing countries, like India, mobile penetration is very high as compare to desktop penetration. Hence, such mobile bases platform allows a larger section to be a part of the crowd sourcing.
  2. Simple user interface ensures that even those who are not very educated or technically sound are able to understand and use it.
  3. Each task(piece of document) is sent to two persons, and the answer is accepted only when both the answer matches. Hence, no compromise with efficiency.
  4. To encourage good performers, the function to calculate the cost of respective tasks also include the past performance/efficiency rate of the worker.
  5. It is not very time consuming. Workers may perform the task while travelling or even at home which watching tv etc.

What do you think can be improved about the system?

  1. To include more than just OCR tasks, for example audio recognition, picture classification etc.
  2. There is a rapid growth in the number of smartphone users. They are now easily accessible to even lower section of the society. Smartphones bring with them larger screens , better quality, better hardware, location facility etc which can be utilized to assign more complex tasks to the workers.

Daemo

What do you like about the system / what are its strengths?

What do you think can be improved about the system?

Flash Teams

Flash Teams are basically computationally guided team of crowd experts supported bu lightweight reproducible and scalable team structure. Interactive systems can manage and manipulate by expanding and shrinking the the team as per requirement. Foundry is an end user authoring platform and run-time manager and it also serve our purpose by enabling the formation of flash teams. Crowd-sourcing system coordinates large group of people to solve the problem that a single individual could not achieve at the same scale. Crowd-sourcing system work by organizing work into micro-tasks. Foundry and Flash Team enable crowd-sourcing of a broad class of goods including design prototyping, course development and film animation in half of the time of traditional self managed teams(as per the paper). The aim of this paper is to provide the value of team structure, modularity and coordinating mechanism but in way that automatically creates team at the cost of increasing flexibility of the crowd.

What do you like about the system / what are its strengths?

  • Flash team composition:
  1. Flash enable us to gather and coordinate paid experts form the crowd to compute complex and independent task quickly and reliably.
  2. Flask teams afford dynamic recruitment and co-ordination of on-demand expertise that is extremely difficult in offline scenarios.
  3. Flash teams can take advantage of timezone differences that can potentially allow them to carry on uninterruptedly for days or weeks.
  4. Flash teams can be combined to create new types of organization with completely flexible boundaries.
  5. The basic unit of flash team is Block. Each block is represent one or more experts performing performing a task.
  6. To avoid confusion or diffusion of responsibilities for each block , the project specifies DRI(Directly Responsible individual) to manage the task completion.
  • Runtime and Coordination:
  1. Users either author a team from scratch or use the existing team with or without change in its composition.
  2. Workers can easily be recruited from oDesk for the particular job.
  3. oDesk provides rating that makes filtering of high quality experts.
  4. Requesters are provided transparency of the whole process so that requester can provide feedback answer and question through the chat system.
  • Computationally enhanced flash teams:
  1. Modular composition of the flash teams i.e small flash teams can be combined to get the bigger organization.
  2. Path Search support for team formation or authoring.Planning algorithm to find set of blocks that connect from original input to final output and provide automatically if such path exist.

What do you think can be improved about the system?

  1. Flash team since this paper are limited to to napkin desing , one can work with different kind of flash teams.
  2. There can be coordination problem and can originate conflicts within the team itself. There could be conflict resolution section for continuous work flow.
  3. Foundry can improved to predict the the time for the completion of particular work by a flash team.
  4. There can also be system to admire the best DRI by accessing number of job of quality that had been completed under his/her guidance.
  5. There can also be flexibility of changing DRI form time to time within the flash team from one job to other for the best result.
  6. System can also be improved to show best flash team available for the tasks according to the previous work done(work/cost can be point).

Milestone Contributors

  • Ameen Mohammed Khan @ameenkhan07
  • Farheen Nilofer @farheen
  • Sachin Sharma @beingcooper
  • Sarah Masood @sarah
  • Shivam Rana @tminima

.