Winter Milestone 5 Leonardykristianto

From crowdresearch
Jump to: navigation, search

Assistive Wage Analytics

Brief introduction of the system

The intuition behind Boomerang is that it's a reputation system that influences the quality of tasks/workers. The feedback the users gave, whether they are requesters/workers will directly impact the works that they receive, or their ability to find good workers. The analytics enhances Boomerang with added variables to further improve the list of tasks that appear in workers feed.

How is the system solving critical problems

The system abandons the normal model of estimated wages/hour that's more commonly seen on 9-to-5 workers and instead move towards a session-based wage estimation because the former just does not match the crowdworkers behaviour and flexible work time. This model works by monitoring the variations in workers' work duration and applies a limited scope to capture a set of data where it can be supplied towards Boomerang. The datasets include the highest earning that a worker have obtained during a specific session, an average earning of different tasks category (image labelling, essays creation, etc) and a suggestion of which tasks or requesters to work with to make the most out of their time. The worker can further customize the way her task feed is served by adjusting the parameters to suit her preferences and needs.

Introducing modules of the system

Below, we introduce the two core modules that makes the system possible. The first being the sessions on how they are monitored and the reasonings behind its application and the second being how the suggestions are being made in a way to help the workers that works together with Boomerang. Because according to @michaelbernstein, in order to write a good paper you have to stake out as big as you can, but no bigger than you must. Thus the solution should somehow be able to work synchronously with the Boomerang concept instead of working against it.

Module 1: Session


From the brainstorming that the crowd research team members have conducted and the feedback from the workers during the worker-requester panel, it was discovered that one of the features that workers continuously seek is the functionality where they can expect, or predict the estimation of wages that they can earn while determining which tasks' offers to work on. However, it's generally accepted that it's hard to predict/calculate how long do workers actually do work since there were many occasions that these works have intervals or breaks between them, or were done in shorter bursts unlike their 9-to-5 counterpart (where workers are expected to be productive exactly for the duration they're working). And thus, it increases the challenges of determining how much these workers will be able to make per hour per task, since these works cannot be quantified within a determined scope.

Workers Time Comparison 2x.png

Module preview

System details

A session is a limited scope on workers behaviour ranging from a shorter duration to a longer one (possibly 1-3 hours). These sessions are monitored with a nominal unit (10, 30 or 60 seconds) and are only counted when they fulfil a certain criteria. The sessions are categorized into four types of workers status/activity on the platform (Daemo). The categories are as follows:

  • 3 - In Work
    This is the state when the workers are logged in, and are currently performing/completing a task.
  • 2 - Active
    This is the state when the workers are logged in, and are actively doing something on the platform like browsing for tasks or interacting with other workers/requesters.
    In this state, workers aren't currently performing a task.
  • 1 - Idle
    This is the state when the workers are logged in, but are not actively browsing any activities or in work.
  • 0 - Logged Out
    This is the state when the workers are not logged in.

Session States 2x.png

As the workers activities vary greatly from time to time (creating bursts, or as we would like to call it, a surge), establishing these states is important in a way that it enables us to apply a pattern to workers' timeline and observe them. A session (or perhaps, better called as working session) is a phenomenon when these patterns return a notable activity across the duration that we have set (1-3 hours). We propose a 75% threshold as the guiding pattern to determine when the system should record down a session, meaning that, for example, when a worker clocked more than 45 minutes cumulatively in working state in an hour, the system will store that as a legitimate working session. These sessions will be compiled, and the calculated and compared data will then be sent to users to determine how much they can earn in their most active sessions.

A Single Session.png

Across 24 hours.png

Module 2: Suggestion

Improved Task Feed.png


The goal of the Assistive Wage Analysis is to arrange the task feed efficiently (and smartly) so that it eliminates the time spent on the "Active" state where workers are looking for tasks to accept to maximize the earnings that they can possibly get. The possible weakness, or limitation with this system is that there's no way to prevent idleness or workers inactivity if they chose to do so. Thus the system ignore those variables instead that might cloud the normal calculation of wage calculation and only take action when surges are detected. This system does not support a wage per hour estimation for every task because it's unrealistic and does not conform to the behaviour of workers that have been observed. We simply cannot predict how much a task can give workers a static wage per hour unless workers do really work on it for >30 mins and the tasks will still be available later (then we can form a calculation based on how fast the workers had worked on the past 30 mins and how much she had earned). Instead, with this system we can tell workers that "This type of tasks are the kind that contributed the most in your top pays, they are either those that are posted by requesters you worked most with, those that are totally in your domain expertise or those that can help you to achieve your previous best record or even beat it. We compiled it to the top of your task feed, so you can go do them now".

We form our estimations based on when workers do really work (typically in surges) and record those sessions, then calculate the average/best wages they can earn with that level of effort to inform them that "if you work this hard, this is how much you can get based on your previous record. If you say that you were lucky last time because you found some really good tasks by chance and they won't likely be available again, fear not because this analytics will then work its brain to find alternative tasks in your expertise that can give you the equal amount that you would've gotten last time provided that you spend the same amount of efforts".