SUMMARY: This page is dedicated to the topic of guild-based leveling.
Leveling arose as a method to address the cold start process of Daemo, or any new software as a matter of fact, when new users entered the system. The idea being that the user needs, initially, a framework to accurately understand the software and its functionality, and eventually the user would become more involved in software organizational actions, such as managing portions of a guild system. As of current discussions, users benefit by increased payment levels, greater information access, increased ability to influence guilds and a growing variety of tasks tied with increased ranks.
EXAMPLE PAPER ABSTRACT
Crowdsourcing platforms are subject to reputation inflation due to repercussions from social pressures. As a result worker-requester relationships can be mired by false impressions of the other's professional profile as presented from a platform. Guild based leveling presents a solution to this issue by creating a system whereby a worker's peers-identify through review the quality of another person's work product. With higher review ratings, the worker achieves a higher level within the guild as a publicly demonstrable award of task-performance competence. In this study, researchers utilized a mixed-method research study using a daily activity/thought journal, scripted tasks, and a primitive micro-world simulation. Researchers identified that leveling mechanism indeed did resulted in better work products as a function of leveling and identified patterns in behavior that changed with increased rank.
GUILD LEVELING STRUCTURE
|1||Learns the system and performs any of a number of tasks with ground truthes|
|2||After a demonstrated competency level, the worker is able to review others with a .5 sigma increase in pay, ability to respond to posts in a forum, introduce standard survey tasks|
|3||Worker receives ability to review micro-task results within a gaming framework (speeds up review and increases pay levels) with a 1 sigma increase in pay, start posts in Daemo forum, introduce standard tasks|
|4||Worker receives voting ability within the Daemo forum and a 1.5 sigma increase in pay. Introduce tasks with real world requesters.|
|5||Fictional level to help constrain the other levels. This is the guild administrator who is played by a confederate team for the purposes of the study.|
Researchers recruited 20 participants from MTurk to perform this study. Over a three week period, participants were given tasks and increased benefits according to the levels. Participants engaged with researchers on a daily basis by receiving and completing tasks through emails and direct access to the Daemo Forum as a function of their leveling. Each completed a daily diary detailing their activities along with quantitative questions tied to the platform and mood. After 3 weeks, the diaries were analyzed using Qualitative Comparison Data Analysis and custom codings that were identified, discussed, and review by a team of researchers using RDQA. Extreme and unique cases were identified for follow up interviews.
Wizard of Oz Method
Tasks were assigned to groups as they emerged through reviews by a sequential Graeco-Latin square design. The task classification scheme is as follows:
- Tasks with known answers
- Standard Surveys
- Standard Tasks
- Tasks from Real World Requesters
Each task will be given on the respective days to each group.
Quantitative analysis was performed to identify behavior changes associated with worker level.