Human-Driven-Daemo (Milestone 5) yoni.dayan

From crowdresearch
Jump to: navigation, search


Fundamental/philosophical/ethical issue with current crowdsourcing platforms

We live in a 21st century characterized by the digitalization of our life and the disruption of our societies and economies by technologies like algorithms and IA that are destroying "old jobs". In reaction, there's a growing awareness and actions from many institutional, academic and corporate actors, to put the emphasis on what we, as human beings, can bring to the table, our "added value" compared to the machine. And this is our creativity, our capacity to blend different disciplines (interdisciplinarity), our curiosity, our empathy, our ability to collaborate. We are social creatures, capable of cooperating for something greater than us and progress in the process, this one of the essential aspects of mankind.

[insert studies, data, reports for example from the World Economic Forum, illustrations]

In opposition to this global trends, and as shown by: 1) A simple observation and the use of most of the crowdsourcing platform 2) Testimonials from crowdworkers, crowdrequesters, including during our hangout during Week 2.

We have a starting postulate that current crowdsourcing platforms embody "digital stakhanovism". They are completely lackluster in terms of leveraging human qualities, and giving the workers the space to bloom in an environment where they will get those feelings of relatedness, connection, and belonging to a community.

Crowdworkers have this impression to be just a little ring of the long keychain, and trapped into a productivist frame where they need to over-work on a task, over-achieve, over-do to get good ratings, focus on what they do in a relative confinement (it's them and their task, with minimal interactions with the requester and the community unless they switch to third-party solutions like discussion boards).

Importance of the "human factor"

Various studies demonstrate the importance of such factors in the commitment of individuals in many aspects of their life:

  • For learning (dropout rates in MOOCs, participants tend to be far more engaged if they study in group, if they belong to a cohort)
  • For their work (insert studies linking good organizational and management practices + well-being of workers, with their productivity)
  • For their life in general (the famous Maslowian pyramid of needs, were belongingness is one of the essential human needs)

Problem statement

How could we "hack" crowdsourcing platforms and the way they are functioning and governed, to bring back the human element at their core?

Personal experience & application to Open Gov

As i think the value of crowdresearch initiatives is to "leverage the crowd", e.g., use our diversity and the different expertise we bring, i will continue to explain my experience in Wikistrat (crowdsourcing consultancy platform) and NovoEd (online course platform from Stanford, and think of how to apply it to the Open Gov problematic (and potentially the guild structure)

1. Wikistrat

As explained in my previous contributions, Wikistrat is a consultancy company (think of Deloitte, McKinsey) based on the principle of "wisdom of the crowd" and structured around crowdsourcing. We use Atlassian's Confluence (like a Wiki but with more collaborative project management features) to mobilize up to hundreds of analysts in simulations ordered by clients, to forecast future trends in geostrategy and technology. I'm a supervisor and analyst there.

- Our 2000+ analysts are gathered in "desks". For example, the "technological trends" desk gather experts in technology. Instead of being drown in a community of 2000+ person, we are in sub-group of a few hundreds top. Each of those desks has its own page, dashboard with newsfeed, discussion board, etc, in short a lot of stimulus, of information, to know who is logged in, who is contributing, who did what, to leave comments, to engage in a conversation, to debate, to help each other, to work together, to create together. This organization allows relatedness, and also a more efficient mobilization of our analysts ("ok, we have a simulation on Africa, let's summon the Africa desk").

>>>>>> Application to Open Gov: Daemo could structure its community of crowdworkers into desks as well. Several possibilities of organizations (by theme, by geographic location, etc.). Many moves to ensure a cross-desk flow though (the simulations are open to several desks, contests, general discussion board, etc.)

- Each analyst has a dedicated page, with his/her profile, tags defining him/her, and can be found through a search engine, reached out to through a messaging system, tagged and therefore summoned if we write his/her name in the body of Confluence pages.

>>>>>> Adapting this to Daemo. Crowdworkers aren't numbers, they are real people, who got their own page describing them, their background, their skill (possibility of showing a skill map, like a competence tree in role playing games), their "tags" (how to "define" someone in keywords, that are then easily found if we want someone specialized in "gaming or "Russia"), receiving messages, etc. Possibility of tagging people in a crowdwork ("ok i'm doing this task, and i know someone who could contribute as well in view of his/her skill on the subject, let's summon him in the page/comment section").

- Analysts can also connect/follow others ("oh, this analyst output is really interesting, we think the same thing! let's observe him/her, perhaps i can learn from him/her").

>>>>>> Reproducing this "social learning" (it's a theorized principle) on Daemo.

- Each pages constituting the output of our simulations have comments sections. Sometimes, the comments are more lively than the edits on the page.

>>>>>> Ensuring that whether the tasks are, there's a space associated with the task to leave comments. Possibility of summoning others through this comment section.

- Each simulation has a dedicated discussion board, lively thanks to the work of supervisors who also act as community managers.

>>>>>> Doing this for Daemo as well.

- Each analyst can "like" (like in Facebook) others' pages and comments. They can also give points to other (akin to the crowdresearch badging system, but more used), those points are a part of each analyst profile. As supervisors, one of our role is to foster such culture of encouragement, which is instrumental to commitment.

>>>>> Replicating this for Daemo, making sure those likes and points are useful, by analyzing if they are fostering increased commitment through concepts of positive peer pressure and emulation, also by putting those points in the crowdworkers profiles, as well as tying those points to their future work (having X points means i can unlock this "personal trait" point on my competencies tree, having X points means my worker rating will increase and i would have better tasks to do, etc.)

2. NovoEd

NovoEd is a social MOOC platform created by Stanford. It's a provider of online courses, that are group-based, and project-driven, which is original compared to most of other LMS (learning management systems) that are more conventional (solo experience with some group flavor, rather passive assessments like quiz and text-heavy assignments, etc.). I'm a Teaching Assistant there.

- We are pushing students to have complete profile pages, with search engine features, etc, for the same dynamics than on Wikistrt

>>>>> Reproduce than on Daemo. Make sure workers are human beings and not a fragment of a distributed workforce. Each worker can see who is who, at least in their chosen overarching structure (like the guild).

- We are pushing students to work in groups. For a course on entrepreneurship, we want them to study and practice how to build a startup in teams of 2 to 10 for 3 months. In a sense, it's replicating the life of a startup. Each learners can see other profiles, look for specific expertise, geographic proximity, and create his/her team or join one. Each team has its own page, a feed of the group work, links to project management platforms (including google docs), shared calendar, hangout link, etc. A real space for group interaction and work. This is what makes NovoEd so special and what all our learners remember of the platform. Despite the distance, despite being on a virtual environment, they got to meet new people, potential associates, learn new cultures, new mindset, discover others throughout 3 months, complete each others, and in the end, having new associates, or at least friends who will support them in their future endeavors.

>>>>> Daemo would stand out if we are able to replicate this philosophy as well, with adapted mechanisms and tools. Even within an already existing gathering (like guilds), we could have groups self-constituted around tasks given by requesters, to work on it together.

- 9/10 of our assignments to assess students learning, are group projects. For example, the final assignment we ask is to design a "startup deck", like a startup pitch explaining in less than 5 min through speech & mockups, what is their product/service, for who they are doing that, how they are doing it, their biz model, etc. Up to 10 people are working on those assignments, even if some of them may seem more adapted to solo-work (like reflections). Nonetheless, it's pushing the group to find alignment between them, it's challenging their ideas, it's pushing them to go out of their comfort zone, for better creativity, engagement and therefore more qualitative output and more commitment.

>>>>> We could imagine creating some kind of "group overlay" on top of tasks even if they are more solo. With a system of redistribution for the money. In order to reproduce those aforementioned dynamics.

- MOOCs are usually asynchronous in nature. Even if we have gating (each part of the course is released in a specific date) and deadlines ("you need to submit X before this date"), learners are free to check the content when they want (7am, 9pm, etc.), and work on the assignments when they are available (even in groups, they usually open a collaborative doc, and leave their output on it in an asynchronous fashion). The interface of team pages (easing the difficulty of finding a common time slot, integration of google doc, etc.) is nonetheless conducive to some synchronous activity (usually, teams gather one time per week to discuss the assignment, etc.). Also, we (the staff) are organizing "synchronous golden nuggets", like hangouts with the professor. Those moments are ensuring that even if we have a cohort of learners coming from different time zones, there are times when we can gather to get this sense of belonging to a community, a big family.

>>>>> Ensuring that Daemo and our guild idea foster those kind of group-wide interactions, through the set up of events (contests, jams, etc.), permanent line of communication (permanent google hangout, chat akin to Slack), rendez-vous, etc.

Open Gov & guilds

All those aforementioned ideas are compatible with the Open Gov & Guild direction that have emerged those last weeks.

How does it scale?

The features can be developed within Daemo, or integrated within Daemo (to reduce development time while still having an integral experience) or using third-party solutions (outside Daemo, which wouldn't be the best option).

Once those features are developed and best practices are disseminated, those human-driven dynamics are rather easy to scale. I've seen them working for MOOC with 2000 people, or 200 000. Still in alignment with the concept of "leveraging the crowd", we can have ambassadors ("guild maesters"? sergeants, etc.), crowdworkers who distinguished themselves by their behavior (helping other, pedagogy, etc.) their output, and their availability, who will act as relays and facilitators in case the group/guild is getting big. Of course, those ambassadors would be recognized (badging, ranks, gamification, etc) and compensated for this work.


Questions from @trygve and answers:

1. How are internal leadership/mentoring resources compensated in Wikistrat?

Internal leadership is through a meritocratic system of ranks and gamification. The more you contribute, help others and engage with your fellows, the more the system automatically give you points, that make you climb a ladderboard with levels. Wikistrat staff mix this metric with their own observation to invite analysts to become supervisors, who will have a special rank, a special visibility, and added compensation.

The system could be improved, with even more transparency behind this "one of our own is emerging and positioning himself/herself as a natural facilitator of the community".

2. How are those that join Wikistrat vetted? Is it before they join, is it a series of tests or another heuristic?

Two ways: people can apply directly on the page with the usual background info, then are going through a conventional application process (including an interview). Depending on their level of expertise, they enter the platform at different ranks (from researcher, the lowest, to senior analyst). The more you gain points, the more you can climb the ladder and improve your rank.

Analysts already in the platform can also recommend others from outside, who are then interviewed, etc.

It's a rather meritocratic system, but still half-rooted in conventional methods.

We could envision a more forward-thinking application to Daemo and our idea of guilds. Depending on several parameters (experience, if someone vetted them, etc), crowdworker would enter Daemo at a certain rank. Completely new individuals with relatively low experience would be still given a chance (unless there are big "no no" in their application or the way they behave, like aggressive people), but with lower rank at their entrance. They will have to climb the ladder as the aforementioned examples have shown.

3. In crowdsourcing, how would you envision a guild leveraging its services to get the best price, engage requesters, attract talent?

A mix of several factors, many of which would be outside the scope of this page centered on the "human element". Still, for the "attracting talent" part, see the example of Wikistrat. As an individual, i would have had difficulty to sell my analyst skills to big institutional and corporate players, and wouldn't be capable of giving them a huge and compete input. "Together we are strong" motto has truth in it, Wikistrat, like our guild would do, is acting as an umbrella, making myself stronger, more heard, completing my skills and my views with others, unlocking clients i wouldn't be able to secure solo, etc.