Crowdsourcing Information for Investigations

This workshop provides participants with the essential methods, tools, examples and considerations to be aware of when planning and managing a crowdsourcing effort. It also demonstrates the role of crowdsourcing as a method increasingly used by journalists, activists, researchers and others to engage with communities to collect information, verify it and build evidence that can help expose issues affecting societies everywhere.

Learning Activities

Opening (15 minutes)

Instructions for trainers

• Grab attention by posing a question or commenting on a relevant topic, image, etc.

• Introduce yourself and the goals of the workshop.

• Optional: Introduce the source of the workshop material (Tactical Tech.)

• Inform participants of the workshop agenda.

• Suggest ground rules for the workshop. Ask participants whether they would like to modify your suggestions or suggest other rules. Ensure that everybody understands and agrees with the ground rules. Specific suggestions about setting ground rules are available in the Facilitator's Guide, section on "Delivering the Workshops".

• Establish the dynamics for groups if needed. For instance, explain if the same groups of participants will need to work together across the entire workshop.

Instructions

• Make a quick round of introductions by asking participants to answer a couple of questions about themselves, their work, their workshop expectations, etc.

  • If you can afford a playful activity with your group, you could also launch a crowdsourcing call for details related to a specific topic or question that participants can relate to (e.g. ask around "who is an experienced researcher?", "who has design skills?", "who has knowledge about digital safety?", etc.)

• Alternatively, you can pick an icebreaker exercise that encourages participants to get creative by drawing answers or ideas on an online whiteboard or, if in-person, stand up and perform some tasks. Check the "Icebreakers" section in the ETI Facilitator's Guide for inspiration.

Introduction to Crowdsourcing (35 min)

Tools / Materials:

• Slides for a power point presentation or pre-written flip-chart papers (for offline)

Instructions

Prepare and give a short presentation including:

• A brief overview of what crowdsourcing is and a short history of how it came to be.

• Provide examples of civil society projects and actions based on crowdsourcing.

• Highlight some considerations for using crowdsourcing, including:

  • Pros of crowdsourcing:

    • Allows tapping into a vast pool of data otherwise inaccessible to organizers
    • Allows engaging diverse contributors
    • Can help save time and costs
    • Opens new avenues for collaboration with contributors and/or others working in the same space

  • Cons of crowdsourcing:

    • Carries the danger of manipulation
    • May require a lot of know-how and resources (e.g..  for setting up tech tools, verifying collected data, and other)
    • Carries the danger of coming up empty-handed (success of crowdsourcing very much depends on effectively engaging contributors)
    • May carry potential risks to organizers and contributors (e,g, when crowdsourcing sensitive data)

• It is important to separately assess each potential case of applying crowdsourcing.

• There are ways of mitigating the risks outlined above. But when the Cons outweigh the Pros, alternatives to crowdsourcing may be considered.

• Examples of who may crowdsource evidence and for what purpose. They may include: media, investigators, rights defenders, other activists, etc.

RESOURCE:

• To prepare this session, use the “Crowdsourcing Reference Document - Session 1: Introduction to the topic” (download PDF or Word file.

Tools / Materials:

• A shared cloud space or a digital whiteboard with the templates: Miro, Mural, etc. (if online.)
• Digital break-out rooms (if online) or separate room areas / tables (if offline/in person.)
• Individual sheets of paper, post-its, pens (if offline.)

Instructions

[10 minutes]

• Divide participants into small groups of 3-5 people each. Recommend that each group allocates specific roles during their activities: Note-Taker, Time-Keeper, Presenter (roles can be rotated), etc.

• Ask groups to brainstorm on the topic:

  • Reasons why different actors may want to crowdsource evidence

• The list of answers may include: journalists who want to proactively collect certain types of content from social media or invite citizens to report something directly, rights defenders or watchdog organizations choosing to monitor certain issues / events by inviting citizens to report or map findings and situations, etc.

• They can take notes in shared files or a whiteboard.

[10 minutes] Debriefing

• Back in the main group/room, a representative of each group shares their findings, 1 minute per group.

• Guided by the facilitator, the whole group examines Pros and Cons for using crowdsourcing for several of the cases they listed and decide whether the method is appropriate.

Setting up a Crowdsourcing Effort (45 min)

Tools / Materials:

• Slides / flip-chart papers with pre-filled activity instructions.
• A shared cloud space or a digital whiteboard with the templates: Miro, Mural, etc. (if online.)
• Digital break-out rooms (if online) or separate room areas / tables (if offline.)
• Individual sheets of paper, sticky notes / post-its, pens (if offline.)
• “Crowdsourcing Activity Templates”, section “Group Activity: Planning a crowdsourcing project” for participants to fill out (online or printed)

Instructions

[5 minutes]

• Give a brief presentation telling participants that before they decide to engage in crowdsourcing, it is important to ask a number of questions. The questions – also useful for discussions in the following activity - are:

  • Why do you want to crowdsource?
  • Who are your contributors?
  • Who are your beneficiaries?
  • What are the main ethical considerations when crowdsourcing evidence?
  • What are the risks?
  • How verifiable is the data collected?
  • Who could become your partner or collaborator?
  • What happens with data afterwards?
  • Should any credit be given and how?

[25 minutes]

• Divide participants into small groups of 3-5 people each. The same groups should work together during the entire workshop from here on.

• Remind groups to allocate specific roles during their activities: Note-Taker, Time-Keeper, Presenter, etc.

• Ask each group to choose an idea / a project (real or fictitious) that would apply crowdsourcing in one of the three generic areas:

  1. a journalistic investigation,
  2. a crisis situation requiring information gathering, or
  3. improving public accountability.

• Groups discuss their respective crowdsourcing “projects” along the lines of main considerations listed below and brainstorm steps they would take to address them.

  • Goals and purpose
  • Target audiences
  • Risks and safety
  • Ethical considerations
  • Reliability of findings
  • Possible collaborators
  • Outputs
  • Credits

• Groups take notes in the template table provided (see “Materials” above.)

[15 minutes] Debriefing

• Back in the main group/room, groups present the results of their work, answer questions and receive feedback from the facilitator.

RESOURCE:

• To prepare this session, use the “Crowdsourcing Reference Document - Session 2: Setting up a crowdsourcing effort” (download PDF or Word file.)

Choosing the Right Crowdsourcing Approach (30 min)

Tools / Materials:

• Slides / flip-chart papers for presentation, prepared in advance.

Instructions

Prepare and give a short presentation including:

• How choosing the right approach depends on one’s goals as well as audiences. For example:

  • journalists might set up a secure channel for citizens to anonymously send in tips,
  • rights defenders may encourage victims to submit evidence of abuse in whatever format they may have it, or
  • election observers may want voters to attempt to categorize the type of irregularity they witness according to some pre-set criteria, etc.

• Present structured vs unstructured data collection methods, emphasizing when each may be suitable, depending on needs, goal, method of data analysis, etc.

• Highlight the difference between “open” and “specific” calls to contribute data.

• Highlight the difference between crowdsourcing data in an unstructured and structured way.

• List the Pros and Cons of each approach.

RESOURCE:

• To prepare this session, use the “Crowdsourcing Reference Document - Session 3: Choosing the right crowdsourcing approach & Session 4: Working with target audience(s) of the crowdsourcing effort” (download PDF or Word file)

Tools / Materials:

• Slides / flip-chart papers with pre-written activity instructions.
• A shared cloud space or a digital whiteboard with the templates: Miro, Mural, etc. (if online.)
• Digital break-out rooms (if online) or separate room areas / tables (if offline.)
• Individual sheets of paper, sticky notes / post-its, pens (if offline.)
• “Crowdsourcing Activity Templates”, section “Group Activity: Identifying the suitable approach” for participants to fill out (online or printed)

Instructions

[10 minutes]

• Ask participants to go back to their previous groups.

• Each group discusses and takes notes about:

  • Which approach would be suitable for their crowdsourcing project idea they came up with in the previous session?

[10 minutes] Debriefing

• Groups present their choices and explain the rationale behind them – allocate 1-2 minutes for each group’s updates.

• Participants receive feedback from the facilitator.

Choosing Technical Tools for Crowdsourcing (15 min)

Tools / Materials:

• Slides / flip-chart papers for presentation, prepared in advance.

Instructions

Prepare and give a short presentation about:

• How to choose tools for crowdsourcing, emphasizing the following:

  • there is an abundance of easy and secure tools developed by and for human rights defenders, journalists, or citizen watchdogs,
  • it is important to choose the right tool that would align with the goals and needs of a crowdsourcing effort, not try to design the crowdsourcing operation around it,
  • avoid getting excited about the latest tools - sometimes, a project may not need the latest and coolest technology, but a simple telephone hotline, text messages, email, etc.

• In order to make the best choices, it is important to consider:

  • technical environment (high vs low tech), including tech habits of the target audience,
  • security and privacy of your contributors,
  • costs, time, and technical skills needed to set up particular tools.

• Provide examples of tools, pros and cons of each, including the costs and the time it takes to set each tool up, as well as links to further information and set up tutorials (see the "Trainer’s Tips" and "Reference Document" below.)

• Invite participants to share any of their relevant experience with using tools for gathering information and/or any interesting case they know about.

RESOURCES: To prepare this session, use:

• “Crowdsourcing Reference Document - Session 5: Choosing technical tools for crowdsourcing” (download PDF or Word file)
• “Crowdsourcing: Trainer’s Tip Sheet” (download PDF or Word file).

Working with Your Target Audience(s) (40 min)

Tools / Materials:

• Slides / flip-chart papers for presentation, prepared in advance.

Instructions

Prepare and give a short presentation highlighting the following:

• The importance of effective community engagement for the success of a crowdsourcing effort

• Guiding questions to ask yourself before doing community engagement

• Methods and channels for successfully targeting and engaging potential contributors

• Privacy and security considerations

• Show relevant cases from published projects around the world (for examples, see the Trainer’s Tips and Reference Document files.)

RESOURCES:

• “Crowdsourcing Reference Document - Session 4: Working with target audience(s) of the crowdsourcing effort” (download PDF or Word file)
• “Crowdsourcing: Trainer’s Tip Sheet” (download PDF or Word file).

Tools / Materials:

• Slides / flip-chart papers with pre-written activity instructions.
• A shared cloud space or a digital whiteboard with the templates: Miro, Mural, etc. (if online.)
• Digital break-out rooms (if online) or separate room areas / tables (if offline.)
• Individual sheets of paper, sticky notes / post-its, pens (if offline.)
• “Crowdsourcing Activity Templates”, section “Group Activity: Strategy for reaching out to audiences” for participants to fill out (online or printed.)

Instructions

[15 minutes]

• Participants go to their respective groups.

• Ask each group to come up with:

  • a strategy (channels, methods, etc.) for reaching out and engaging people in the crowdsourcing effort.

[15 minutes] Debriefing

• Groups present their choices and explain the rationale behind them – allocate 2 minutes for each group’s updates.

• Participants receive feedback from the facilitator.

Verifying crowdsourced data (10 min)

Tools / Materials

• Slides / flip-chart papers for presentation, prepared in advance.

Instructions

Prepare and give a short presentation highlighting the following:

• Verifying crowdsourced data is extremely important before it can be considered as trustworthy (for proof / evidence.)

• Depending on the type and format of collected data, consider how much verification it requires and what expertise / capacity is needed to do so.

• A crowdsourcing project may end up with three types of data:

  • Unverifiable data: some data may be impossible to verify as it can be novel and lack corroborating sources. In this case, think of not verification, but vetting - attempt to refute data before publishing. If you’re unable to verify the data but still want to publicize it, provide a clear disclaimer marking it as “unverified.”
  • Partially verified data: decide how much verification is deemed “enough” for the data to be used / published.
  • Fully verified data: data that has several corroborating sources.

• Verifying crowd-mapped data.

• Verifying information on social media.

• Provide relevant cases from published projects around the world (for examples, see the Trainer’s Tips and Reference Document files.)

RESOURCES: To prepare this session, use:

• The “Crowdsourcing Reference Document - Session 6: Verification” (download PDF or Word file)
• “Crowdsourcing: Trainer’s Tip Sheet” (download PDF or Word file).

Analysing Data and Presenting Findings (35 min)

Tools / Materials:

• Slides / flip-chart papers for presentation, prepared in advance.

Instructions

Prepare and give a short presentation highlighting the following:

• Considerations about how to analyze and present findings.

  • It is important to present crowdsourced data in an honest and truthful manner.

• When publicizing findings, describe the methods of data collection and how conclusions were reached (in case of conducting analysis).

• On each related publication, clearly state whether and to what extent you have been able to verify crowdsourced data.

• Findings need to be presented in an interesting and engaging manner. One way to do it is to think of a “story” to tell with the data.

RESOURCES: To prepare this session, use the following:

• “Crowdsourcing Reference Document - Session 7: Analysing data and presenting the findings” (download PDF or Word file)
• “Crowdsourcing: Trainer’s Tip Sheet” (download PDF or Word file).

Tools / Materials:

• Slides / flip-chart papers with pre-written activity instructions.
• A shared cloud space or a digital whiteboard with the templates: Miro, Mural, etc. (if online.)
• Digital break-out rooms (if online) or separate room areas / tables (if offline.)
• Individual sheets of paper, sticky notes / post-its, pens (if offline.)
• “Crowdsourcing Activity Templates”, section “Group Activity: Share your findings” for participants to fill out (online or printed)

Instructions

[10 minutes]

• Participants go to their respective groups and initial crowdsourcing project idea. The task is to:

  • Think of the kind of data you may expect to receive.**
  • How will you analyse this data and present your findings?

[10 minutes] Debriefing

• Groups share their ideas in the plenary.

• Participants receive feedback from the facilitator and discuss any challenges and questions.

Closure (15 minutes)

Tools/Materials

• Shared drawing pad / slide / whiteboard (online)
• Whiteboard / flip-chart paper, post-its, markers (in-person)

Instructions

• Ask participants to create a takeaway poster by sharing their answers to the following question in the shared whiteboard / drawing board:

  • What are your main takeaways from today's workshop?

• Give participants a few minutes to write and/or draw their thoughts and read the thoughts of others.

Debriefing

• The facilitator can highlight some of the points on the board.

Tools/Materials: No materials needed.

Instructions

• Wrap up the workshop and sum up its contents. This is the moment to add any final words or highlight a main message.

• Run a quick review of the session. Each participants would say:

  • one thing they find very good about the session and
  • one thing they would improve for the next time

• Encourage participants to ask questions or give some final tips.

• Share contact information if relevant and any follow-up details.

Contact Us

Please reach out to us at Exposing the Invisible if you:

• have any questions about this workshop plan and facilitation guidelines,
• use this workshop plan and want to share feedback and suggestions that can help to improve them,
• adapt the workshop plan to a specific context and want to share the results with us,
• want to suggest new activities, tips or examples that can be added to this workshop,
• want to share your expertise and collaborate with us on developing and testing new workshops.

Contact: eti@tacticaltech.org (GPG Key / fingerprint: BD30 C622 D030 FCF1 38EC C26D DD04 627E 1411 0C02).

Credits and Licensing

CC BY-SA 4.0

This content is produced by Tactical Tech's Exposing the Invisible project, and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license

• Workshop author: Tetyana Bohdanova
• Instructional design: A. Hayder
• Editorial and content: Christy Lange, Laura Ranca, Wael Eskandar
• Graphic design: Yiorgos Bagakis
• Website development: Laurent Dellere, Saqib Sohail
• Project coordination and supervision: Christy Lange, Laura Ranca, Lieke Ploeger, Marek Tuszynski, Safa Ghnaim, Wael Eskandar

This resource has been developed as part of the Collaborative and Investigative Journalism Initiative (CIJI) co-funded by the European Commission under the Pilot Project: "Supporting investigative journalism and media freedom in the EU" (DG CONNECT).

This text reflects the author’s view and the Commission is not responsible for any use that may be made of the information it contains.