This workshop introduces the basics of networked disinformation and manipulation campaigns, helping participants acquire a methodology for discovering, mapping and tracking networks of disinformation in different languages and contexts. Participants gain awareness that disinformation and manipulation campaigns are multi-dimensional, and that digital investigators need to adopt a networked approach (looking for connections on multiple platforms) in order to "expose the invisible" behind disinformation campaigns.
Workshop Overview
Topic: How to discover, map and track networks of disinformation.
Aims:
- To introduce participants to the basics of networked disinformation and manipulation campaigns;
- To acquire a methodology for discovering, mapping and tracking networks of disinformation in different languages and contexts;
- To raise awareness that disinformation and manipulation campaigns are multi-dimensional, and that digital investigators need to adopt a networked approach (looking for connections on multiple platforms) in order to 'expose the invisible' behind disinformation campaigns.
General guidelines for trainers:
- This workshop can be divided into 30-50 minute sessions. Between sessions, a short break or a quick energizer activity can be added.
- For small group activities, divide participants into groups of 3-5 people.
- You can assign roles depending on the number of participants. The roles can include: Facilitator, Recorder/Note-taker, Timekeeper, Presenter, Artist (whenever a visual presentation is required.)
- For online workshops, we recommend sharing a timer on the screen during energizers and small group activities.
- Whenever possible, adapt the examples and cases to the context of your audience.
Mode of delivery: online / in-person workshops
Workshop duration (without breaks): 4 hours (including optional activities)
Number of participants: 6 to 24
Related workshops: this workshop can be combined with “Safety First: Basics of preventive digital safety”.
Related guides, resources:
- "Media Manipulation Code Book" (Archived with the Wayback Machine on 29 May 2022 here), by the Technology and Social Change Project at the Harvard Kennedy School Shorenstein Center for Media, Politics, and Public Policy. Full project repository.
Workshop activities and templates, to download:
- "Networks of Disinformation (NoD) - Activity Sheet" (Excel file to download)
- "Networks of Disinformation (NoD) Sheet - Example" (Excel file to download)
- External Tool - CrowdTangle Chrome Extension - some workshop activities include the use of this extension; if you decide to use it, we recommend asking participants to install it in advance (it also needs Chrome or Brave browser installed).
NOTE about the 'NoD Activity Sheets' and how to use them
The "Networks of Disinformation (NoD) - Activity Sheet":
- The NoD sheet can be used during the workshop by both the trainer and the participants. The trainer can use it to show how to collect data about case studies. Participants are invited to use it for notes and findings during each workshop activity. We recommend sharing it with the participants at the beginning of the workshop or even before, to give them time to familiarize themselves with its structure.
- The sheet has two tabs: 1. Campaign Mapping - to be used in the "How disinformation and manipulation campaigns work" section; and 2. Networks Mapping - to be used in the following sections on "The 4-step methodology".
- For each step, you will find instructions about which column should be filled.
- Note that as the network mapping process is often not so linear, you will sometimes need to go back and forth between the different steps.
The "Networks of Disinformation (NoD) Sheet - Example"
- This is a pre-filled version of the "Networks Mapping" sheet, which you can use as an example.
- We have used a case study from a BuzzFeed investigation (archived here) on a network of disinformation websites operating in Italy.
- You can adapt the case studies and make your own version of a pre-filled sheet based on other local/regional cases that may be more relevant to your workshop audience.
Learning Activities
Opening (20 minutes)
Workshop Introduction
Read Watch Listen | 5 minutes
Tools / Materials:
- "Networks of Disinformation (NoD) - Activity Sheet" (Excel file to download)
- "Networks of Disinformation (NoD) Sheet - Example" (Excel file to download)
Instructions
-
Grab attention by posing a question or commenting on a relevant topic, image, etc.
-
Introduce yourself and the goals of the workshop.
-
Inform participants of the workshop agenda.
-
Introduce the workshop topic and the 4-step methodology for discovering, mapping and tracking disinformation networks.
-
Show the Networks of Disinformation (NoD) Sheets and explain how it will be used.
-
Suggest ground rules for the workshop: how you expect participants to act and react, respect each other, etc. If you are running a longer training you could also consider working on a commonly agreed Code of Conduct or shared agreement (see some tips in the "Shared Agreements" section of Tactical Tech's Gender and Tech curriculum guide.)
Participants' introductions / Quiz
Produce | 15 minutes
Tools/Materials
- For online: Quiz tool (Kahoot, Slido or equivalent)
- For offline: printed sheets of paper with the quiz below / post it notes, pens, whiteboard, flipchart paper.
Instructions
[5 minutes] Introductions
- Make a quick round of introductions by asking participants to answer a couple of questions about themselves, their work and / or their workshop expectations.
[5 minutes] Quiz
- Launch an interactive quiz session with Kahoot, Slido or another quiz tool of your choice, asking participants to answer the following questions. - The goal of the quiz is not to assess participants' competencies, but to introduce the main topics of the workshop in a participatory way.
-
Which tool do you use if you want to know who has registered the domain https://sputniknews.com/?
- CrowdTangle
- Whoisology
- BuzzSumo
-
What is the following code "ca-pub-1234567891234567"?
- Google AdSense code
- IP address
- Google Analytics code
- Facebook Pixel code
-
Do you know any cases of investigations exposing disinformation/manipulation networks in your country/language?
- Yes (If Yes, please share the main finding(s) in one sentence.)
- No
If you run this workshop in person, we still recommend using an online quiz. However, an offline alternative would be to provide the questions and answer options on a large flip-chart paper or whiteboard (listing the three questions and answers as columns). Participants can stick post-its or make marker notes with their answers. This allows for easier review by trainers and participants.
[5 minutes] Debriefing
- Invite some volunteers to share their answers to the quiz and comment.
- Confirm or provide the right answers to the quiz.
How disinformation and manipulation campaigns work (40 minutes)
Case studies and key definitions
Read Watch Listen | 15 minutes
Tools / Materials
- For online: computer and shared screen function for presentations
- For offline: computer and projector
- Media Manipulation Casebook (to help prepare the session)
Instructions
Prepare a presentation including the following points:
-
Case studies - Briefly show some examples of journalistic investigations unveiling disinformation networks:
- A network of Italian anti-Immigrant websites (BuzzFeed)
- A network of pro-Russia websites (Graphika, diagram p. 6)
- A network of pro-Huawei websites (Twitter)
- The Kumanovo-connection: a network of Macedonian websites (Lead Stories)
-
Networked disinformation campaigns: taxonomy - Explain how online media manipulation operations work, focusing on channels, tactics, scale and impact definitions, based on the Media Manipulation Casebook - Code Book:
-
Channels used for spreading the campaign:
- Owned websites
- Social media (Facebook, Twitter, YouTube)
- Messaging apps (WhatsApp, Telegram, Signal, etc)
- Editorial platforms (Medium, etc)
- Other examples
-
Tactics used for spreading the campaign:
- Social media amplification
- Community building
- Bots
- Advertising
- Copypasta
- Forgery
- Typosquatting
- Keyword squatting
-
Scale of the campaign / region or context targeted
- Hyperlocal
- Regional
- National
- International
-
Quantitative reach of the campaign
- Number of potential people reached on different channels (websites, social media accounts, street campaigns, etc.)
-
Goals of the campaign
- Abuse and personal harm
- Mass influence of opinions and attitudes
- Inciting to hatred or violence
- Reputational harm
- Social unrest
- …
RESOURCES:
- "Media Manipulation Casebook - Code Book" (Archived with the Wayback Machine on 29 May 2022 here), by the Technology and Social Change Project at the Harvard Kennedy School Shorenstein Center for Media, Politics, and Public Policy, Project repository.
Identifying main aspects of networked disinformation campaigns
Collaborate | 25 minutes
Materials
- Networks of Disinformation (NoD) Sheet, tab named "Campaign Mapping"
- Break-out rooms (for online) / or separate areas/tables in the room (for offline)
Instructions
[20 minutes]
-
Divide participants in groups of 3-5 people, randomly (depending on the number of participants.)
-
Ask each group to analyze one of the following case studies:
- A network of Italian "Anti-Immigrant" websites (BuzzFeed)
- A network of pro-Russia websites (Global Voices)
- A network of pro-Huawei websites (Twitter)
- The Kumanovo-connection: a network of Macedonian websites (Lead Stories)
-
Ask each group to fill in the "Campaign Mapping" tab in the (NoD) Sheet. For each case study, they should list:
- channels
- tactics
- scale
- impact
- goals
- …according to the taxonomy presented in the previous section.
You can adapt the examples to your audience's needs/context or ask them to come up with examples if they have basic awareness or experience.
Debriefing
[5 minutes]
- Participants share the findings of the "Campaign Mapping" activity
- You can comment and add to their findings.
Step 1: Identifying disinformation websites (40 minutes)
Step 1: Identifying a disinformation website
Read Watch Listen | 15 minutes
Tools / Materials
- For online: computer and shared screen function for presentations
- For offline: computer and projector
Instructions
Prepare a presentation including the following points:
-
Introduce this step by stressing why it is important to anchor your investigation to one or more websites:
- Websites let you have access to historical data, business connections and network dynamics that usually are not available for social media accounts.
- So it's important to identify one or more origin websites to start from.
-
Explain how disinformation/manipulation networks work. Usually they exploit two main strategies:
- Offering a beat coverage of polarizing topics (Covid-19 vaccines, migration, etc.)
- Covering "data voids" (example: "ivermectin"), especially during breaking news events or spreading conspiracy theories (example: "Adrenochrome")
-
Introduce the three main ways to identify a website for analysis**:
- Exploring a database of disinformation websites, like the list of fake news websites on Wikipedia, UkraineFacts, Covid-19 Misinformation Database, Russia-Ukraine Disinformation Tracking Center
- Looking for most shared stories on current affairs events on social listening services (like Buzzsumo, CrowdTangle Chrome Extension or CrowdTangle if available.)
- Identifying a "data void" (information gap) and running searches on different services
-
Show how these methodologies work through three case studies:
- Wikipedia List of Fake News Websites - "The Gateway Pundit" is listed. If you run a relationships search on Builtwith you can see it's part of a bigger network of websites
- Covid-19 Database - Search for "ivermectin" (screenshot). You get this AFP Story talking about an article on the website "The Gateway Pundit" (link - archived link). - If you run a relationships search on Builtwith you can see it's part of a bigger network of websites
- Buzzsumo - Search for "Adrenochrome" (screenshot). You get a result for WeloveTrump.com (link / archived link) - If you run a relationships search on Builtwith, you can see it's part of a bigger network of websites
Identifying disinformation websites active in your country
Investigate | 25 minutes
Tools/Materials
- Social listening (Buzzsumo, CrowdTangle Chrome Extension or CrowdTangle if available.
- Fact-checking websites and databases: UkraineFacts, Covid-19 Misinformation Database, Russia-Ukraine Disinformation Tracking Center, Wikipedia list of "Fake news websites"
- Break-out rooms (for online) / or separate areas/tables in the room (for offline)
- Networks of Disinformation (NoD) Sheet: tab named "Campaign Mapping" / columns labeled "Email", "IP", "Google Analytics", "Google Adsense"
- Networks of Disinformation (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Name", "Domain Name".
Instructions
[20 minutes]
-
Divide participants in groups of 3-5 people (if there are more participants working on the same area/language they can team up accordingly). From here on, the same groups will work together throughout the workshop.
-
Based on the tools introduced above, ask participants to conduct a small investigation to find websites spreading disinformation in their own countries or languages.
-
Ask participants to add the identified websites (links) on the (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Name", "Domain Name".
[5 minutes] Debriefing
-
Participants share the websites they have identified, emphasizing the process they have used (rather than the content of the websites)
-
You can comment and add to their ideas as well as ask them to share challenges encountered along the way.
Step 2: Who is behind a website? (25 minutes)
Step 3: Who is behind a website
Read Watch Listen | 10 minutes
Materials/Tools
- For online: computer and shared screen function for presentations
- For offline: computer and projector
- Websites for demonstration: Whois Domain Tools; Whoisology.com; Whoxy.com
Instructions
-
Explain how "Whois" tools work, highlighting limits and possibilities
-
Run a search on https://whois.domaintools.com/ with an example websites (like http://sputniknews.com/ - if the website is not restricted in your area - or ZeroHedge.com) and explain what are the most relevant data:
- not just registrant name/address/contacts, but also date of registration, date of last update, etc.
-
Show some of the tools available:
- Whoisology: useful for "connected domains" info
- Whoxy: useful for historical data
RESOURCE:
- How to See What's Behind a Website Guide, Exposing the Invisible Kit
Whois practice
Produce | 15 minutes
Tools/Materials
- Websites for demonstration: Whoisology.com; Whois Domain Tools; Whoxy.com; Networks of Disinformation (NoD) Sheet
Instructions
[10 minutes]
-
Ask participants to return to their groups
-
Using the tools introduced above, each group conducts a search for the websites identified in the previous step
-
Ask groups to collect available data on the (NoD) Sheet: tab named "Networks Mapping" / columns labelled "Whois"
-
Suggest that each group assigns a note-taker and a presenter for each activity (these roles can rotate)
[5 minutes] Debriefing
-
Participants share what they found and possible problems they have encountered
-
You can comment and add to their ideas.
Step 3: Follow the money! (35 minutes)
Step 3: Reverse IDs
Read Watch Listen | 20 minutes
Tools/Materials
- For online: computer and shared screen function for presentations
- For offline: computer and projector
- Websites for demonstration: Builtwith; Dnslytics; SpyOnWeb; AnalyzeId; Wayback Machine
- Networks of Disinformation (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Email", "IP", "Google Analytics", "Google AdSense"
Instructions
Give a presentation including the following points:
-
Introduce the concept of Tracking IDs (tracking identification number allocated to each website) and briefly explain how they work
-
Show how to find IDs on any website: View the website source code ("ctrl+U" or "command+U" depending on device) > Search for "UA-" or "ca-pub"
- you can demo this on any website such as: https://historyofvaccines.org/)
-
If IDs are not available, you can try to go back with Wayback Machine and see if any ID codes were available in the past
-
Show the different tools available for tracking AdTech IDs. Always check the same ID on different services, because they have different databases and different pricing options.
-
Builtwith: It's the best option to start with. The "Relationship profile" tool is very useful for finding shared IDs and trackers. Example welovetrump.com (Link – Screenshot)
-
Dnslytics: it has a very good database. Let you "reverse" different kinds of information present on a website: Adsense, Analytics, IP, Mail server, Name Server (Screenshot) with example of a search results for thegatewaypundit.com). You have a free version, or a monthly pass fee.
-
SpyOnWeb: you can use this tool for confirming or expanding results you get on other services (Link - Screenshot). It seems to have a smaller database than other services, but sometimes it helps you find new information.
-
AnalyzeId: Very good tool. It shows you results on a table, with a lot of IDs connections, including Amazon affiliate, Sharethis, email, Facebook apps, and a confidence rating (Link - Screenshot). You can also export data in .csv format. The only problem is that it doesn't show you all the results. To see everything you need to subscribe to a paid monthly plan.
-
Wayback Machine: When you don't find any ID on a website, you can try to do a search on Wayback Machine and see if you find any ID in a older version of the website. You can find very useful information by doing this!
-
-
Remember: run multiple searches on the different tools
-
Demonstrate how participants can collect the ID data of websites in the (NoD) Sheet, tab named "Networks Mapping" / columns labeled "Email", "IP", "Google Analytics", "Google ASsense".
- This serves as a short methodology demo to guide them for the next practical activity, you can use the provided case from the pre-filled template or provide another one that is more relevant to your audience.
RESOURCES:
When preparing the presentation on IDs, you can use resources such as:
- "Investigating websites" section of the "Verification Handbook For Disinformation And Media Manipulation", by Craig Silverman.
- "Unveiling Hidden Connections with Google Analytics Ids", by Lawrence Alexander, Bellingcat, July 23, 2015 (some tools may be outdated but the method and most of the resources are valid.)
Reverse IDs practice
Produce | 15 minutes
Tools/Materials
- Break-out rooms (for online) / or separate areas/tables in the room (for offline)
- Websites: Builtwith.com; Dnslytics; SpyOnWeb; AnalyzeId; [Wayback Machine](https://archive.org/web/;
- Networks of Disinformation (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Email", "IP", "Google Analytics", "Google Adsense".
Instructions
[10 minutes]
-
Ask participants to return to their groups, with the following instructions:
-
Using the tools introduced above, conduct a search on the websites identified during Step 2
-
Collect available data on (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Email", "IP", "Google Analytics", "Google Adsense"
-
Highlight websites having the same IDs and to start to cluster them
Debriefing
[5 minutes]
-
Participants share what they found and possible issues they have encountered
-
You can comment and add to their findings.
Step 4: Social Amplification (30 minutes)
Step 4: Social Amplification
Read Watch Listen | 15 minutes
Tools/Materials
- For online: computer and shared screen function for presentations
- For offline: computer and projector
- CrowdTangle Chrome Extension
- Networks of Disinformation (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Facebook", "Instagram", "Twitter", "Other social media"
Instructions
Give a presentation including the following points:
-
Introduce the concept of Social Amplification as the process of distributing content to as many people as possible via various channels and strategies.
-
Starting from previously featured case studies, show participants how social amplification is relevant for spreading online disinformation and manipulation campaigns.
-
Show participants how CrowdTangle Chrome Extension works on the featured websites (NOTE that CrowdTangle requires signing up with a Facebook or Instagram account).
- Use a pre-tested sample case. You can also try some of the examples found by participants.
-
Demonstrate to participants how they can find data on CrowdTangle and how to collect it in their selected websites in the (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Facebook", "Instagram", "Twitter", "Other social media".
-
Mention that in the next section, participants should replicate this activity, taking into consideration the websites they have found before.
RESOURCE:
- For preparing the presentation and demo: "CrowdTangle: frequently asked questions"
Social tracking practice
Produce | 15 minutes
Tools/Materials
- Break-out rooms (for online) / or separate areas/tables in the room (for offline)
- CrowdTangle Chrome Extension
- Networks of Disinformation (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Facebook", "Instagram", "Twitter", "Other social media"
Instructions
[10 minutes]
-
Ask participants to return to their groups, with the following instructions:
- Install the CrowdTangle Chrome Extension on their browsers (if they haven't done so yet; ideally this should already be installed by the start of the workshop.)
- Run multiple searches on the most interesting website(s) previously identified in the initial local/country exercise.
- Collect linked social accounts data on (NoD) Sheet: tab named "Networks Mapping" / columns labeled "Facebook", "Instagram", "Twitter", "Other social media"
Debriefing
[5 minutes]
-
Participants share what they found and possible problems they have encountered
-
You can comment and add to their findings.
[Optional] Team Investigation (30 minutes)
Identifying disinformation networks
Investigate | 30 minutes
Tools/Materials
- Break-out 'rooms' (for online) / or separate areas/tables in the room (for offline)
- Networks of Disinformation (NoD) Sheet: tab named "Networks Mapping" / all the columns they have used in previous activities
Instructions
[25 minutes]
-
From the disinformation networks discovered by participants during the previous group activities, pick the most interesting case from each group and assign it to them for this task.
-
Ask participants to return to their groups, with the following instructions:
- Expand the search of connections between your website and any related social media accounts
-
Collect identified data on (NoD) Sheet: tab named "Networks Mapping" / all the columns they have used in previous activities. (Use the "Notes" column of the NoD sheet for any additional remarks or questions / challenges to address in the Debriefing.)
-
Groups should assign a note-taker (to help fill out the sheet with findings / notes) and a presenter for their findings.
Debriefing
[5 minutes]
-
Participants share what they found and possible problems they have encountered
-
You can comment and add to their findings.
Closure (15 minutes)
Wrap-up Activity: Takeaway Poster
Produce | 5 minutes
Tools/Materials
- Shared drawing pad / slide / whiteboard (online)
- Whiteboard / flip-chart paper, post-its, markers (offline)
Instructions
-
Ask participants to create a takeaway poster by sharing their answers to the following question in the shared whiteboard / drawing board:
- What are your main takeaways from today's workshop?
-
Give participants a few minutes to write and/or draw their thoughts and read the thoughts of others.
Debriefing
- Highlight some of the points on the board.
Conclusion
Read Watch Listen | 10 minutes
Tools/Materials: No materials needed.
Instructions
-
Wrap up the workshop and sum up its contents.
-
Run a quick review of the session. Each participants would say:
- one thing they found very good about the session and
- one thing they would improve for the next time
-
You can encourage participants to ask questions or give some final tips.
-
Share contact information if relevant and any follow-up details.
To keep participants informed about what is going on at all times, trainers can effectively sum up workshop contents following these steps:
-
- [in the introduction] tell participants what is going to happen;
-
- [during each part of the session / workshop] remind them what is happening;
-
- [at the end of the session/workshop] tell them what just happened. In addition, at the end, trainers need to make sure they point out which expectation have been addressed.
Further Resources
-
"Get Your Facts Straight: The Basics of Fact-Checking", by Deepak Adhikari, guide in the Exposing the Invisible: The Kit, Tactical Tech
-
"Information Disorder: The Definitional Toolbox" — in three parts: "Glossary"; "Mapping the Landscape"; "Useful Graphics"), by Claire Wardle, FirstDraft.
Contact Us
Please reach out to us at Exposing the Invisible if you:
- have any questions about this workshop plan and facilitation guidelines,
- use this workshop plan and want to share feedback and suggestions that can help to improve them,
- adapt the workshop plan to a specific context and want to share the results with us,
- want to suggest new activities, tips or examples that can be added to this workshop,
- want to share your expertise and collaborate with us on developing and testing new workshops.
Contact: eti@tacticaltech.org (GPG Key / fingerprint: BD30 C622 D030 FCF1 38EC C26D DD04 627E 1411 0C02).
Credits and Licensing
CC BY-SA 4.0
This content is produced by Tactical Tech's Exposing the Invisible project, and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license
- Workshop author: Nicola Bruno
- Editorial and content: Christy Lange, Laura Ranca
- Instructional design: A. Hayder
- Graphic design: Yiorgos Bagakis
- Website development: Laurent Dellere, Saqib Sohail
- Project coordination and supervision: Christy Lange, Laura Ranca, Lieke Ploeger, Marek Tuszynski, Safa Ghnaim, Wael Eskandar
This resource has been developed as part of the Collaborative and Investigative Journalism Initiative (CIJI) co-funded by the European Commission under the Pilot Project: "Supporting investigative journalism and media freedom in the EU" (DG CONNECT).
This text reflects the author’s view and the Commission is not responsible for any use that may be made of the information it contains.