What is the index?
The index provides disinformation risk ratings for news sites in media markets around the globe. The risk ratings are neutral, independent and transparent. The ratings are done at the site level. They provide for an unbiased ‘gold standard’ to assess disinformation risks. The GDI and its ratings are non-partisan and non-political.
The index has been developed with a Technical Advisory Group of eight members from around the world. Index members include Camille François (Graphika) and Dr. Scott Hale (Meedan/Oxford Internet Institute), Olaf Steenfadt (Reporters without Borders/Journalism Trust Inititiave), and Cristina Tardáguila (International Fact Checking Network/Poynter),
How is the index computed?
The GDI risk rating is based on expert-identified disinformation ‘flags’ that can be combined to help give an accurate and unbiased risk assessment of a news domain. A site’s risk rating is determined by an automated and manual assessment. The GDI produces risk ratings for news domains in any given media market.
What are the pillars?
The index has four pillars:
- Structure: This pillar is automated and looks at the different metadata and computational signals of the news domains. It uses artificial intelligence that has been developed based on a sample of 20,000 known disinformation domains
- Content: This pillar manually assess different disinformation flags for articles published to a specific domain. These include an article’s credibility, sensationalism, hate speech and impartiality. It is based on a anonymised review of 10 of the top-shared articles on a domain that have been randomly selected. The review is done by a researcher and the source of the articles is not disclosed to them.
- Operational: This pillar assesses the underlying policies, standards and rules that domains abide by to establish trust and reliability. The disinformation flags are drawn from the universal standard that has been set out by the Journalism Trust Initiative (JTI). This pillar focuses on conflict of interest and a site’s operational integrity and accountability. An independent researcher reviews whether a site has JTI-compliant policies. Eventually, the findings will be based on what sites self-report to the JTI standard.
- Context: This pillar assesses the reputational practices, reliability, and trustworthiness of a news domain. These disinformation flags are assessed by an independent expert survey of respondents from across the political spectrum.
The index has been piloted in South Africa and the UK. As part of the pilot, a total of 30 media outlets have been assessed in each country. Findings for the overall media market are available here.
How is the index scored?
Each domain in the index will be assigned a risk rating based on a score of 0 to 100. A lower score indicates a higher risk of disinformation, while a higher score indicates a lower risk. The scores are not binary. No score should be used to claim a site is a disinformation domain (or not).
Based on the aggregate score across the 4 pillars, each news domain is assigned a risk-level for disinformation: low, medium-low, medium, medium-high or high. The GDI rating does not assess whether a specific news domain is actually carrying disinformation. This differentiation is critical.
How will the data be used?
The data provides a neutral, trusted and unbiased disinformation risk rating for each news domain assessed. It does not determine which news stories are inaccurate or which news domains are disinformation sites. The ratings look at the signals to determine a site’s overall disinformation risk. The ratings are not to be used for censoring content or sites, curtailing freedom of expression, or limiting media freedoms.
The risk ratings are intended to help brands and ad tech firms have more transparent and trusted information about the news sites where they place ads.
By knowing the disinformation risk levels of a site, brands and ad tech firms can make more informed decisions about where ads are served, based on their own self-determined risk thresholds.
The data can also be used by platforms as part of an array of signals that they use to determine which sites to up-rank or down-rank.