What is the index?
The index assesses the risk of news domains in media markets around the globe of disinforming their public. The index is structured as a neutral, independent and transparent assessment of a news domain’s disinformation risk at the site level.
The index has been developed with a Technical Advisory Group of eight members from around the world. Index members include Camille François (Graphika) and Dr. Scott Hale (Meedan/Oxford Internet Institute), Olaf Steenfadt (Reporters without Borders/Journalism Trust Inititiave, and Cristina Tardáguila (International Fact Checking Network/Poynter),
How is the index computed?
The GDI risk rating is based on disinformation ‘flags’ that can be combined to help give an accurate risk assessment of a news domain. The rating is determined by an automated and manual assessment of the news domains in any given media market.
What are the pillars?
The index has four pillars:
- Structure: This pillar is automated and looks at the different metadata and computational signals of the news domains. It uses artificial intelligence that has been developed based on a sample of 20,000 known disinformation domains
- Content: This pillar manually assess different disinformation flags for articles published to a specific domain. These include an article’s credibility, sensationalism, hate speech and impartiality. It is based on a review of 10 of the top-shared articles on a domain that have been randomly selected. The review is done by a researcher.
- Operational: This pillar assesses the underlying policies and rules that domains abide by to establish trust and reliability. The disinformation flags are drawn from the Journalism Trust Initiative (JTI). They include conflict of interest and the lack of operational integrity and accountability. The review is done by a researcher currently but it will eventually be based on data from the JTI standard.
- Context: This pillar assesses the reputational practices, reliability, and trustworthiness of a news domain. These disinformation flags are assessed by an expert survey. There are 50-100 experts per country.
The index is being piloted in South Africa and the UK. Findings will be shared in November 2019. As part of the pilot, a total of 30 media outlets are being assessed in each country.
How is the index scored?
Each domain in the index will be assigned a risk rating based on a score of 0 to 100. A lower score indicates a higher risk of disinformation, while a higher score indicates a lower risk.
Based on the aggregate score across the 4 pillars, each news domain is assigned a risk-level for disinformation: low, medium-low, medium, medium-high or high. The GDI rating does not assess whether a specific news domain is actually carrying disinformation. This differentiation is critical.
How will the data be used?
The data will provide a disinformation risk rating for each news domain assessed. It does not determine which news stories are inaccurate or which news domains are carrying disinformation. The risk ratings are intended to help brands and ad tech firms have more transparent and trusted information about the news sites where they place ads.
By knowing the disinformation risk levels of a site, brands and ad tech firms can make more informed decisions about where ads are served, based on their own self-determined risk thresholds.