Global Digital Activism Data Set 2.0: Inclusion Criterion and Search Patterns

[UPDATED] The second version of the Global Digital Activism Data Set seeks to study digital persuasion campaigns around the world.  This post describes our method of developing an inclusion criterion and search patterns to create a sample of cases.

Conceptual Definition

The second version of the Global Digital Activism Data Set (GDADS2) looks at digital persuasion campaigns.  The conceptual definition is derived from a canonical campaign definition from the social movement scholar Charles Tilly:

A sustained, organized public effort making collective claims of target authorities (2004, p. 3).

The modifier “persuasion” was added to differentiate the campaigns in Tilly’s definitions from other types of collective action called campaigns in the popular lexicon, such as fundraising campaigns, mutual aid campaigns, and awareness-raising campaigns, where an “sustained, organized public effort” is made, but without claims of an authority figure.  The modifier “digital” was added to identify the use of a digital media in the campaign.  The conceptual definition of a digital persuasion campaign in this study is:

A sustained, organized public effort making collective claims of target authorities in which initiators or supporters use digital media.

Campaigns in the digital age are often leaderless and decentralized, with different actors and groups of actors operating independently or with minimal coordination.  For example, groups of supporters may implement tactics in support of the campaign goal without the campaign initiator’s knowledge (or consent).  Different actors may also have different targets because they perceive different entities as being able to implement the goal.  For these reasons a digital activism campaign is defined not by continuity of actors or continuity of targets, but by continuity of goal.  Participants in a campaign may not all be known to one another and may not seek to achieve the goal in the same way, but they must all be working to achieve the same goal.

Inclusion Criterion

The inclusion criteria used to creating the sampling frame is as follows.  The first four criterion are direct operationalizations of the conceptual definition.

  1. Digital (includes at least one tactic that uses digital media, by either a support or initiator)
  2. Organized public effort  (seeks to engage citizens as participants)
  3. Collective claims (goal(s) made on behalf of a group of citizens)
  4. Target (seeks to influence or otherwise affect an entity of authority perceived as having the ability to implement the goal)
  5. Civic (initiator group not exclusively composed of government of for-profit entities)
Two additional criteria were added to ensure the quality of information collected.
  1. Reliable 3rd party source “reputation for fact-checking and accuracy” (Wikipedia, 2013)
  2. Sufficient information to code (information on goal, target, initiator, and digital media used must be present, though it may be ambiguous.)
Finally, an addition pair of criteria were added to prevent redundancy of cases.
  1. Annual events only included once
  2. Campaigns are defined at their largest definable unit (subsidiary parts of larger campaign not identified as separate campaigns)

Search Patterns

We searched three types of sources for cases: news sources specializing in reporting of global digital activism, news sources from around the world that do not specialize in digital activism reporting, and the case list of the first version of the Global Digital Activism Data Set.

We used 2012 as a test year, reviewing a broad range of sources within these three areas and then identifying the most efficient sources, which we returned to to pull sources for the remaining years.  To quantify “efficiency” we created a ratio that compared new cases identified in that source to total articles reviewed.  Sources with an efficiency ratio of 5% or greater were retained and reused for 2011 back.  Sources with less than 5% efficiency were not used again and the campaigns identified through these sources were excluded to maintain a uniform sampling frame across years.  Using this method, 7,078 posts and articles were reviewed for 2012, leading to the identification of 435 campaigns which fit the inclusion criterion, a combined efficiency rate of 6.15%.

The sources analyzed were as follows.  We began by compiling a list of sources that cover digital activism campaigns around the world.  These sources are:

  1. Global Voices Online
  2. Actipedia
  3. Movements.org
  4. techPresident
  5. Mashable

In cases where the source had a scope beyond digital activism, the sites’ digital activism categories were identified.   This was the case with Global Voices Online  and Mashable.  On the former site we reviewed the “digital activism” and “protest” categories, on the latter, the “social good” section.

In addition to these specialized digital activism sources we also searched Lexis-Nexis using the following search strings for 2012: “digital protest” OR “online protest” OR “net protest” OR “internet protest” OR “web protest” OR “mobile protest” OR “cyberprotest.” (Similar search string around the terms “activism” and “campaign” were tested but not retained for lack of efficiency.)

The Lexis-Nexis search was for global news sources in English and included newspapers, blogs, newsletters, industry trade press, magazines and journals, web-based publications, news transcripts, and aggregate news sources.  Newswires and press releases were excluded because they are not created by 3rd parties.  The newspapers did not cover anglophone countries exclusively, though geographic and linguistic coverage derived primarily from Global Voices Online.   A statistic test will be undertaken after cases are collected to ensure that language is not a statistically significant determinant of case inclusion.

Finally, the case list from the 1st version of the Global Digital Activism Data Set was also reviewed according to the new inclusion criterion.  The efficiencies for 2012 were as follows:
  1. Global Digital Activism Data Set v. 1.0: 55.56%
  2. Global Voices 1 (digital activism category): 27.00%
  3. Global Voices 1 (protest category): 8.60%
  4. Actipedia: 7.77%
  5. Movements.org: 0.00%
  6. Lexis-Nexis Search String 2 (protest string): 7.57%
  7. techPresident: 1.18%
  8. Mashable: 3.02%

Sources 1, 2, 3, 4, and 7 were retained and used for the review of 2011 and previous years.  Sources 5, 6, 8, and 9 were not used again and the 2012 cases from those sources were excluded.

Assigned Sources

For each case, a coder is given a maximum of two sources:

  1. Source1 – Authoritative 3rd Party Source: The first source is the most complete and authoritative source on that campaign.  It is a 3rd-party source identified through the search patterns and subjected to the inclusion criteria described above.  Source1 is always provided to the coder and provides the basis for coding.
  2. Outcome Source:  Often Source1 describes the digital activism campaign in medias res.  That is, it is written during the campaign and does not include its outcome.  Because knowing the campaign outcome is an important part of analysis, if the outcome is not given in Source1, the source collector may do a Google search to locate a source which gives that outcome.  Outcome, as the dependent variable, is not part of the inclusion criteria.  A campaign is included in the data set even if no outcome is available.  In these cases a note “no additional outcome source” is provided to coders.  If the case outcome is located in Source1, this is also indicated to coders, though the nature of that outcome is not.  Coders do not need to read the entire outcome source.  They need read only as far as they need to to determine the campaign outcome, which is most often given at the beginning of the article. The text that the coders should read is indicated by the source collector to ensure that all coders read the sames text.  Because outcome is sometimes included in Source1 and is sometimes not available, an additional outcome source is not always provided to coders.

References

Tilly, C. (2004). Social Movements, 1768-2004. Boulder, CO: Paradigm Publishers.

Mary Joyce
Written by Mary Joyce