Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Topics of Controversy: An Empirical Analysis of Web Censorship Lists

Topics of Controversy: An Empirical Analysis of Web Censorship Lists Abstract Studies of Internet censorship rely on an experimental technique called probing. From a client within each country under investigation, the experimenter attempts to access network resources that are suspected to be censored, and records what happens. The set of resources to be probed is a crucial, but often neglected, element of the experimental design. We analyze the content and longevity of 758,191 webpages drawn from 22 different probe lists, of which 15 are alleged to be actual blacklists of censored webpages in particular countries, three were compiled using a priori criteria for selecting pages with an elevated chance of being censored, and four are controls. We find that the lists have very little overlap in terms of specific pages. Mechanically assigning a topic to each page, however, reveals common themes, and suggests that handcurated probe lists may be neglecting certain frequently censored topics. We also find that pages on controversial topics tend to have much shorter lifetimes than pages on uncontroversial topics. Hence, probe lists need to be continuously updated to be useful. To carry out this analysis, we have developed automated infrastructure for collecting snapshots of webpages, weeding out irrelevant material (e.g. site “boilerplate” and parked domains), translating text, assigning topics, and detecting topic changes. The system scales to hundreds of thousands of pages collected. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Proceedings on Privacy Enhancing Technologies de Gruyter

Topics of Controversy: An Empirical Analysis of Web Censorship Lists

Download PDF
20 pages

Loading next page...
 
/lp/de-gruyter/topics-of-controversy-an-empirical-analysis-of-web-censorship-lists-dwq26AhlHP
Publisher
de Gruyter
Copyright
Copyright © 2017 by the
ISSN
2299-0984
eISSN
2299-0984
DOI
10.1515/popets-2017-0004
Publisher site
See Article on Publisher Site

Abstract

Abstract Studies of Internet censorship rely on an experimental technique called probing. From a client within each country under investigation, the experimenter attempts to access network resources that are suspected to be censored, and records what happens. The set of resources to be probed is a crucial, but often neglected, element of the experimental design. We analyze the content and longevity of 758,191 webpages drawn from 22 different probe lists, of which 15 are alleged to be actual blacklists of censored webpages in particular countries, three were compiled using a priori criteria for selecting pages with an elevated chance of being censored, and four are controls. We find that the lists have very little overlap in terms of specific pages. Mechanically assigning a topic to each page, however, reveals common themes, and suggests that handcurated probe lists may be neglecting certain frequently censored topics. We also find that pages on controversial topics tend to have much shorter lifetimes than pages on uncontroversial topics. Hence, probe lists need to be continuously updated to be useful. To carry out this analysis, we have developed automated infrastructure for collecting snapshots of webpages, weeding out irrelevant material (e.g. site “boilerplate” and parked domains), translating text, assigning topics, and detecting topic changes. The system scales to hundreds of thousands of pages collected.

Journal

Proceedings on Privacy Enhancing Technologies – de Gruyter

Published: Jan 1, 2017

References

  • Jonathan eds Access Contested Identity Resistance in Asian Cyberspace Access MIT
    Ronald Deibert
  • Blei Latent Allocation
    David
  • Caroline Does Pornography - Blocking Software Block Access to Health Information on the Internet ? The of the
    Richardson
  • Behind the Great Firewall of China Queue
    Daniel Anderson
  • Internet Censorship in Iran First Free and Open Communications on the Internet
    Simurgh Aryan
  • Brief of amici curiae Automattic Google Twitter Universal Music th
    Joseph
  • Chris Uwe and Torsten Scalable Construction of High - Quality Web Corpora for Technology
    Biemann
  • DOM Based Content Extraction via Density Research and Development in
    Sun
  • Readability Web service
  • Stefan lightweight and efficient tool for cleaning Web pages on Language Resources and Evaluation Resources
    Evert
  • Censorship is Futile First
    Zubair Nabi
  • Internet Censorship in China : Where Does the Filtering Occur ? Passive and Active
    Xueyang Xu
  • Chris The Long of Domain Names
    Janos Szurdi
  • The weirdest people in the world and
    Joseph Henrich
  • Empirical Study of a National - Scale Distributed Intrusion Detection System : Backbone - Level Filtering of HTML Responses in China
    Jong Chun Park
  • Jonathan eds Access Denied : The Practice and Policy of Global Internet Filtering Access MIT
    Ronald Deibert
  • Tracker for Internet Censorship Communications
    Jedidiah
  • Web Censorship Monitor Free and Open Communications on the Internet
    Andreas Sfakianakis
  • Ignoring the Great Firewall of China
    Richard Clayton
  • Empirical Study of a National - Scale Distributed Intrusion Detection System : Backbone - Level Filtering of HTML Responses in China
    Jong Chun Park
  • Open of Network Interference Free and Open Communications on the Internet
    Arturo Filastò
  • and The Most Controversial Topics in Wikipedia multilingual and geographical analysis Global Wikipedia International and cross - cultural issues in online collaboration Ed by
    Taha Yasseri
  • Internet Censorship in Iran First Free and Open Communications on the Internet
    Simurgh Aryan
  • Method for Identifying and Confirming the Use of URL Filtering Products for Censorship Internet Measurement Conference
    Jakub Dalek
  • Inferring Mechanics of Web Censorship Around the World Free and Open Communications on the Internet
    John
  • Stefan Geoffrey Dagger Dynamics of Web Search Cloaking Communications
    David
  • Ignoring the Great Firewall of China
    Richard Clayton
  • Combining a Segmentation - Like Approach and a Density - Based Approach in Content Extraction Technology
    Shuang Lin
  • Censorship is Futile First
    Zubair Nabi
  • The Initiative United States Overview of Censorship Web report
    OpenNet
  • Sam Encore Lightweight Measurement of Web Censorship with Cross - Origin Requests
    Burnett
  • Automating Change Detection and Notification of Web Pages and Systems
    Sharma Chakravarthy
  • The Anatomy of Web Censorship in Pakistan Free and Open Communications on the Internet
    Zubair Nabi
  • Blei Multilingual Topic Models for Unaligned Uncertainty in Artificial Ed by Jeff
    Jordan
  • Stefan Geoffrey Dagger Dynamics of Web Search Cloaking Communications
    David
  • Look at the Consequences of Internet Censorship Through an ISP Lens Internet Measurement Conference
    Sheharbano Khattak
  • Nonparametric Estimation from Incomplete Observations of the
    Edward
  • Three Researchers Five Conjectures : An Empirical Analysis of TOM - Skype Censorship and Surveillance Free and Open Communications on the Internet
    Jeffrey Knockel
  • Censorship in the Analyzing Internet Filtering in Internet Measurement Conference
    Abdelberi Chaabane
  • Efficient methods for topic model inference on streaming document collections Knowledge Discovery and Data Mining Ed by IV Françoise
    Limin Yao
  • Vietnamese Software
    Luu Tuan
  • The Common Crawl Foundation Common Crawl Web site
  • The Growth of Global Internet Censorship and Circumvention Survey Communications of the
    Ramesh Subramanian
  • Combining a Segmentation - Like Approach and a Density - Based Approach in Content Extraction Technology
    Shuang Lin
  • Stefan lightweight and efficient tool for cleaning Web pages on Language Resources and Evaluation Resources
    Evert
  • Web Censorship Monitor Free and Open Communications on the Internet
    Andreas Sfakianakis
  • Leo Random Forests
    Breiman
  • Jonathan eds Access Denied : The Practice and Policy of Global Internet Filtering Access MIT
    Ronald Deibert
  • Google Translate API Web service
  • Software application
    Ariya Hidayat
  • Understanding inverse document frequency on theoretical arguments for of
    Stephen Robertson
  • The Velocity of Censorship : High - Fidelity Detection of Microblog Post Deletions
    Tao Zhu
  • Jonathan eds Access Controlled The Shaping of Rights Rule in Cyberspace Access MIT
    Ronald Deibert
  • Jonathan eds Access Controlled The Shaping of Rights Rule in Cyberspace Access MIT
    Ronald Deibert
  • Automated Detection and Fingerprinting of Censorship Block Pages Internet Measurement Conference
    Ben Jones
  • Vietnamese Software
    Luu Tuan
  • Applying Conditional Random Fields to Japanese Morphological Empirical Methods in Natural Language
    Taku Kudo
  • Marcel Stefan Glasnost Enabling End Users to Detect Traffic Networked Systems Design and Implementation
    Dischinger
  • on the On Measuring Censors Resources and Motivations Free and Open Communications on the Internet
    Nicholas Aase
  • Software
    Theppitak Karoonboonyanan
  • Censorship in the Analyzing Internet Filtering in Internet Measurement Conference
    Abdelberi Chaabane
  • Blei Multilingual Topic Models for Unaligned Uncertainty in Artificial Ed by Jeff
    Jordan
  • Inferring Mechanics of Web Censorship Around the World Free and Open Communications on the Internet
    John
  • Look at the Consequences of Internet Censorship Through an ISP Lens Internet Measurement Conference
    Sheharbano Khattak
  • Three Researchers Five Conjectures : An Empirical Analysis of TOM - Skype Censorship and Surveillance Free and Open Communications on the Internet
    Jeffrey Knockel
  • Internet Archive Wayback Machine Web service
  • for Toolkit Software
    Andrew
  • Around the Web in Six Documenting a Large - Scale Crawl
    Sarker Tanzir Ahmed
  • Readability Web service
  • Method for Identifying and Confirming the Use of URL Filtering Products for Censorship Internet Measurement Conference
    Jakub Dalek
  • Compact Detection Software
    Dick
  • The Initiative Country Profiles Web site
    OpenNet
  • Internet Archive Wayback Machine Web service
  • DOM Based Content Extraction via Density Research and Development in
    Sun
  • on the On Measuring Censors Resources and Motivations Free and Open Communications on the Internet
    Nicholas Aase
  • for Toolkit Software
    Andrew
  • Parking Analyzing Detecting Parked Network Distributed System Internet
    Thomas Vissers
  • The weirdest people in the world and
    Joseph Henrich
  • Topic Models Empirical in Processing
    David
  • Christin Automatically Detecting Vulnerable Websites Before They Turn Malicious
    Kyle Soska
  • The Initiative Country profiles Thailand Web report
    OpenNet
  • An Algorithm for Suffix Stripping Readings in Information by San
    Porter
  • Open of Network Interference Free and Open Communications on the Internet
    Arturo Filastò
  • Automating Change Detection and Notification of Web Pages and Systems
    Sharma Chakravarthy
  • Big Brother on a budget : How Internet surveillance got so cheap Ars
    Sean Gallagher
  • An Algorithm for Suffix Stripping Readings in Information by San
    Porter
  • Topic Models Empirical in Processing
    David
  • Internet Censorship in China : Where Does the Filtering Occur ? Passive and Active
    Xueyang Xu
  • Pi Galley Optimizing ChineseWord Segmentation for Machine Translation Performance Statistical Machine Translation Association for
    Chuan Chang
  • and The Most Controversial Topics in Wikipedia multilingual and geographical analysis Global Wikipedia International and cross - cultural issues in online collaboration Ed by
    Taha Yasseri
  • The Initiative URL testing lists Git repository
    OpenNet
  • Parking Analyzing Detecting Parked Network Distributed System Internet
    Thomas Vissers
  • Brief of amici curiae Automattic Google Twitter Universal Music th
    Joseph
  • Chris The Long of Domain Names
    Janos Szurdi
  • Regional Variation in Chinese Internet Filtering
    Joss Wright
  • Evaluation methods for topic models International Conference on Machine Learning Ed by Andrea
    Hanna
  • Google Translate API Web service
  • Regional Variation in Chinese Internet Filtering
    Joss Wright
  • Blei Latent Allocation
    David
  • Software
    Theppitak Karoonboonyanan
  • Big Brother on a budget : How Internet surveillance got so cheap Ars
    Sean Gallagher
  • Characterizing Web Censorship Worldwide : Another Look at the OpenNet Initiative Data on the
    Phillipa Gill
  • Segmentation of Informal Arabic with Domain Adaptation Short
    Will Monroe
  • Christin Automatically Detecting Vulnerable Websites Before They Turn Malicious
    Kyle Soska
  • Understanding inverse document frequency on theoretical arguments for of
    Stephen Robertson
  • Efficient methods for topic model inference on streaming document collections Knowledge Discovery and Data Mining Ed by IV Françoise
    Limin Yao
  • Characterizing Web Censorship Worldwide : Another Look at the OpenNet Initiative Data on the
    Phillipa Gill
  • Bar Template Detection via Data Mining and Its Applications
    Yossef
  • The Initiative URL testing lists Git repository
    OpenNet
  • The Anatomy of Web Censorship in Pakistan Free and Open Communications on the Internet
    Zubair Nabi
  • Philipp Global Network Interference Detection over the RIPE Atlas Network Free and Open Communications on the Internet
    Collin Anderson
  • Around the Web in Six Documenting a Large - Scale Crawl
    Sarker Tanzir Ahmed
  • Jonathan eds Access Contested Identity Resistance in Asian Cyberspace Access MIT
    Ronald Deibert
  • The Growth of Global Internet Censorship and Circumvention Survey Communications of the
    Ramesh Subramanian
  • The Initiative United States Overview of Censorship Web report
    OpenNet
  • Sam Encore Lightweight Measurement of Web Censorship with Cross - Origin Requests
    Burnett
  • Applying Conditional Random Fields to Japanese Morphological Empirical Methods in Natural Language
    Taku Kudo
  • The Initiative Country Profiles Web site
    OpenNet
  • Behind the Great Firewall of China Queue
    Daniel Anderson
  • Bar Template Detection via Data Mining and Its Applications
    Yossef
  • Diversifying Restricted Boltzmann Machine for Document Modeling Knowledge Discovery and Data Mining
    Pengtao Xie
  • Caroline Does Pornography - Blocking Software Block Access to Health Information on the Internet ? The of the
    Richardson
  • Tracker for Internet Censorship Communications
    Jedidiah
  • The Common Crawl Foundation Common Crawl Web site
  • Pi Galley Optimizing ChineseWord Segmentation for Machine Translation Performance Statistical Machine Translation Association for
    Chuan Chang
  • Diversifying Restricted Boltzmann Machine for Document Modeling Knowledge Discovery and Data Mining
    Pengtao Xie
  • The Velocity of Censorship : High - Fidelity Detection of Microblog Post Deletions
    Tao Zhu
  • The Initiative Country profiles Thailand Web report
    OpenNet
  • Leo Random Forests
    Breiman
  • Automated Detection and Fingerprinting of Censorship Block Pages Internet Measurement Conference
    Ben Jones
  • de Ian Censorship Mapping Sources Legality Free and Open Communications on the Internet
    JossWright
  • Evaluation methods for topic models International Conference on Machine Learning Ed by Andrea
    Hanna
  • Philipp Global Network Interference Detection over the RIPE Atlas Network Free and Open Communications on the Internet
    Collin Anderson
  • Marcel Stefan Glasnost Enabling End Users to Detect Traffic Networked Systems Design and Implementation
    Dischinger
  • Software application
    Ariya Hidayat
  • Segmentation of Informal Arabic with Domain Adaptation Short
    Will Monroe
  • Compact Detection Software
    Dick
  • de Ian Censorship Mapping Sources Legality Free and Open Communications on the Internet
    JossWright
  • Chris Uwe and Torsten Scalable Construction of High - Quality Web Corpora for Technology
    Biemann
  • Nonparametric Estimation from Incomplete Observations of the
    Edward