Your Source for Leaks Around the World!

Mining Big Data for Public Political Sentiment on “Technology and Democracy”

In Archive, Big Data, Science & Technology, Surveillance on September 8, 2013 at 12:31 AM




The United Nations estimates that more than 2.7 billion people will be online this year. What if there was a way to leverage the power of 2.7 billion people to tell their stories about how they feel about political topics? NDI recently received a grant from Crimson Hexagon to use their ForSight platform to delve deep into the public Internet to get answers to complex political questions.  The platform is based on mathematical algorithms developed by a team of academics at Harvard.  As we are rolling out Crimson Hexagon’s platform for specific programs we run, we wanted to test it on a topic we are keenly interested in.  So, we decided we wanted to know more about what the global discussions are relating to “Technology and Democracy.”

Forsight uses a powerful supervised machine learning algorithm developed by Harvard University’s Institute for Quantitative Social Science. It allows users to examine a huge volume of public information on the Internet such as tweets and Weibo posts, public Facebook posts, forums, Youtube, news, blogs, comments, and reviews — all in nearly real time. Forsight identifies content by keyword associations and users “train” the algorithm by categorizing a small set of the found content into specific categories. Once trained, the platform scans the internet and its historical database for content that is matching the user-defined criteria and categorizes it according to the user’s specific schema.  Because Forsight allows users to go back in time it is very useful for learning more about the context of events, or other political and social phenomena, prior to their occurrence.

technology-democracy 1

The graph above shows public opinion as expressed through the available sources that Crimson Hexagon scans on the internet from May 31 – August 23.  A few things are noteworthy: First, we noticed that in June a disproportionate amount of negativity associated with technology and democracy that continues throughout much of the month. As it turns out this time frame coincided with Edward Snowden’s revelations of the extent of NSA surveillance. Many of the articles, comments, tweets, and posts lamented the negative influence technology was having on democracy as a result of extensive state surveillance.

technology-democracy 2

The image above shows the topics of conversation across more than 1,000 pieces of content. There is definitely a section of topics concerned with technology’s negative effects on democracy, particularly in the United States, yet there were numerous other topics being talked about as well.

technology-democracy 3

We also wanted to understand who was talking about technology and democracy and where they were talking. The image below shows an intensity map of the conversation around the world. We are, of course, only looking for content in English, but it is clear that this is indeed a global conversation.

Related Link: Emotive: New Computer Program Reads Up to 10,000 Tweets a Second to Map Public Sentiment

  1. […] Mining Big Data for Public Political Sentiment on “Technology and Democracy” […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: