March 11 Debunking Assignment – Taking on Big Shampoo

Debunking Assignment – Big Shampoo

I took this assignment to mean “use persuasive media techniques” more than to factually challenge something.

I shot this with my Iphone while getting my hair done because I like these guys.  And I’ve always wondered what’s the real difference between salon shampoo and the CVS stuff.

I could have read ingredient labels and compared them, scientifically,  for some insight into what these products do.  But I was pressed for time and these salon owners really charmed me, so they made a better story – if not better actual science.

Also – I usually find my hair salons by word of mouth – as I think many people do.

I also thought a video would capture attention – it’s  easy to click through, and done with a little humor would keep me watching.  It was also a way to use a new technique, which I’m trying to do with each assignment.

The logical power of the argument here isn’t very strong; the fun was in making it with these guys.  Go give some business to Salon Continentale in Belmont.  They’ll make you look good.

(Tell a story that makes truth claims about a disputed subject
Using techniques from the Debunking Handbook,
and using what you know about motivated reasoning,
leading with values, persuade a broad audience
– including those hostile to your claims – of the truth of your assertions)

Posted in All

Finding out what happened to UVa student


A UVa student was arrested and seemingly beaten by officials in a nightlife area near campus. I learned about it through social media Wednesday night, and Storified my discoveries.

<div><iframe src=”//” width=”100%” height=”750″ frameborder=”no” allowtransparency=”true”></iframe><script src=”//”></script><noscript>[<a href=”//” target=”_blank”>View the story ” MIT Social Story – Martese Johnson Arrest UVA” on Storify</a>]</noscript></div>

Patrick – Patty – Paddy

For our assignment, Bianca Datta and I created a simple web page that demonstrates a possible tool for journalists in understanding a lot of social media data. We took the idea of creating a story via social media and attempted to automate it a bit more.

On the left is a feed of tweets based on the topic ‘St. Patrick’s Day’. The search terms we used were ‘StPatty’, ‘StPaddy’, and ‘StPatrick’ . Once we had these tweets in our database, we processed them to get topics using LDA via a Node.js package. The top ranking topics were then displaying on the right side of the webpage using a very simple web graphic. Based on the term’s use, the topic would be allotted a larger height for more use and smaller height for less use. The left column shows common topic terms in tweets with the word ‘Patty’. The middle column shows topic terms for ‘Paddy’ and the right column shows topic terms for ‘Patrick’.

This is a screenshot of the page:

Screen Shot 2015-03-18 at 4.34.12 AM


We also gathered content from Instagram, Vine, and Flickr and processed each of those to get topic rankings.

link :


Posted in All

The money-time of the Israeli election day as seen by Twitter

(warning, the data did not play well with me)

Election days in Israel climax at 10pm. In this precise time the three major broadcasting networks announce their polling results. These results tend to be quite accurate and give a good indication on the final results. The moments right before and after 10pm are extremely emotional for the people of Israel.

With these elections, held on March 17th, I decided to take a closer look at these crucial moments and how they are manifested in Twitter. I used the Twitter search API to pull tweets that were created between 9:55pm – 10:05pm and contained the word “Israel”.  I also used a service called Alchemy API to perform automatic sentiment analysis on these tweets.

In the following graph, each dot represents a tweet. the horizontal axis is time relative to 10pm and the vertical axis is sentiment between 1 to -1 where 1 is positive with high probability and -1 is negative with high probability.

Screen Shot 2015-03-18 at 2.56.33 AM

It’s pretty to hard to see anything from this chart besides the fact that it seems to be slightly more dense on the right side, suggesting that more tweets were tweeted after 10pm than before.

In an attempt to cleanup the results, I batched tweets in 30 seconds intervals and calculated the average for them. In this chart each dot marks a batch of tweets and the vertical axis is the average sentiment of the entire batch.

Screen Shot 2015-03-18 at 3.05.50 AM

Unfortunately, It’s still hard to identify a trend besides the fact that the average sentiment is negative – that’s usually a property of news stories.

As a last resort, I decided to only take a look at the quantities of tweets. These tweets don’t necessarily represent the entire twitter firehose of data but looking at the first chart, they might still provide an insight.

Screen Shot 2015-03-18 at 3.05.59 AM

Still no clear trend. However, the peak at 10:00pm which represent tweets between 10:00pm – 10:30pm does make a lot of sense – It’s right after the results.

That’s data science. Sometimes you get inspiring results and sometime you’re just wasting CPU cycles.


Posted in All

Tsarnaev trial through tweets

This is my attempt to curate the tweets of the many journalists live-tweeting the Tsarnaev trial. After taking quite a while to collect all the data, I tried some pretty complicated things to try to group them about event chronologically. Ultimately the grouping didn’t ever work that well, and it still wasn’t producing a good summary. After trying a lot of different things, I found it was best to simply highlight tweets that had many more retweets than usual for that user, so I stuck with that approach to automatically choose which of the >10k of Tsarnaev trial-related tweets to highlight from these users.

I have had been having a bit of trouble debugging the second part of the timeline feature, but for now at least here is the semi-automatic livetweet summary of the Tsarnaev trial:

Chris Borland & Concussions in the NFL

On Wednesday, March 18th Chris Borland of the San Francisco 49ers retired from professional football, citing fear of future head injuries. This set off a flurry of discussion both on ESPN and Reddit (where it garnered 3000 upvotes, making it one of the most popular threads of the day).

I wanted to investigate how public discussion on the injury and concussions differed from discussion on the Media. To do this, I used a tool I’ve been working on for research inspired by Media Clouds. The tool compares in real time word clouds from Twitter vs AP news sources and tries to highlight differences. The tool is still in BETA so I was unable to give an interactive demo, but here is a screenshot:

Screenshot 2015-03-18 01.16.51

On the left hand side is traditional news outlets, on the right is sources from twitter.

News sources seem to be covering Borland’s impressive career, including his college football record at Wisconsin. However, the discussion on Twitter is two-fold, one is expressing shock at the retirement and the second comparing him to Patrick Willis, another linebacker for the 49ers who retired only days earlier due to religious reasons. This aspect of the story is not covered very much in news outlets, but with Willis and Borland gone, 49ers fans have felt like their season has been derailed. Public discussion seems to focus more on the consequences to the 49ers season whereas news outlets are discussing the repercussions to the NFL as a whole.

Posted in All

A Curious Case In Tzu Chi Foundation’s Recent Scandal


On March 16, 2015, Tzu Chi Foundation, one of Taiwan’s foremost humanitarian NGO, announced in a press conference to rescind its appeal on development project in one of Taipei’s nature reserve. Since the controversy saw its heated public debate since late-February, Tzu Chi has faced widespread public criticism, and more recently, accusations of financial activities that may violate regulations.

Interestingly, the issue has a long history dating back to 2005.  In a decade of legal controversy, there was never as much public attention gathered previously. The interplay between the new city administration and social media clearly helped the event to develop.

In 1997, Tzu Chi purchased a land in Taipei’s Neihu reserve and planned to change the property name to social benefit property to accommodate recycling factory. However, due to the fear the construction may affect water drainage that create potential hazard, Tzu Chi faced strong opposition from environmental groups. Tzu Chi officially made appeal to Taipei City government in 2005 for environment safety review, and had been under review since.

As soon as Taipei City Mayor Ko was elected in December 2014, he openly expressed that he was against Tzu Chi’s development project appeals. In an interview on February 23, 2015, Mayor Ko criticized Tzu Chi was very “weird”. This immediately triggered a fierce online response.


Nun Shu’s initial response to Mayor Ko’s interview

Nun Tzu-Hui Shu, the spokesperson for Tzu Chi, wrote a post on Facebook denouncing Mayor Ko “returned evil for good” and his actions will ultimately suffer “divine consequences”. This remark went viral in Taiwanese social media. On, a bbs site that functions very similarly to Reddit, the discussions soared to an unprecedented level. Because of Tzu Chi’s religious background (it is founded and led by Buddhist nun), many more posts emerged accusing Tzu Chi acted very cult-like. As one user accused,

Screen Shot 2015-03-17 at 12.08.42 PM

The founder recruited doctors not based on merit, but by how much they enact “appreciation” to the founder… once when the founder visited a hospital in Hsin-Chu, when she exited the car, the doctors would kneel as a sign of invitation…

 With the growing discussion on social media, many public figures start to cast other doubts on Tzu Chi’s operation. University of Taipei’s urban planning professor Chung-Hsin Yang posted study on Tzu Chi’s property transaction record showing Tzu Chi would purchase industrial properties and rename for residential uses.

Screen Shot 2015-03-17 at 12.33.01 PM

Scholars openly criticized Tzu Chi for violating zoning regulations

Amid the public debate, with Nun Shu trying to convince the public that Tzu Chi is abiding to regulations, rumors on social media became viral that show doubts on Tzu Chi’s organizational structure. One notable example was how Tzu Chi deploys a caste system where only people who have donated a certain amount of money can promote up the rank ladder. Some users started sharing a crystal sculpture of a Buddha sold by Tzu Chi costs more than $11,000 USD with suspected production cost of less than $1000 USD, which spurred an outrage in the community, prompting many users posting photoshoped mockups of the sculpture.

Screen Shot 2015-03-17 at 12.47.05 PM1266

Social Media response to Tzu Chi’s crystal sculpture

With a sustained public attention on Tzu Chi’s controversy, people started to provide more cases indicating a potential scandal may be taking place. On Match 10, one user claiming to be an “insider” posted that another Tzu Chi’s development project in Taichung taking place at a historic site was undergoing construction without proper examination of potential damage to historic remains. Other users cast doubts on Tzu Chi’s money flow.

On March 16, Tzu Chi announced to rescind on the controversial appeal on Taipei’s development project, hoping to end the controversy. However, the hosts at one of the premiere politics talk shows quitted in response to Tzu Chi’s press conference, claiming that “Tzu Chi has controlled the media to show only the good sides”. The public onslaught ensued, with more users on producing accusations, claiming Tzu Chi’s portfolio includes foreign stocks, along with notorious companies such as Monsanto. Tzu Chi’s worry is now only a beginning.

Social Media Feedback Loop

In comparison to the past, the Tzu Chi controversy has gathered a lot more attention. It is interesting to see how there is a symbiotic growth between the discussions on and public figures’ Facebook posts. We can see there is a strong autocorrelation between number of posts and attentions on, and Facebook attentions. In the past, either of the two sources were as heavily used so people only chatted controversial topics locally. The figure demonstrates how social media acts as information amplifier.

Screen Shot 2015-03-17 at 11.54.18 PM

A typical interface

Data Visualization:


Posted in All

Social Reality Check: is our social network an ideology bubble?

Disclaimer: As a designer, I prefer to take on a personal point of view instead of trying to turn myself into a good reporter. There’s also the fact that I am not an extraordinary English speaker (and too perfectionist to think this is ok). So bear with me. 😉

Last week, Miguel and I were talking about the conservative wave that has been sweeping the social networks in Brazil. Our subject, at the time, was the recent implementation of a network of cycle paths in the capital of São Paulo, the largest and most congested city in the country.

São Paulo has way too many cars. For a population of around 12 million, the state capital registered 5.4 million cars – roughly one for every two people, according to the state’s Department of Motor Vehicles (Detran). The metro network in the city is burdened, overcrowded, and does not reach the entire urban area. Buses are also crowded and subject to traffic conditions.

In June 2013 people marched in the city, protesting against the increase in bus and subway fares. When cases of police violence against demonstrators became public, the popular movement expanded and millions of people took to the streets of the whole country.

After these events, the mayor of São Paulo Fernando Haddad attempted to reduce traffic chaos through unpopular actions such as creating exclusive bus lanes, improving fleets and remodeling public roads, culminating with the implementation of a network of cycle paths in 2014. At this time, a war between the drivers culture and the bike culture drew up very clearly on social networks, and soon transformed into a textual war between individualists and collectivists. The mayor received a barrage of criticism from angry drivers who complained about the reduced space in the streets for their cars. Most pedestrians and cyclists, however, approved, and Haddad’s popularity grew among young people.

The mayor of São Paulo, Fernando Haddad, eventually uses his bike to commute

The mayor of São Paulo, Fernando Haddad, eventually uses his bike to commute

My own timeline of Facebook, however, seemed almost immune to that criticism against the cycle paths. Messages with very conservative content rarely appear in my Facebook feed, and I was only aware of them because of comments made by outraged friends or articles in magazines and newspapers.

From this fact came the idea to address the ideological bubbles on social networks. Do our private feeds on social networks correspond to general opinion in the real world?

With the recent protests in Brazil against the rule of President Dilma Rousseff, we thought that the war we spoke of would certainly intensify. It would be interesting to compare the Facebook public news feed to the private feeds of specific users, trying to create a visualization of the results.

Using Facebook API, I scanned the public stream for posts in Portuguese containing the name of the president, and stored dozens of files like this one, so that I could analyse the results. Unfortunately, however, I could not code the search in private news feeds in time to follow the course of events in social networks, so there was not enough data for a good comparison.

But here is a word cloud generated from public posts collected during the protests:
Word Cloud - Public stream (Dilma Rousseff)
And here, a cloud generated from newsfeeds of five people who considered themselves contrary to the protests of March 15.
Word Cloud - Private stream (Dilma Rousseff)
An interesting thing to notice is the frequency of the words ‘direita’ (rightist) and ‘ditadura’ (dictatorship) in the private cloud. Among people contrary to the recent protests, the general opinion is that the participants of such demonstrations are people from the white middle class, mostly individualistic, and supporters of the dictatorship. It may be also significant that in the public timeline the word ‘protest’ is more frequent than ‘demonstration’, while in private timelines this relationship is reversed.
Unfortunately, we could not get to the in-depth comparative analysis we intended this week, but with more time I think I would try not the word cloud format, but the word network format, as in Textexture tool website.
Posted in All

The Northern Lights of Twitter

It is March 17, 2015, and a solar storm is brewing. The strongest storm since August 2005. This means one thing: for much of the Northern Hemisphere, a good chance of seeing the Northern Lights, or Aurora Borealis.

If you want updates on where in the world you can see the Northern Lights, you can check NOAA’s real-time aurora forecast, but the map is not very intuitive and is hard to translate into “should I go outside and look right now?” So instead, I checked Twitter.

Click here for The Northern Lights of Twitter Map with Timestamps.

Click here for The Northern Lights of Twitter Storify.

According to Twitter users, the Northern Lights are visible tonight in Ontario, Alberta, Nebraska, Wisconsin, Minnesota, Maine, and Norway. (I performed the survey by monitoring the search term “northern lights” on Twitter, since it was more popular than the hashtag #NorthernLights, and cherry-picking geographically disparate locations to record. By the nature of the search I’m sure I missed many locations. Just while composing this post I have seen additional posts in Alaska, Sweden, Michigan, and Ireland.)