Have you ever heard of Karl Ernst Krafft?

He was a Swiss astrologer in the early 20th century. Krafft correctly predicted that Hitler would be in danger between November 7 and November 10, 1939, and wrote to a friend working for Himmler, warning him of an attempt on Hitler’s life. After a bomb exploded in the Munich beer hall November 8, barely missing Hitler, Krafft became a favorite with the Nazis and was made their court astrologer. However, when towards the end of the war, he (correctly) predicted that bombs would soon destroy the Propaganda ministry in Berlin, he was viewed as a traitor. He was put into jail, and died of typhus on the journey to the Buchenwald concentration camp.

Reading the final blog post of famed election forecaster Nate Silver on his fivethirtyeightblog, he says 10.41AM on election day morning of Nov 8, 2016 “…..Clinton is a 71 percent favorite to win the election according to our polls-only model and a 72 percent favorite according to our polls-plus model…..”
Next morning Hillary Clinton conceded defeat to right-wing populist Donald Trump.

Donald Trump is only the last in a long list of populist leaders swaying public opinion, where “ordinary” people are afraid to admit their preferences, leading conventional pollsters astray. I still remember a visit together with my colleague Manfred Vogel to the offices of highly respected Swiss pollster Claude Longchamp. At that time Longchamp was reeling from a similarly spectacular mis-prediction, where he had forecast a clear rejection of a Swiss National referendum to forbid construction of minaret towers for mosques in Switzerland (the “Minaret Initiative”), only to see the Swiss voters clearly approving this restrictive clause. In my own social media analysis on Twitter, Facebook, and blogs, using an early version of the six honest signals of communication I had identified a surprisingly strong showing for the Minaret Initiative well before the election.

The explanation for the surprising success of the “Minaret Initiative”, Brexit, and Donald Trump, is that “ordinary” people are afraid to express their true beliefs when asked in phone polls. In a silent revolution of the disenfranchised, they lie to the pollsters, to only voice what they really believe on election day.  The social network created using our galaxyscope tool through analyzing Wikipedia links, blog links, and re-tweet follower networks illustrates this point. Donald Trump - for all his billionaire bluster - is a clear outsider, and underdog of the establishment. Many people therefore will be reluctant to tell their preference for Trump to the pollsters.
 

The "ordinary" people will however tell what they really think on social media to their friends, so interpreting their “honest signals” would be a better way than asking on the phone. This analysis is extremely hard and time consuming, because social media usage and popularity of particular tools change year by year. It used to be that Facebook posts could be easily read by anybody – not anymore. It used to be that Chileans used to express themselves on Twitter – not anymore. In the age of Snapchat, Whatsapp, WeChat, Instagram, and legions of other point-to-point communication tools it gets near to impossible to collect this datastream (well, maybe there might be a hidden backdoor if you are NSA, but not for ordinary researchers). Therefore, the art and science of prediction is now to interpret publicly available sources such as Google and Wikipedia search logs, blog posts, Tweets, Wikipedia pages, and online news articles, and disentangle their honest signals from the straightforward network picture shown above.

However, as Karl Ernst Krafft illustrates, predicting the outcome of politics can be extremely costly for the one doing the prediction. Frequently the messenger will be shot. So no political prediction this time!

Thanks to Alain Egli from GDI for pointing me to the nice picture illustrating embeddedness of Hillary Clinton compared to Donald Trump