快猫短视频

Editorial 鈥�Predictive power won鈥檛 take away the big decisions鈥�

WHEN will the civil war in Syria subside? Will there be fighting on the Korean peninsula? The answers defy the best human minds, but there may be a tool to help them: the world鈥檚 largest database of geopolitical events has been released. And as it is refined, it could make forecasting events 鈥� and conflicts, in particular 鈥� as common as predicting the weather.

The Global Data on Events, Location and Tone data set (GDELT) contains nearly a quarter of a billion events going back to 1979 and hoovers up 100,000 new events every day. Its software scours media sources, such as the Associated Press, Agence France Presse and Xinhua, the main news agency in China. Collectively, the sources monitored cover every country in the world.

The software automatically extracts information from these news reports then uses natural-language processing to turn them into data points. For example, if a report contains the line 鈥淪udanese students and police fought in the Egyptian capital鈥� it codes the event as 鈥淪UDEDU fought COP鈥�. Next, the system finds the nearest mention of a city or locality in the text 鈥� in this case Cairo 鈥� and adds its latitude and longitude to the event data. The system can recognise different phrasings of who did what to whom and where. This helps it avoid duplicating events when they are mentioned in several news reports.

鈥淭he size and scope make this data set unique,鈥� says at the University of Illinois at Urbana-Champaign. 鈥淣obody had ever constructed a global event database over a long time frame.鈥� Leetaru and co-developer at Pennsylvania State University have plans to extend the data set back to 1800.

, who worked on an analysis of GDELT data at Penn State, calls it a 鈥渂reakthrough data set鈥�. As part of his doctoral thesis, Yonamine used a version of a machine learning algorithm more commonly used for financial projecting to forecast the Afghanistan conflict, which has raged since 2001.

Yonamine fed data on the conflict up until 2008 into the algorithm. It works by applying a statistical model to a series of data points over a period of time and extrapolating that pattern. He found that it accurately tracked the spread of violence across the country鈥檚 317 districts month by month between 2008 and 2012. His system made successful predictions for which districts would witness violent events in 47 out of the 48 months.

Better statistical models should improve the results further. Yonamine says that a predictive model updated daily could be used by Afghan businesses to choose the safest route to transport goods, for example.

Extracting information automatically is essential, says Leetaru. 鈥淎 protest is a very human thing,鈥� he says. 鈥淏ut if you want to look at a pattern, you have to quantify that.鈥� The system could also let you capture trends, such as the mood darkening in a region before the situation boils over. 鈥淭here is simply no way for a human to take in everything that happened in Egypt and make sense of it,鈥� he says, referring to the escalating protests that ultimately led to the ousting of President Hosni Mubarak in 2011.

鈥淭here is no way for a person to take in everything that happened in Egypt and make sense of it鈥�

鈥淚鈥檓 very optimistic about big data,鈥� says at the University of Konstanz in Germany. 鈥淏ut the strongest predictor of violence is previous violence. The real challenge is predicting new violence.鈥�

Weidmann thinks we will not have truly useful event forecasters until it is possible to mine data from social networks and other informal sources. Mainstream media tends to cover events only after they have happened. 鈥淏ig data needs to go deeper,鈥� he says.

That鈥檚 not likely to be easy. On-the-ground information is often sparse in unstable situations. However, in the case of the Syrian civil war, GDELT鈥檚 combination of geographical data and a diverse collection of news sources allowed 快猫短视频 to give a broad look at how the conflict has swept through the country since its inception in 2011.

As GDELT is refined and the time period it covers expands, its value is likely to go beyond questions of international policy. The financial world, for example, is increasingly relying on analyses of huge tranches of information.

That information can come from seemingly unlikely sources, such as those throwaway terms users put into search engines. Google recently opened up its records on what people are searching for and how it changes over time. For example, between 2004 and 2011 there was an increase in finance-related terms such as 鈥渄ebt鈥�, 鈥渄ow jones鈥�, and 鈥渦nemployment鈥�. at the University of Warwick, UK, and colleagues analysed this data and identified patterns that they believe could be used as early warning signs of a future financial crisis ().

鈥淓veryone is scrambling to extract value from big data,鈥� says Yonamine, who is now a data scientist with Allstate Insurance in Chicago. 鈥淲e have underestimated how much explanatory power there is in social dynamics.鈥�

Social networks are another obvious source, but applying predictive techniques to the rivers of posts and tweets that flow from them could be misleading 鈥� false information is commonplace. When a report of an explosion in the White House recently appeared on the hacked Twitter account of the Associated Press, for example, stock markets briefly plummeted, even though there was no corroboration.

This illustrates the need to refine how the information is processed. 鈥淭o some extent it doesn鈥檛 matter if a reported event happened or not,鈥� says Yonamine. 鈥淚nformation does not have to be real to drive future events.鈥�

People are now deliberately exploiting this side of big data, says Yonamine. Product reviews strategically planted on retail websites like Amazon often skew the results of data mining software, which collects reviews to help refine the recommender systems that predict what consumers might like, for example. Tracking down fake reviews is already a game of cat and mouse, he says. 鈥淚t鈥檚 machine learning algorithm versus machine learning algorithm.鈥�

鈥淭racking fake reviews is pitting machine learning algorithm against machine learning algorithm鈥�

This article appeared in print under the headline 鈥淭rouble on the horizon鈥�