Recent years have seen an increasing buzz around how ‘Big Data’ can uncover patterns of human behaviour and help predict social trends. Most social activities today leave digital imprints that can be collected and stored in the form of large datasets of transactional data. Access to this data presents powerful and often unanticipated opportunities for researchers and policy makers to generate new, precise, and rapid insights into economic, social and political practices and processes, as well as to tackle longstanding problems that have hitherto been impossible to address, such as how political movements like the ‘Arab Spring’ and Occupy originate and spread.

While big data can allow the design of efficient and realistic policy and administrative change, it also brings ethical challenges (for example, when it is used for probabilistic policy-making), raising issues of justice, equity and privacy. It also presents clear methodological and technical challenges: big data generation and analysis requires expertise and skills which can be a particular challenge to governmental organizations, given their dubious record on the guardianship of large scale datasets, the management of large technology-based projects, and capacity to innovate. It is these opportunities and challenges that were addressed by the recent conference “Internet, Politics, Policy 2012: Big Data, Big Challenges?” organised by the Oxford Internet Institute (University of Oxford) on behalf of the OII-edited academic journal Policy and Internet. Over the two days of paper and poster presentations and discussion it explored the new research frontiers opened up by big data as well as its limitations, serving as a forum to encourage discussion across disciplinary boundaries on how to exploit this data to inform policy debates and advance social science research.

The conference was organised along three tracks: “Policy,” “Politics,” and Data+Methods (see the programme) with panels focusing on the impact of big data on (for example) political campaigning, collective action and political dissent, sentiment analysis, prediction of large-scale social movements, government, public policy, social networks, data visualisation, and privacy. Webcasts are now available of the keynote talks given by Nigel Shadbolt (University of Southampton and Open Data Institute) and Duncan Watts (Microsoft Research). A webcast is also available of the opening plenary panel, which set the scene for the conference, discussing the potential and challenges of big data for public policy-making, with participation from Helen Margetts (OII), Lance Bennett (University of Washington, Seattle), Theo Bertram (UK Policy Manager, Google), and Patrick McSharry (Mathematical Institute, University of Oxford), chaired by Victoria Nash (OII).

The evening receptions were held in the Ashmolean Museum (allowing us to project exciting data visualisations onto their shiny white walls), and the University’s Natural History Museum, which provided a rather more fossil-focused ambience. We are very pleased to note that the “Best Paper” winners were Thomas Chadefaux (ETH Zurich) for his paper: Early Warning Signals for War in the News, and Jonathan Bright (EUI) for his paper: The Dynamics of Parliamentary Discourse in the UK: 1936-2011. The Google-sponsored “Best Poster” prize winners were Shawn Walker (University of Washington) for his poster (with Joe Eckert, Jeff Hemsley, Robert Mason, and Karine Nahon): SoMe Tools for Social Media Research, and Giovanni Grasso (University of Oxford) for his poster (with Tim Furche, Georg Gottlob, and Christian Schallhart): OXPath: Everyone can Automate the Web!

Many of the conference papers are available on the conference website; the conference special issue on big data will be published in the journal Policy and Internet in 2013.