Chair: Prof. Leigh C. Anderson 
Date/Venue: June 10, University College London

Submission of additional material by course conveyors: 13th May.

Registration deadline for workshops/tutorials: 31st May (subject to availability).

Morning session times: 9.15 – 10.45 am [break 20 mins] resume 11.05 – 12.35 pm

Lunch 12.35 pm – 13.30 pm

Afternoon session times: 13.30 – 15.00 pm [break 20 mins] resume 15.20 – 16.50 pm

Morning Sessions (9:15 – 12:35):

  • “Understanding the Data & Curation choices behind the Indicator: SDGs & LSMS-ISA Measures of Progress  – Instructor(s): Ayala Wineman and Leigh Anderson; University of Washington
  • “Digital Ethics & Algorithm Assessment” – Instructor(s): Zeynep Engin and Adriano S. Koshiyama; University College London
  • [This session has been moved to the afternoon] “Data Sharing & Data Trusts” – Instructor(s): Gefion Thuermer, Johanna Walker; University of Southampton, Peter Wells; Open Data Institute, and Kieron O’Hara; University of Southampton
  • “Data Stewardship in Action: Workshop on Making Data Collaboratives Systematic, Sustainable & Responsible” – Instructor(s): Stefaan Verhulst, The GovLab, New York University

Afternoon Sessions (13:30 – 16:50):

  • “Introduction to Artificial Intelligence in Government” – Instructor(s): Jasmine Grimsley and Isabela Breton, Data Science Campus, Office for National Statistics, and, Barbara Webber – Cabinet Office
  • “Big-Data for Policy Making & Digital Transformation” – Instructor(s): Francesco Mureddu, Lisbon Council, Italy
  • “Collaborating with Universities: Marriage made in heaven?” – Instructor(s): Olga Sergushova , Vania Sena and Gina Yannitell Reinhardt; University of Essex
  • “Data Sharing & Data Trusts” – Instructor(s): Gefion Thuermer, Johanna Walker; University of Southampton, Peter Wells; Open Data Institute, and Kieron O’Hara; University of Southampton [please note, this session was previously advertised as a morning session]

Details of the Workshops and Tutorials:

  • Understanding the data & curation choices behind the Indicator: SDGs & LSMS-ISA measures of progress (Morning) 

Ayala Wineman – University of Washington

Seldom visible are the steps necessary to prepare raw data for its many uses: analysis, merging with other data, training machines, or tracking progress in dashboards. These “data curation” decisions include cleaning, construction, and display choices.   In this workshop we illustrate how data curation choices can dramatically alter final numbers and interpretations, and hence affect investment and policy decisions. We discuss these issues in the context of several Sustainable Development Goals, and using examples from the Living Standards Measurement Study, Integrated Surveys on Agriculture in Ethiopia, Tanzania and Nigeria. 

Aims and Objectives

For senior decision-makers our goal is for attendees to become more critical consumers of indicators.  We first illustrate how large differences in average estimates can arise from how data outliers are managed (e.g., winsorizing vs. median absolute deviation). We then work through how simple proxy choices and definitions of these measures themselves can vary in ways that matter to policy. Rural, for example, may be defined by political or administrative units (e.g. Prime Minister’s Office-Regional Administration and Local Government); by human settlements (e.g. adopted by the Ministry of Lands and Human Settlements Development), statistically (e.g. adopted by the National Bureau of Statistics), or simply by population density (Muzzini, 2008). We conclude by examining the potential consequences of two data trends: the increasingly common tendency to use these indicators in dashboards that compare, for example, outcomes across regions; and recent research efforts to use meta-analyses to synthesize findings across multiple studies. 

Intended Audience

Policy-makers, practitioners, foundation, and other decision-makers who use SDG and other constructed indicators and compiled data to track progress, invest, or otherwise as evidence and inputs into decision-making. The workshop is also appropriate for producers of these indicators, to consider their cleaning and display choices. For those with training in Stata, .do files are available.  

  • Digital Ethics & Algorithm Assessment (Morning) 

Zeynep Engin and Adriano S. Koshiyama – University College London

This tutorial will be split into two parts: 

1. Overview of Digital Ethics & Key Concepts: the first part of the session will concentrate on providing an overview of the Digital Ethics landscape visiting a range of issues around data, algorithms and all types of interactions in this space. It will then introduce the key concepts of this growing field of engineering and computer science – including algorithmic bias and discrimination, privacy, explainability, regulation, and legality. The concepts will be introduced through exploring case studies from a diverse range of application domains including recruitment, advertising, banking, policing and criminal justice system. 

2. Technical Requisites for Algorithm Assessment: we aim to equip the attendees with a factsheet on what to request in numbers and written declarations during an algorithm risk assessment. In particular, it will focus on Fairness, Transparency and Robustness (FTR), arguably the three most important, well researched and technically developed topics. For each RTR construct we will present (i) its conceptual and/or mathematical definition; (ii) outline the main questions it is applied to; (iii) present how it is fitted in a case study; and (iv) what type of numbers, charts, explanations, etc. a digital ethics analyst would need to come up with a proper assessment. 

Course aims and objectives

By the end of this tutorial we expect that the participant will understand the key concepts that underlie Digital Ethics, Fairness, Transparency and Robustness. Also, the participant will be capable of recognizing applications where these concepts can be used to refine automated decision-making (finance, government, retail, healthcare, etc.). As a final learning outcome, the attendee will be able to evaluate an algorithm from a Fairness, Transparency and Robustness perspectives, and come up with an initial judgment of its appropriateness to a specific problem.

Intended Audience

Policy-makers, practitioners and anyone with an interest in becoming more familiar with the language and principles of digital ethics.

  • Data Stewardship in Action: Workshop on Making Data Collaboratives Systematic, Sustainable & Responsible (Morning) 

Stefaan Verhulst – The GovLab, New York University     

There are many predictable uses for data collected and held by the private sector that can transform public policy. These traditionally untapped data assets have fuelled interest in “data collaboratives.” 

Today, establishing and sustaining these new collaborative and accountable approaches requires significant time, effort, and resources for both data holders on the supply side and institutions representing the demand. By establishing data stewardship as a function, recognized within the private sector as a valued responsibility, Data Collaboratives can become more predictable, scaleable, sustainable and de-risked. 

Session 

This session will take stock and review existing efforts of data stewardship to professionalize and systematize the cross-sector exchange of data to create new public value. Participants will brainstorm actionable approaches for responsibly sharing corporate data to address a specific, concrete public problem raised by a public official who can ground the conversation in a real issue. 

Through this discussion, the participants will also reflect on lessons learned and previous practice to share broader lessons regarding data collaboration and data stewardship. Participants will consider the value proposition(s) of these emerging practices; questions related to risks and mitigation strategies; technical, legal, and cultural barriers and challenges; tools and methodologies for creating an impact through data collaboration and stewardship; best practices for achieving sustainability; and innovative metrics of success and evaluation techniques, among other issues. 

Outcomes 

Participants will become familiar with public/private data collaboratives, as well as corporate data steward practices through a discussion focused on solving a real-world problem. They will also learn about the challenges and demands in establishing sustainable and meaningful partnerships. 

The session will deliver several approaches to making data collaboratives more systematic, sustainable and responsible and provide an avenue for post-workshop engagement with all participants interested in prototyping solutions. 

In addition to sharing knowledge on tools, methodologies, and questions related to data stewardship, the workshop will inspire Data for Policy participants to consider new data-driven approaches for solving public problems. It aims to increase experimentation and leveraging the insights and tools discussed in the workshop to decrease the transaction cost, time, and energy needed to establish data collaboratives.

  • Introduction to Artificial Intelligence in Government (Afternoon) 

Jasmine Grimsley, Isabela Breton – Office for National Statistics, and Barbara Webber – Cabinet Office

This course will be delivered in collaboration between the Cabinet Office/GDS Academy and the Data Science Campus at the Office for National Statistics. It is targeted at anyone interested in understanding what AI is, how it’s being used in government now and how to get started with AI.

Course aims and objectives

This course will deepen your knowledge of the use of AI in government. By the end of the course, you will be able to:

• describe AI in terms of machine learning, deep learning and robotic process automation, and identify what is needed for each to be successful

• recognise key terms such as supervised and unsupervised learning

• discuss the importance of ethics and transparency in using this new technology in government

• explain the facts of AI and dispel myths and science fiction

Intended Audience

This 3 hour course is suitable for anybody interested in what AI can do. It would be of interest to leaders in a non-digital environment as well as practitioners who are considering robotic process automation or AI. This workshop will introduce AI and machine learning, discuss some myths in AI, ethics and where to get started in government.

  • Big-Data for Policy Making & Digital Transformation (Afternoon) 

Francesco Mureddu – Lisbon Council, Italy

It is expected that a particularly important actor, such as the public sector, should constitute a successful disruption paradigm through adopting novel approaches and state-of-the-art ICTs to use data and help establish new types of evidence-informed policies. However, despite continuous investments and initiatives in the public sector, it is hard to allege that “we are already there” when it comes to full exploitation of data towards aiding the public sector to meet the emerging societal challenges. Therefore, the workshop would like to offer its perspective on how barriers that impede big data driven modernisation in policy making can be overcome. 

Course aims and objectives

The aim of the workshop is to train relevant stakeholders in the use of Big Data in the Policy Making, explaining in particular the disruption that such technology can bring to public administration. 

The workshop aims at explaining how to renovate the public sector on a cross-border level by presenting methods, technologies, tools and applications from both the public & the private sector, stepping on the power of open innovation and the rich opportunities for analysis and informed policy making generated by big data. 

We will propose short and midterm milestones and relevant actions needed towards achieving the expected impacts for the public sector and society at large, by discussing key policy making challenges of how to: • Extract citizens’ opinions • Identify real-time proxies for official statistics • Anticipate detection of problems • Uncover causal relationships behind policy issues • Anticipate or monitor in real time the impact of policies • Identify key stakeholders to be involved in or target by specific policies • Generate a fruitful involvement of citizens in the policy making activity

Intended Audience

This course is suitable for policy-makers, public sector practitioners, and others particularly interested in Horizon 2020 and understanding value co-creation in public services for transforming European public Administrations.

  • Collaborating with Universities: Marriage made in Heaven? (Afternoon)

Olga Sergushova, Vania Sena and Gina Yannitell Reinhardt – University of Essex 

The Catalyst Project, funded by the Higher Education Funding Council for England (HEFCE) and monitored by the Office for Students (OfS) is a partnership between the University of Essex and the County Councils for Essex and Suffolk that uses cross-disciplinary expertise in data analytics to assess risks for vulnerable members of the community and provide evaluation techniques to fully understand the impact of Council initiatives.

Course aims and objectives

To illustrate the benefits and risks of partnerships between Universities and Councils, by presenting two interactive predictive and evaluative examples: 

1. The Catalyst Project’s Risk Stratification team partnered with the Suffolk Multi Agency Safeguarding Hub (MASH) to build and test a machine learning approach (algorithm). This session will include short talks from representatives of both organisations, a hands on demonstration of a risk model platform, and discussion of benefits/risks of using live data. Participants will be invited to interact with the platform and play with a mock up dataset.  We use this example to discuss how to achieve data sharing agreements, deal with confidentiality issues, and communicate predictive analysis and machine learning techniques and outputs to lay users.

2) The recently developed Spotlight Evaluation Toolkit is an easy to access and streamlined evaluation process for policy makers and public commissioners. We begin with an introduction to evaluation, and an overview of different approaches and assessments of evaluation’s importance given risks and limited budgets. We then use the ‘Accidental Dwelling Fires in Essex’ as an example, with  commentary from the Essex Fire and Rescue Home Safety team describing how working with an academic partner has ensured that evaluation has been integrated into their Fire Service approach, and will become vital in  reporting prevention activity. Participants can test the tool and can start to develop an evaluation framework for one of their programmes as part of the session.

Intended Audience

Policy-makers, public programme commissioners, and others interested in potential university partnerships, and examples of methods and tools for using predictive analysis and evaluation to support public sector decision-making.

  • Data Sharing & Data Trusts (Afternoon; please note that this was previously advertised as a morning session)

Gefion Thuermer, Johanna Walker; University of Southampton, Peter Wells; Open Data Institute, and Kieron O’Hara; University of Southampton

Course aims and objectives

The goal of the workshop is to enable participants to

a) Recognise the benefits of and obstacles to data sharing for themselves

b) Define why they may or may not want to share data

c) If they do want to share data, develop the grand picture of how this should be done

This will be achieved through a series of talks, case studies and activities. The talks will provide an overview of the subject area, and ensure all participants are on the same page. A case study will then exemplify the individual aspects of data sharing. The activities will guide attendees to collaboratively answer key questions about data sharing:

1. What is data sharing, where and how can or should data be shared

2. What role trust plays in data sharing and how it is generated and maintained

3. Which models of data access and sharing are suitable in different circumstances

Participants from different backgrounds will learn from the combined expertise of the researchers from the University of Southampton, the practitioners from the ODI, and also from each other. This will enable them to understand data sharing not only from their own perspective, but also from the perspective of others with whom they might share data. 

Intended Audience

This 3 hour course is suitable for anybody interested in data sharing. Participants should have some general awareness of data sharing. This could be as practitioners, where they share, receive, or otherwise work with data; as researchers, where they collect, analyse, or attempt to access data; or as policy makers, who work on the regulation of data sharing processes.