7. Monitoring data use
This step can be implemented as follows
The person promoting data sharing in the organisation
- monitors the use and benefits of published data.
- reports on the use and benefits of data.
Monitoring of use
This part describes why and how the organisation should monitor the utilisation rate of the opened data.
No official recommendations exist on monitoring the utilisation rate of opened datasets.
Decisions to invest in open data publications often must be justified by value creation, or benefits derived from the investment. The organisation may also have set quantitative targets for such benefits. It is important to measure the benefits created by open data, making it possible to report them and to use them to justify future investments.
Organisations that have already opened their data are often interested in finding out the extent to which their datasets have been viewed in or downloaded from the data portal. It is important to note that while the viewing and downloading data concerning datasets published on the portal or other online service do not indicate if the dataset has actually been used and how, they do give an indication of the interest attracted by the opened dataset.
Measuring open data is difficult, but the scale of its use can be measured and estimated. Possible metrics include view and download statistics and interface use. Media visibility as well as the number of organisations and applications that use the data can additionally be measured. Information on open data use can, for example, be collected by means of media monitoring (keywords) and surveys or by offering various incentives to report on data use.
Examples of ways of measuring data use
| Measurement area | Example of a metric |
|---|---|
| Downloads | Number of files downloaded |
| Views | Number of files browsed |
| API use | API usage statistics |
| Monitoring of data use | Media monitoring and press articles |
| Monitoring of data use | Number of organisations using the data |
| Monitoring of data use | Number of applications using the data |
| Monitoring of data use | Use of data in hackathons |
| Monitoring of data use | Contacts by higher education institutions |
| Feedback | Contacts, number, content, background organisations |
The format in which the data was published also affects the monitoring of its use. Different statistics can be obtained on the use of data published in a dynamic form, often through an API, and its use is reasonably easy to monitor. On the other hand, monitoring the use of static data (files) is more challenging.
Publishing data as a file
Static data is usually downloaded only once, after which the user has unlimited access to it. Static data downloads do not necessarily convey any information about the extent of data use or the impact of its publication. Open data licences also allow the re-sharing of data, which means that other parties can also share the file in their services. For example, an individual media may download the dataset once and publish a news item based on it, which spreads to several other media. Consequently, even one download can have a high impact.
Providing data through an API
Analytics can be collected on API use, based on which the interface can be developed, and data quality can possibly also be improved. Statistics can be compiled using various programs, including Google Analytics. For example, analytics can be collected on the general volume and methods of data use, interface requests and user locations. If an API key is needed to use the interface, the keys can help tell data users apart. API keys can also be distributed anonymously, allowing data users to retain their anonymity, and the use of the API key does not undermine the basic principles of openness.
Monitoring of benefits
This part describes how the organisation can monitor the benefits gained from the use of the opened datasets.
No official recommendations exist on monitoring the benefits gained from opened datasets.
The benefits and impacts of opening data are impossible to estimate accurately, as the opening of data has many indirect benefits that are difficult to measure. No established set of indicators exists for evaluating the impact of data, as the development of research and measurement tools is only taking its first steps.
Organisations that have already opened their data have striven to monitor and measure the outcomes and impacts of opening data after publishing their datasets.
The following methods can be used to obtain information on the impact of the opened data:
- Collecting statistics on data downloads and, if possible, use cases.
- Monitoring data use in new services, innovations and applications.
- Collecting information on the impacts of the data directly from users by means of various surveys.
- Conducting a cost-benefit analysis on the economic benefits of opening the data. The monetary value of many benefits is difficult to measure, however.
- Commissioning reports or studies.
When assessing the impacts, it is advisable to take into consideration the entire life cycle of the data and the impacts of opening it from a wider societal perspective. Assessing the impacts can help the organisation to improve its effectiveness, fulfil its public mission better and increase the transparency of its work. Monitoring data is also useful when, for example, Finland reports annually on the benefits created by open data in an EU comparison study.