7. Monitoring data use
This step can be implemented as follows
The person promoting data sharing in the organisation
- monitors the use and benefits of published data.
- reports on the use and benefits of data.
Monitoring of use
This part describes why and how the organisation should monitor the utilisation rate of the opened data.
No official recommendations exist on monitoring the utilisation rate of opened datasets.
Decisions to invest in open data publications often must be justified by value creation, or benefits derived from the investment. The organisation may also have set quantitative targets for such benefits. It is important to measure the benefits created by open data, making it possible to report them and to use them to justify future investments.
Organisations that have already opened their data are often interested in finding out the extent to which their datasets have been viewed in or downloaded from the data portal. It is important to note that while the viewing and downloading data concerning datasets published on the portal or other online service do not indicate if the dataset has actually been used and how, they do give an indication of the interest attracted by the opened dataset.
Measuring open data is difficult, but the scale of its use can be measured and estimated. Possible metrics include view and download statistics and interface use. Media visibility as well as the number of organisations and applications that use the data can additionally be measured. Information on open data use can, for example, be collected by means of media monitoring (keywords) and surveys or by offering various incentives to report on data use.
Examples of ways of measuring data use
|Measurement area||Example of a metric|
|Downloads||Number of files downloaded|
|Views||Number of files browsed|
|API use||API usage statistics|
|Monitoring of data use||Media monitoring and press articles|
|Monitoring of data use||Number of organisations using the data|
|Monitoring of data use||Number of applications using the data|
|Monitoring of data use||Use of data in hackathons|
|Monitoring of data use||Contacts by higher education institutions|
|Feedback||Contacts, number, content, background organisations|
The format in which the data was published also affects the monitoring of its use. Different statistics can be obtained on the use of data published in a dynamic form, often through an API, and its use is reasonably easy to monitor. On the other hand, monitoring the use of static data (files) is more challenging.
Publishing data as a file
Static data is usually downloaded only once, after which the user has unlimited access to it. Static data downloads do not necessarily convey any information about the extent of data use or the impact of its publication. Open data licences also allow the re-sharing of data, which means that other parties can also share the file in their services. For example, an individual media may download the dataset once and publish a news item based on it, which spreads to several other media. Consequently, even one download can have a high impact.
Providing data through an API
Analytics can be collected on API use, based on which the interface can be developed, and data quality can possibly also be improved. Statistics can be compiled using various programs, including Google Analytics. For example, analytics can be collected on the general volume and methods of data use, interface requests and user locations. If an API key is needed to use the interface, the keys can help tell data users apart. API keys can also be distributed anonymously, allowing data users to retain their anonymity, and the use of the API key does not undermine the basic principles of openness.
Monitoring methods used in opendata.fi service
The views and downloads of data published on opendata.fi are displayed openly on the page for each dataset. In addition, the Statistics page of the service provides information on the most popular datasets and similar. The service also has an application gallery, which maintains an overview of applications that use open data, with emphasis on the central government’s open datasets. The opendata.fi service also uses media monitoring (based on such keywords as open data, hackathon).
Monitoring methods used by the Finnish Meteorological Institute
The Finnish Meteorological Institute uses Spatineo Monitor for usage monitoring (downloads, user groups). In addition, the development of a continuous situational picture and dashboard is under way with Spatineo. The Finnish Meteorological Institute reports on the use of its open data in its data balance sheet.
Monitoring of benefits
This part describes how the organisation can monitor the benefits gained from the use of the opened datasets.
No official recommendations exist on monitoring the benefits gained from opened datasets.
The benefits and impacts of opening data are impossible to estimate accurately, as the opening of data has many indirect benefits that are difficult to measure. No established set of indicators exists for evaluating the impact of data, as the development of research and measurement tools is only taking its first steps.
Organisations that have already opened their data have striven to monitor and measure the outcomes and impacts of opening data after publishing their datasets.
The following methods can be used to obtain information on the impact of the opened data:
- Collecting statistics on data downloads and, if possible, use cases.
- Monitoring data use in new services, innovations and applications.
- Collecting information on the impacts of the data directly from users by means of various surveys.
- Conducting a cost-benefit analysis on the economic benefits of opening the data. The monetary value of many benefits is difficult to measure, however.
- Commissioning reports or studies.
When assessing the impacts, it is advisable to take into consideration the entire life cycle of the data and the impacts of opening it from a wider societal perspective. Assessing the impacts can help the organisation to improve its effectiveness, fulfil its public mission better and increase the transparency of its work. Monitoring data is also useful when, for example, Finland reports annually on the benefits created by open data in an EU comparison study.
The data.europa.eu service administrated by the European Commission conducts an annual Open Data Maturity in Europe survey, in which European countries are compared on the basis of the impact of open data.
It assesses the level of maturity in the field of open data against four dimensions:
- Political impact: How has open data improved the efficiency of public administration? How have transparency and responsibility increased in public administration? Has open data been used to support decision-making? How? Are events being organised to support knowledge-based decision-making? How has impact been monitored?
- Social impact: Has open data promoted democracy and equality? For example, has it increased minorities’ participation in decision-making and access to administrative services? Are civil society activities supported?
- Environmental impact: Does open data affect the status of the environment? Does it affect sustainable development and the circular economy? In what way has it been used as part of resource efficiency and environmental protection?
- Economic impact: What kind of economic value does open data create? Has the value created by open data been examined at national level? Have the financial benefits for the administration been investigated (during the past year)? Has other research been conducted on the impact of open data?
Read more about assessing impact on data.europa.eu.
Example: Open environmental data
The Finnish Environment Institute offers open data to assist in the building of a sustainable environment and society. The Institute's open data service provides datasets on surface waters and groundwater, the Baltic Sea, environmental pollution and disturbances, valuable natural environments, soil cover and the built environment, among other things. This data has mainly been produced and collected by the agencies of the central government's environmental administration, in particular the Finnish Environment Institute and the Centres for Economic Development, Transport and the Environment.
The Finnish Environment Institute's goal is to be a pioneer in the development of the environmental data value chain. Open data serves this strategic goal. To support the assessment of impact, open data use is monitored by annually collecting information on interface usage and file downloads.
In addition, the Finnish Environment Institute has conducted surveys addressed to, and individual interviews with, open data users. Surveys can give a general idea of such aspects as the users of open environmental data, the environmental themes that attract users’ interest, and general information about the uses made of the data. In surveys, as uses of open environmental data have been mentioned various tasks related to decision-making and planning, research and education, investigating the quality of the environment, influencing matters related to user's immediate surroundings and maintaining or developing different products and services.
Example: The Finnish Meteorological Institute
The Finnish Meteorological Institute conducted an open data impact study in 2019. This analysis was found useful, and the Finnish Meteorological Institute is planning to conduct similar studies at regular intervals in the future. The results of the study were published in the Finnish Meteorological Institute's Data balance sheet 2020.
Studies and reports on the use of open data
Turunen, S. & Ruokonen, T. 2018. Avoimen datan hyödyntäminen kuntasektorilla (Using open data in municipalities). Download avoimen_datan_hyodyntaminen_kuntasektorilla.pdf-file.
Herrala, Antti. 2018. Benefits from Open Data: Barriers of Supply and Demand of Open Data in Private Organizations .
Muurinen, R. & Open Knowledge Finland. 2017. Liiketoimintaa avoimesta datasta. 100 tapauksen ajankohtaiskatsaus (pdf). (Business from open data. 100 use cases).
Koski, H., Honkanen, M., Luukkonen, J., Pajarinen, M. & Ropponen, T. 2017. Avoimen datan hyödyntäminen ja vaikuttavuus. Publication of Government’s analysis, assessment and research activities. (The use and impact of open data).
Koski, H. 2015. The Impact of open data – a preliminary study.Ministry of Finance publications.
Kukkamäki, J., Mikkonen, A., Stormi, I. & Mäntyneva, M. 2020. Avoimen datan hyödyntäminen opetuksessa ja tutkimuksessa. (Using open data in education and research). In J. Kukkamäki & M. Tarkkala (eds.), Avoin Häme. HAMK Unlimited Journal 25 Aug 2020.
Poikola, A., Kola, P. & Hintikka, K. A. 2010. Julkinen data - johdatus tietovarantojen avaamiseen. (Public data - guide to opening resources). Ministry of Transport and Communications.
Verhulst, S. G. & Young, A. 2018. Open Data Demand. Toward an open data demand assessment and segmentation methodology.
Ahonen-Rainio, P., Mäkelä J. & Virrantaus K. 2014. Menetelmä avoimen maastotiedon vaikuttavuuden arvioimiseksi (pdf). (A Method to assessing impact of open spatial data)
Muukkonen, M., Leppälä, S., Kauhanen-Simanainen, A. & Myllyniemi, N. 2022. Julkisen sektorin tietojen maksullisuuden ja maksuttomuuden vaikutukset. (Public sector data - the impact of fees). Publication of Government’s analysis, assessment and research activities 2022/50.