It is important to clearly designate a single party (responsible role) for opening the data, which parties outside the organisation can contact about open data questions. It is also a good idea to organise developer meetings and similar, where you can collect feedback even before opening the data.

The data controller should consider how metadata and a verbal description of the dataset will be produced, and how any need for user support will be responded to after publication.

Ensuring digital security is important when planning and going ahead with opening data. Digital security includes matters related to risk management, continuity management and preparedness as well as cyber security, information security and data protection. Citizens, companies and communities must be able to rely on ethically sustainable, open and transparent public administration services also in the digital environment.

Government Resolution on Digital Security in the Public Sector (Ministry of Finance publications 2020:23) defines the principles of development and key services for promoting security in digital environments. To promote digitalisation and digital security in a balanced manner, the Ministry of Finance has appointed a strategic management group for digitalisation and digital security in public administration.
Find out about actions and documents related to developing digital security (in Finnish).

As part of the operating model for data sharing, an assessment method was developed to help public administration organisations to assess the potential benefits of opening and sharing their datasets and the risks and costs associated with data sharing. This tool is also known as the BRC method (benefits, risks and costs). 

Download the BRC assessment tool (in Finnish, Excel file)

The BRC tool is an Excel sheet that provides an overview of assessment results based on the responses the user has filled in, helping to determine the potential benefits, risk profile and costs arising from opening the dataset. It should be noted that the summary based on the responses is only an indicative overview of different observations, not a recommendation. Each organisation makes the decisions on opening its data independently, taking into account legislation (including access rights and data disclosure), official guidelines and its internal policies. For example, the summary can be used as background material to justify the potential benefits of data sharing to those who make the decision on opening data.

The BRC method is based on assessment methods used by public administration organisations in Finland and internationally. Such parties as the National Institute of Standards and Technology, the University of Washington, Harvard University and several other expert organisations have been involved in developing the first version of the method.

The assessment method can be used to:

• prioritise the order in which datasets are opened when resources are limited
• identify datasets whose opening involves different risks
• reach an understanding of the costs incurred from opening data
• identify datasets with the greatest potential benefits for external stakeholders (data users) 
• analyse possible income obtained from sharing data
• introduce a systematic approach to the opening of datasets and decision-making on data sharing

The assessment method is intended for those responsible for opening data in organisations. They may include those responsible for datasets, heads of information management or data opening coordinators. In addition, it is advisable to involve specialists of each area from technical experts to data protection officers in the different stages of the assessment.

State Treasury analysts carry out analyses on assignment. In such analyses, the shared information platform of the central government is mainly used, to which datasets specified in the assignment are imported. Together with the customer, the analyst defines the data areas needed for the analysis using a data navigator. The navigator contains descriptions of the data residing in the systems of central government's joint service providers. In this description, the service provider and agencies have specified if a field may contain personal or secret information. A predefined set of data masking rules is generated for the described fields.

The analyst places an order for data that matches the assignment with the data engineer. The data engineer retrieves the necessary columns from the service providers' systems through interfaces, removing any unnecessary columns from the dataset to minimise the data, and masking the data following the regulations:

• Text fields that may contain personal data are deleted
  • For example in financial monitoring data, description texts 1 and 2 for the monitoring target are deleted
  • For example, fields containing personal names or e-mails are deleted
• In the dataset, personal identifiers are encrypted using an encryption algorithm, ensuring that the original value cannot be identified while preserving the uniqueness of the identifier
  • For example, using a cryptographic sealing function, the contents of fields containing personal identity codes are converted into a string from which the original value cannot be directly derived
• The State Treasury may not have access rights to the data at a certain level of accuracy, but the data aggregates produced from it may be public. In these cases, the service provider aggregates the data to a public level defined together with the agencies on a case-by-case basis. In this context, aggregation refers to the re-grouping of data on the basis of one or more factors to a less accurate level.
  • For example, instead of the operating unit, the sum or average of the accounting unit level is shown
  • For example, trips to different continents are shown, instead of to individual countries

The masked and minimized data is transferred to the platform for the use of the analyst. The analysis is carried out based on the masked data that does not contain directly identifiable personal data. If the analyst finds that the data may contain direct personal data, however, they inform the data engineer of this, making it possible to adjust the masking rules for the fields in question, and refrain from processing the dataset before the correction has been made and the dataset is free from personal data. Once the analysis has been carried out, the analyst aggregates the results to a statistical level before the results are presented to the customer. This means ensuring that the groups to be shown contain data concerning at least five persons, making sure that no individuals can be identified in the results.

To ensure the data protection of a dataset, it is important to examine if there are target units in the dataset whose identity or attributes could be disclosed directly or indirectly. A precondition for direct identification is that the dataset includes a unique attribute of the target unit, such as a name, address or business ID. Indirect identification is possible when the target unit can be identified on the basis of several attributes, for example ‘mayor’ as the occupation and the municipality in which the person works as additional information. Some attributes of an individual target unit may also be revealed without identifying the target unit when a larger group to which the target unit belongs shares some of the same properties. In a survey on well-being at work, for example, all employees of a certain department have responded to the survey and expressed their dissatisfaction with the physical working environment.

When assessing this risk of disclosure, there is a major difference between talking about unit-level datasets or data that has been aggregated in some way. When processing unit-level data in which the properties of an individual target unit are examined by target unit, indirect disclosure may still be possible even if the data has been aggregated by attributes. Longitudinal datasets which examine the situation of the target unit over the long term are a good example of this. The mobility or work history of a person can very quickly lead to a situation where the possibility of indirect identification cannot be excluded, even if the data is aggregated to some extent. In the case of unit-specific datasets, the risk of disclosure should consequently be examined across a broad front, taking several attributes into account simultaneously. In general, anonymisation of unit-level datasets using aggregation and data delimitation results in small datasets mainly used as examples. Alternative data protection methods include data scrambling, (multiple) imputation or the production of synthetic datasets. 

Statistics Finland has produced anonymous unit-level datasets intended for teaching purposes. The results obtained from these datasets may be indicative, but they are in no way suitable for producing statistical reports or scientific research. Read more about teaching datasets.

Aggregated data refers to data in which the attribute values of several target units have been compiled. This data can be divided into frequency tables describing the number of target units and quantity tables describing attribute values which, for example, describe totals or averages of an attribute. For frequency tables, the disclosure risk is determined by the cell value of each cell as a threshold value which the target units in the cell must reach at minimum. The threshold value depends on the attributes to be examined. In its official population statistics, Statistics Finland partly includes even individual persons in the statistics. Generally, however, at least three target units in a cell are required to ensure data protection. Applying this minimum value avoids situations where two target units sharing the same attributes could infer each other's values on the basis of the published data. Statistics Finland applies a higher threshold when examining data at a geographical level more accurate than a municipality (the threshold may be as high as fifty when looking at grid data) and, usually, a threshold of ten is applied to special category data referred to in the General Data Protection Regulation or crime data. 

For quantity tables, merely looking at the threshold value is not enough to prevent the attribute values of another target unit from being inferred if the target units are in the same cell. In this case, Statistics Finland also applies the dominance rule to identify cells involving a disclosure risk. This means that cells in which a single target unit, or several target units together dominate (produce the majority of the value of the cell), are flagged as subject to protection. For example, if the cell examines company turnover by industry and region, it should not be possible to infer the value of an individual large company in a cell where the other companies have very low turnovers in proportion to the largest.

The threshold value or dominance rule can be used to determine the cells presenting a primary disclosure risk. If the data is deleted (masked) in the published dataset, recalculating their values is easy if the dataset also contains marginal amounts, or totals of rows and columns. In this case, additional masking must be used to ensure data protection. Special software is available for complementary masking, the use of which ensures adequate protection when determining the cells subject to secondary masking. Such special software includes Tau-Argus and sdcTable R package. More information about software on GitHub.  

For more information on data protection, see Statistics Finland’s resources for researchers:

• Data protection instructions of services for researchers (in Finnish, pdf)
• Research datasets in remote use guide (in Finnish)

In cooperation with the City of Helsinki's data protection officer, Helsinki Region Infoshare has created instructions for opening survey datasets (and other data containing personal data) (in Finnish).

VAHTI is a cooperative, preparatory, and coordinating body of organisations responsible for the development of digital security in public administration. Organisations can use best practices and VAHTI guidelines to develop different areas of security. 

VAHTI activities were transferred to the Digital and Population Data Services Agency in early 2020.

• Information about VAHTI activities on the website of the Digital and Population Data Services Agency.
• VAHTI guidelines drawn up in 2001–2017. Some of the guidelines contain references to outdated legislation.

Outdated recommendations can still be applied if legislative amendments are taken into account.

The Digital and Population Data Services Agency has produced several training packages on life with digital security on eOppiva, for example online training on risk management in the digital world and the ABC of data protection.

A file is a suitable way of sharing small and/or static datasets that do not change much or often. High quality open data is shared using an open file format.

An open file format means a non-commercial format that anyone can use free of charge. The use of open file formats is not restricted by copyrights, patents, trademarks or other restrictions. For example, Microsoft’s .docx or .xslx file formats are commercial, not open, and using them with free software is difficult. Open file formats usually enable software-independent re-use of data. This is important in order to ensure that commercial rights do not restrict the re-use of data.

The list below contains tips for publishing different types of datasets:

• Text data: TXT. The easiest and most reliable file format for publishing text files is .txt.
• Tabular data: CSV. The best and easiest file format for tables is .csv (Comma-separated Values). CSV files can be easily created using common spreadsheet programs, including Microsoft Office Excel, by selecting csv as the file format when saving.
• Spatial data, small vector data: GeoJSON, KML, Esri shapefile (shp) or GeoPackage. The first two options use the global WGS84 coordinate system, which can be easily processed with a variety of programs and tools. An shp file, on the other hand, supports several coordinate systems, including those developed for the Finnish conditions.
• Location data, big raster data: GeoTIFF or NetCDF. To publish data in raster format, for example GeoTIFF file format can be used. 

When sharing data in PDF format, it is advisable to pay attention to the PDF version used and ensure that the data is in a machine-readable format. Adobe developed and patented PDF in the 1990s as a commercial file format. In 2008, its version 1.7 (ISO 32000-1) was standardised as an almost open format, while some of its features remained Adobe’s property (including Adobe XML Forms Architecture, Adobe JavaScript). In PDF 2.0 version (ISO-32000-2) published in 2017, however, all features were open. Read more about open file formats. 

Read a comprehensive list of open file formats in Wikipedia.

In Tim Berners-Lee's five star model, data published in an open file format should have at least 3/5 stars.
 

What is an API?

Application Programming Interfaces (APIs) are documented interfaces through which software, applications, or systems can exchange data or functionalities. The API provides data or a functionality in a machine-readable, documented format, making it possible for some other software, application or system to use it programmatically. 

In this operating model, API, application programming interface and technical interface referred to in the Public Information Management Act mean the same thing. It should be noted that rather than referring to an interface intended for end users, APIs are always used by some other software, application, application component or system.

Why use APIs for data sharing?

Sharing data through APIs is in many ways advisable and useful, especially if the volume of the data is very large and the dataset is updated frequently or in real time, in other words comprises dynamic data. This includes train timetables or weather data. However, it is worth remembering that file sharing is also useful especially for those persons and parties who are unable to use APIs. If an API is not otherwise in use, file sharing may demand significantly less resources of the sharing party than implementing and maintaining a new interface.

An API can be a web-based and, for example, a file-based or database-based interface implemented with REST, SOAP or GraphQL technologies. The essential point is that the API provides data in a machine-readable, documented format, making it possible for some other software, application or system to use it programmatically. Providing the data through web-based interfaces is a good idea if this is possible and consistent with the purpose of use. 

Web-based APIs can be used in both internal and external interfaces, and a wide range of information security controls can be implemented in them. The file format to be shared depends on the communication protocol. For example, web interfaces usually use a https based communication protocol or architecture, such as REST. API interfaces are also highly suitable for sharing statistical data residing in a database. For example, see the datasets in Statistics Finland's open databases.  

It is important for the organisation to determine which types of datasets are offered or used through APIs internally and externally, and which datasets should be accessible through user interfaces. Internal access and use can be implemented through internal interfaces (internal APIs). To provide for external access and use, partner interfaces (partner APIs) or public interfaces (public APIs) can be used, depending on the classification of the data. The essential point is that the interfaces are taken into account as part of the organisation's other information management and operating processes as well as the goals of knowledge management. 

Using the national API principles in the design and development of APIs is advisable. The public administration API principles provide support and instructions for public administration actors in the development, management and file formats of APIs. Among other things, the API principles provide support for specifying the interfaces, assigning of responsibilities, promotion of interoperability, procurement, testing and implementation of APIs. When designing the API, it is important to specify how changes to the life cycle plan or service level of the API are managed.

Additional information and support material for API development, management and file formats:

• Public administration API principles
• Information Management Board’s recommendation on technical interfaces and viewing access (in Finnish) (Ministry of Finance publications 2021:21)
• National Land Survey of Finland’s instructions for API key use
• API technologies and their implementation (in English)

The table below can be used to support the selection of an appropriate form of data sharing. It strives to highlight the differences between the possible forms.

The Finnish Meteorological Institute shares its data through its own interface services and Amazon AWS.

Helsinki Region Infoshare service for the cities in the Helsinki Metropolitan Area has compiled tips for assessing technical feasibility. The questions below will support the assessment. 

In which format should data be opened?

As a file:

• File in which the data is maintained (xlsx/csv/shp/ …) 
• The data is exported manually from the system
• The data is exported automatically from the system
• Usually a quick and free-of-charge way to open data, but often requires manual and memory-resident updates

Through an API:

• An API is created for data that is exported from the system automatically
• An API is produced for the system/system copy
• More work and resources are required at the beginning, but no separate updates are required

Questions that should be considered when selecting the data format:

• How often is the data updated?
• How large is the data volume?
• Is it real-time or, for example, annually compiled data?
• How much manual work does editing the data take?
• What could the data be used for?
• Are there any standards?
• Has any other party already opened similar data? How was it done? Would it be possible to open the data in a similar format?

Helsinki Region Infoshare has developed a tool named Datasette that makes it possible to publish data available through an interface in a file format. Read more about Datasette on HRI’s website (in Finnish).  

A public Data Quality Framework has been developed under the leadership of Statistics Finland and through broad-based cooperation within public administration. This work has been carried out as part of the Ministry of Finance's project on Opening up and using data. The data quality criteria and indicators were published in spring 2020. 

The data quality criteria can be used to describe and assess the quality of data. They also help data users to assess if the data is of sufficiently good quality for the intended purpose. In the longer term, the quality criteria help improve the quality of data and information resources.

The quality criteria are intended as a flexible tool; not all criteria or, in particular, indicators are necessarily relevant to all situations or data sets. It should additionally be noted that the purpose of the data affects the level of each quality criterion that should be aimed for. For instance, for some purposes continuously updated data is needed (pandemic monitoring), whereas for others, annual or even less frequent updates (location of old buildings) is sufficient. While the quality criteria and their indicators add up to a hierarchical structure, they also affect and are linked to each other.

The quality criteria of the quality framework, and especially their indicators, focus on structured data. From the data user’s perspective, the quality criteria for datasets have been grouped under three questions.

How well does the data describe reality?

• Timeliness: Timeliness describes the time dimension of datasets. The closer the reference date of the dataset is to the present, the more timely the data will be. The reference date is the date to which the data relates.
• Coherence (regularity, logical integrity of data): Coherence indicates that the dataset is consistent and non-contradictory. Coherence can also be used to describe consistency between different datasets.
• Completeness (extensiveness): Completeness describes the temporal and regional coverage of the dataset as well as the target units and attribute data that are aimed for. On the other hand, completeness indicates the extent to which the dataset contains the desired data.
• Correctness (validity): Correctness describes the extent to which the data in the dataset corresponds to reality. By examining the correctness of data, systematic distortions in the dataset may also be picked up.
• Accuracy (unbiasedness): Accuracy describes how well the data in the dataset corresponds to what is aimed for and how precise the data is.

How has the data been described?

• Traceability (non-repudiation): Traceability indicates that any changes made to the dataset and its data can be traced. The origin of the data is known.
• Intelligibility (interpretability, comprehensibility): Intelligibility describes the extent to which the dataset has metadata that helps to understand the data when in use.
• Compliance with recommendations (interoperability, semantic uniformity, consistency): Compliance indicates that the dataset and its attribute data comply with known standards, practices and statutes and that they have been reported in connection with the dataset.

How can the data be used?

• Machine readability: Machine readability indicates if the data has been structured to enable computerised processing, and processing in different information systems.
• Punctuality (timeliness): Punctuality means that the dataset is available on the given date and with sufficient frequency to reflect changes in the dataset.
• Access rights: Access rights describe how the rights to use the dataset have been defined and what users may do with the data, in other words what purposes the data can be used for. 

Read more:

• Summary of quality criteria on Statistics Finland’s website (in Finnish)
• Dataset quality criteria and indicators (in Finnish, pdf)​​​​​​

The CC0 licence means that all copyrights to the data are waived. Data licenced under a CC0 licence has been fully released for free use, both for commercial and non-commercial purposes. The data user does not need to acknowledge the origin of the data or request permission for its use. 

The metadata of datasets is often published under a CC0 licence. For example, the metadata on hri.fi service is CC0-licensed, which makes it possible for the metadata to be automatically copied to the opendata.fi service.

CC BY 4.0 or CC Attribution International 4.0 licence obliges the data user to acknowledge the origin of the data, as the name indicates. The data user must acknowledge the source, provide a link to the licence and indicate any changes that may have been made to the data. CC BY 4.0 licensed data can be used freely.

Example of acknowledgement

Helsinki Region Infoshare service recommends the following acknowledgement of using data published on the service: "Source: Revenue and expenditure of the City of Helsinki. Data maintained by Helsinki City Executive Office. Dataset downloaded from Helsinki Region Infoshare service on 15 November 2021 under Creative Commons Attribution 4.0 licence".

Rather than an official decision on opening data being made, in the City of Helsinki the data owner specifies the data to be opened without a formal decision-making process. The number of datasets opened in HRI is small enough to avoid any need for prioritisation. Any data opening processes that have been under preparation for longer are shown on the HRI website (in Finnish). 

At the Finnish Meteorological Institute, decisions on opening data through an interface and prioritisation are made by a steering group operating within the Institute.

The opendata.fi service is a free publication platform for the entire public administration. The platform operates on a self-service principle, which means that each authority can freely use it for opening and using data.