7. Maintenance and updates

This step can be implemented as follows

The data administrator and person promoting data sharing

  • update the shared data according to the specified frequency or need, for example when an error has been detected in the dataset, and
  • also update the metadata of the published data.

The person promoting data sharing together with the communications unit

  • inform data users of changes when necessary.

It is important to determine who is responsible for the maintenance and monitoring of the opened data and its metadata. For this reason, it is a good idea to appoint a responsible party for the opened data and include their details in the related metadata. This responsibility may, for example, be assigned to a unit or a role.

Maintaining metadata

This section describes what you should consider when maintaining and updating metadata. Below is a list of good practices for maintaining the metadata of opened data:

  • Metadata should be updated whenever e.g. any changes are made to the content, structure, or basic data of a dataset or API. This is important for ensuring that the opened data can be used and reused, for example.
    • In addition, concurrent support for several API versions should be taken into account whenever necessary.
  • The metadata should describe the update frequency of the datasets or APIs, and this information should always remain up to date. Where an API is used, the data may be automatically updated on a continuous basis, whereas a dataset may be updated annually, for example. 
  • Remember to also update the language versions of your metadata so that the data can be used by those who do not speak Finnish.

Practices employed by organisations

Practices of Helsinki Region Infoshare

The metadata of the data must be updated at least as frequently as the data itself. The metadata must indicate when manually updated data has been updated and when the next update can be expected. The metadata for automatically updated data should also be reviewed from time to time to check that it matches the data content.

The organisation should provide information about data updates in the same way as it communicates about the opening of new datasets, at least in the case of major updates. If possible, it is advisable to communicate about events such as API version updates in advance.

Practices of the Finnish Environment Institute

The Finnish Environment Institute uses a tool for describing the metadata (metadata editor) as well as a separate metadata service for end users

The metadata is always updated together with the dataset. The organisation checks at least once a year that the metadata for continuously updated datasets is up to date. The responsibility for producing the metadata and keeping it up to date rests with the persons responsible for the data and the system administrators of information systems.

Both the metadata editor and the metadata service offer open APIs for harvesting metadata. This means that any modified data are automatically updated in the Paikkatietohakemisto service maintained by the National Land Survey of Finland (metadata compliant with the INSPIRE Directive), the opendata.fi service maintained by the Digital and Population Data Services Agency (metadata for Finnish Environment Institute’s open datasets) and CSC's Etsin.fi service.

Updating opened data

This section describes how opened data should be updated and what you should do if an error is detected in your dataset.

Information risk management should be taken into account in the management of the updates made to the opened dataset. The Information Management Board has issued a set of recommendations for applying certain information security statutes (in Finnish) (Ministry of Finance publications 2021:65), according to which information risk management is a continuous activity, and the information management entity should describe the objectives, responsibilities and key methods related to it.

The metadata should describe the frequency at which the APIs and datasets will be updated. If, for some reason, the dataset needs to be modified before the date of the next update, the metadata should be checked and, if necessary, updated at the same time.

Some data portals provide the data administrator with the option of receiving an automated e-mail notification when the date specified for the next update in the metadata of the opened dataset approaches. This helps the data administrator ensure that the dataset and its metadata are up to date.

Updating datasets

A dataset can be updated either by adding data to an existing dataset or by creating a completely new version of the dataset. The FAIR principles recommend the creation of a new file.

The size of the dataset should remain at a level where its utilisation will still be feasible for users. It is preferable to divide a large file into multiple files, as a larger file size makes it difficult to process the material and often requires programming skills. Large file sizes also set higher quality requirements for data connections.

The dataset can be updated either manually or automatically. If the dataset is updated manually, its metadata must be checked and updated at the same time. However, if possible, the preferrable option is to update the data automatically.

You should also check the dataset’s inbound and outbound URIs and URLs regularly, so that your users can find the dataset.

Updating APIs

If data is shared through an API, any changes to the API's structure will affect the data’s users. If the API’s structure is modified, it is a good idea to create a new version of it, as any changes to its structure will require updating the programs that utilise it. When versioning the API, remember to pay attention to the publication of both backwards compatible and incompatible changes and fixes. In connection with this, be prepared to provide support for several API versions simultaneously, if necessary.

It is important for the data administrator to provide information about any changes to the API to all current and potential users of the API, either through different communication channels or directly, if the use of the API requires signing up for an API key. It is a good idea to inform users about API version updates in advance. In addition, it is important to describe the changes made to the API in its metadata. It is common practice to inform users about the lifecycle of each API version and provide them with enough time to migrate to a newer version.

We recommend following the national API principles when maintaining your API. It is important to create a process for maintaining APIs and have indicators for monitoring API use. Read more about the API principles, Publications of the Ministry of Finance 2022:12 (pdf).

Update needs of opened data and how you should act in problem situations

A new version of the data is available

The data contained in the dataset should always be kept up to date.

Even if a new version of the data is published, you should not delete the old version of the data. Historical data can be used for various purposes, such as annual comparisons. 

Improving data quality

You should focus on continuously improving the quality of your published data. For example, the data administrator can improve the coverage of their data by providing it in several different file formats.

The data administrator should encourage users to report any problems, deficiencies, or weaknesses in the quality of the data. Users can submit their proposals for improving the data's quality.

Users can be invited to contact the party specified in the metadata directly, such as the contact person, customer service, or registry office of the organisation that opened the data.

The data administrator should respond as quickly as possible to any shortcomings or problems identified and take the necessary corrective action. 

Fixing and communicating about data errors

The data administrator should draft a plan for any situations where an error is detected in the opened data or personal data or other sensitive information is discovered in the data, despite conducting the necessary data protection and information security checks. The plan should specify the responsible persons and measures to be taken in such situations. It is also important for the organisation to plan and agree on how data protection and information security issues relevant to the opened data are monitored and developed.

In problem situations involving data protection and information security, the data administrator should react immediately. The data administrator must remove the dataset (from the administrator’s website and/or the data portal), contact the organisation's data protection officer and information security experts, and launch information activities following the pre-prepared plan.

It is also important for the data administrator to react quickly to other errors detected in the dataset. The data administrator should rapidly assess how and when the organisation can take the necessary corrective actions and inform users about this following the communication plan. The data administrator can either remove the dataset completely until further notice or update the metadata with information about the identified error and announce when the error will be corrected.

The preparation of a crisis plan should be integrated into the organisation's crisis communication plan, if any. The plan’s preparation process can make use of e.g. the VAHTI cooperation network’s publication Management of information security incidents VAHTI 8/2017. (In Finnish, pdf)

Response to problem situations at Statistics Finland

The primary objective of correcting errors in statistical data shared on online services or in printed publications is to guarantee that correct data is used and ensure that incorrect data is corrected as effectively as possible. Transparent correction procedures also help to maintain user confidence in the producer of the statistics.

If significant errors are detected in a statistical publication or the online service, the publication guideline for the Official Statistics of Finland (OSF)(in Finnish) is followed in the procedure for correcting them. 

Main rule

Information about a material error should be spread at least as widely and visibly as about the original data. However, if the incorrect data has attracted a great deal of publicity, it may be necessary to publish a separate notice about the correction with wider distribution and higher visibility than what was given to the original data.

The information about the error must remain permanently visible, except for in database tables. Significant errors and the dates of their correction should be added to the page describing changes in the statistics.

Publishing and notices about data publication

Significant errors occurring in connection with the publication of a dataset are corrected by publishing a notice about the correction, which is distributed as widely and visibly as the incorrect data. In the online service, the error should be corrected as quickly and visibly as possible. A separate reference to the correction procedure should be added, for example to the page describing changes in statistics.

Support materials on the topic

This section contains support material related to the topics discussed in this step.

Training courses on the data.europa.eu website: