Monitoring & learning¶
It is important to monitor the adoption and use of standards: particularly those with a policy goal. Through monitoring, and reflecting on adoption, there are opportunities to identify ways to improve the standard, it’s documentation and associated tools, and to develop interventions that create a thriving ecosystem of publishers and users around a standard.
There are many different aspects to monitoring and learning:
Pipeline and publishers - tracking how many organisations have expressed interest in the standard, started adopting it, or successfully published.
Validation - tracking how much of the data produced is valid against the standard’s schema, or against additional rulesets, and identifying common interoperability issues.
Coverage - tracking the relative levels of use for particular fields, data elements and codelists, as well as looking at the use of extensions or additional fields in standards that support this.
Quality and usability - tracking whether data is clear, accurate and usable, and being put into use.
Community - tracking the size and levels of activity in the community around the standard, identifying whether a market is emerging around it, and looking at the range of contributors to standard development.
Components¶
The following components are often used as part of a monitoring strategy.
Patterns¶
The following approaches can form part of a monitoring strategy:
Quality frameworks - that can judge the ‘level’ of publication, and incentivise publishers to improve their data.
Setting targets - for the number of publishers, valid data, or number of community contributions. Measuring against targets can help keep engagement and implementations on track.
Individual publisher feedback reports - offering an opportunity for regular deep-dive engagement with publishers.
Tool certification - either self-certified, or with external certification - as a means to engage tool developers, check and learn from the way data is being used.