Monitoring & learning¶
It is important to monitor the adoption and use of standards: particularly those with a policy goal. Through monitoring, and reflecting on adoption, there are opportunities to identify ways to improve the standard, it's documentation and associated tools, and to develop interventions that create a thriving ecosystem of publishers and users around a standard.
There are many different aspects to monitoring and learning:
- Pipeline and publishers - tracking how many organisations have expressed interest in the standard, started adopting it, or successfully published.
- Validation - tracking how much of the data produced is valid against the standards schema, or against additional rulesets, and identifying common interoperability issues.
- Coverage - tracking the relative levels of use for particular fields, data elements and codelists, as well as looking at the use of extensions or additional fields in standards that support this.
- Quality and usability - tracking whether data is clear, accurate and usable, and being put into use.
- Community - tracking the size and levels of activity in the community around the standard, identifying whether or not a market is emerging around it, and looking at the range of contributors to standard development.
Components¶
The following components are often used as part of a monitoring strategy.
Patterns¶
The following approaches can form part of a monitoring strategy:
- Quality frameworks - that can judge the 'level' of publication, and incentivise publishers to improve their data.
- Setting targets - for the number of publishers, valid data, or number of community contributions. Measuring against targets can help keep engagement and implementations on track.
- Individual publisher feedback reports - offering an opportunity for regular deep-dive engagement with publishers.
- Tool certification - either self-certified, or with external certification - as a means to engage tool developers, check and learn from the way data is being used.