Five Things We’ve Learned About OpenMRS and Quality Assurance

At OpenMRS, we understand the importance of providing users with a quality product that they can trust. Getting to quality means adopting effective quality assurance processes – and using proven quality assurance tools that our community can easily adopt and practice.  

Through conversations with OpenMRS developers, implementers, and quality assurance professionals, we know that quality assurance is more effective when: 

• the whole team is involved – from business analysts to developers to testers. 
• the QA process is aligned with the software development lifecycle, followed by squads and release teams.
• there are clearly laid out standards and guidelines for the testing process and use of the framework.
• we have clear and well-documented expectations about the testing process and results. 

It’s one thing to know what makes quality assurance effective – and another to put it into action on a routine basis.

What do we know about the quality assurance processes and tools used by the OpenMRS community?

In recent months, our Quality Assurance Team has been exploring ways to improve our quality assurance process and tools, both manual and automated. The team has reviewed existing resources as well as existing platforms, including Cypress and Puppeteer, to evaluate their compatibility with the QA team goal.

Here are five things that the QA Team’s review revealed:

1. Selenium is already set up for testing the OpenMRS Reference Application. Our community has already invested in setting up automated UI test cases using Selenium. At this point in time, there are about over 100 automated UI test cases that have been developed and the results captured via Sauce Labs.
2. OpenMRS core has a robust CI testing in place. Thanks to the efforts of our Infrastructure Team, we have established robust CI testing. We currently have over 4,000 combination automated unit tests and integration tests, with a moderate test coverage of about  60 % with the highest being on the API code.
3. Documented requirements and testing SOPs. Our review coincided with testing for a new Reference Application release. This ended up being a great opportunity to observe how the community currently tests releases of the Reference Application. Clearly documented requirements assists anyone coming on board to understand the specifics of the development work being done for a release, which in turn guides the development of test cases. One of the gaps the team identified was the lack of adequate documentation detailing out the requirements or features that were included in the release. Additionally, guidelines on how testing should be done, what is required, how comprehensive it should, and testing templates are a gap in the community testing process. The establishment of standard procedures will go a long way in directing the community on how best to conduct testing in a manner that is acceptable to the community. 
4. Obsolete resources: There is substantially good documentation that has been developed to guide the QA process and made available to the community on the OpenMRS Wiki.  However, most are out of date. In addition to updating QA process documentation, this highlights the need to have a well-established process for continuously updating documentation to ensure they are usable.
5. Planning for testing (timelines and limited resources): The review also looked at our plan for testing and asked: How much time will we plan to spend on testing? What resources can we dedicate to testing? Our review found that the testing timelines being allocated were unclear and inadequate, due to various reasons. These included spillover of development time or delayed turn around time in regards to feedback & issues found. Multiple reasons can explain this, but a key factor noted is limited resources, a natural challenge in any volunteer-driven environment. To address this issue, the community can adopt approaches that will increase the traffic of volunteers who take on QA assignments or tasks – as well as set stringent timeframes that can guide the quality assurance process for any product.

These results shaped the QA Team’s next steps: to look into quality assurance best practices and tools that can enhance what we already have in place as well as address what isn’t working well and any identified gaps. We’ll tell you more about what the QA team has been looking at in our next blog post. In the meantime, you can find out more about the QA Team and how to join us on the QA Team’s Wiki page.