This is the page for the Quality Assurance team. It includes links for QA services, documentation, tests and any information related to the different QA processes in the project.
Weekly tests reports
Tools used for QA tasks and infrastructure.
- apertis-test-cases: Source of automated and manual test cases. Used to generate the test case site and to perform automated tests.
- apertis-test-cases-web: Source used to generate the index page for the test case site.
- Apertis infrastructure dashboard: Tracking of potential packaging issues.
- lqa: It submits the automated tests jobs to LAVA. It also offers a LAVA API that can be used by other scripts.
Robot Framework is a generic test automation framework for acceptance testing and acceptance test-driven development (ATDD). It has easy-to-use tabular test data syntax and it utilizes the keyword-driven testing approach. Its testing capabilities can be extended by test libraries implemented with either Python or Java, and users can create new higher-level keywords from existing ones using the same syntax that is used for creating test cases. It is open source software released under Apache License 2. [Read More]
Test Data Reporting
Testing is a fundamental part of the project, but it is not so useful unless it goes along with an accurate and convenient model to report the results of such a testing. The QA Test Report is an application that has been developed to save and report the test results for the Apertis images. It supports both automated tests results executed by LAVA and manual tests results submitted by a tester. [Read More]
The test cases, both manual and automated, are written in the LAVA test definition file format, which stores the instructions to run the automated tests in YAML files. Git is used as the data storage backend for all the test cases. The current Apertis tests can be found in the Apertis Test Cases repository. The test cases are versioned using Git branches to enable functionality change without breaking tests for older releases. [Read More]
Immutable Rootfs Tests
Testing on a immutable rootfs Tests should be self-contained and not require changes to the rootfs: any change to the rootfs causes the tested system to diverge from the one used in production, reducing the value of testing. Changing the rootfs is also impossible or very cumbersome to do with some deployment systems such as dm-verity or OSTree. Other systems may simply not ship package management systems like apt/dpkg due to size constraints, making package dependencies not viable. [Read More]
LQA is both a tool and API for LAVA quality assurance tasks. It stands for LAVA Quality Assurance and its features go from submitting LAVA jobs, collect tests results, and query most of the LAVA metadata. It is a tool developed by Collabora and it is the standard way to submit test jobs for Apertis images. Installation Fetch the latest code from the git repository and install using python pip as it will handle automatically the required dependencies:: [Read More]
Test Case Guidelines
The following are guidelines to create and edit test cases. Please follow them unless there is a compelling reason to make an exception. Workflow Before developing a new test case: Make sure the test case doesn’t exist by checking the Test Cases list. Make sure nobody is already working on a test case by checking the Phabricator tasks. Determine the main feature to focus test upon and set the test case identifier accordingly. [Read More]