Table of Contents:
QA Report App is the Apertis tool for reporting and viewing the results of both manual and automated (via LAVA) QA testing.
Primary Configuration File
QA Report App’s configuration is handled via the file secrets/config.yaml
.
General Options
Resource Links
QA Report App needs to be able to generate links to the test cases and images:
|
|
test-cases-url
and bridge-url
should both be the same value: the URL where
QA Report App is running at. (There are two separate options for historical
reasons; this will be cleaned up in the future.) image-root
should point to
the web location where images downloads are available at.
Debug Logging
Debug logging from the service can be enabled via:
|
|
Defining Supported Platforms and Deployments
QA Report App is designed to take in test case results from a given platform
that the test cases were run on, such as fixedfunction-armhf-uboot-public
or
minimal-armhf-uboot-internal
, along with a deployment describing how the OS
was installed onto the platform (e.g. apt
, ostree
, lxc
, nfs
).
The support platforms and deployments can be configured via the images
section
of the configuration file:
|
|
Defining Variants and Architectures
Image platforms as described contain both variants (different versions of the same distribution) and architectures. For example, given the following platforms:
- fixedfunction-armhf-uboot-public
- tiny-lxc-amd64-amd64-uefi-internal
Each of these starts with a variant (fixedfunction
, tiny-lxc
) that is
immediately followed by the architecture (armhf
, amd64
). The supported
variants and architectures should be defined at the top of the images
section
of the config file:
|
|
Defining Deployments
|
|
Deployments that can be selected by the user when manually submitting test
results go in the user-selectable
section, with each deployment name mapped
to a human-readable description. For instance, we may have:
|
|
In this case, ostree
, is the deployment name, and OSTree
is the
human-readable description that will be shown in the UI.
If, instead, a deployment should not appear for manual test result submission,
but instead only be available for automatically reported LAVA results, the
deployment names should be placed in the lava-only
section, e.g.:
|
|
No human-readable name is required, since these are never shown explicitly in the UI.
Defining Platforms
|
|
Image platforms, such as fixedfunction-armhf-uboot-public
, are defined in
groups called “platform groups”. Each group contains:
- A list of platforms defined in this group.
- A list of deployments supported by the given platforms.
- (Optional) A list of release/version ranges that support the given platforms.
The purpose of placing platforms into these groups is to be able to define multiple platforms that support the same deployments same conditions in one place, reducing repition.
Example of some basic platform groups:
|
|
Platform Support Ranges
|
|
The supported-by
key defines a list of ranges of releases that support the
platforms in the current group.
The individual ranges inside have two keys, at least one of which must be given:
first
: the first / oldest version that supports this platform. If not given, it is assumed that the platform has been supported since the very first release.last
defines the last / newest version that supports this platform. If not given, it is assumed that the platform is still supported by the very latest release.
Each of these endpoints of the range in turn contains release
and version
,
to set the release and version of the given endpoint. The version is optional
and, if not given, all versions in the release are counted. For example, this
platform group is valid for all releases from v2021 to v2022 version 20220624:
|
|
The end ranges can be defined to be exclusive instead of inclusive, via:
|
|
This would normally mean “every release since and including v2021 is
supported”, but setting inclusive: false
changes it to mean “every release
since and not including v2021”.
Multiple support ranges can be given:
|
|
Some more examples:
|
|
Changing the Test Cases Source
Test cases are, by default, pulled from the Apertis test cases Git
repository. The individual
YAML files describing test cases are then taken from every branch starting with
apertis/
, with the text afterwards being treated as the release (e.g. given
apertis/v2020
, the release would be v2020
). If you want to change the URL
and the branch prefix (the apertis/
), set test-cases-repository
:
|
|
where:
URL
is a URL pointing to a Git repository containing the test cases.BRANCH-PREFIX
is the prefix
Note that the portion of the branch name after the branch prefix is treated as the release/suite.
Bug Tracker Integration
Phabricator
|
|
This sets up integration with Phabricator, for automated issue creation.
/PATH/TO/ARCRC
should be the path to an
Arcanist
global configuration file. One of these can be generated at ~/.arcrc
by
running:
arc set-config default https://PHABRICATOR-URL/
arc install-certificate
The phabricator-tasks
section controls the behavior of tasks that are
automatically created by QA Report App from LAVA failures.
space
is the Phabricator space that should have visibility into the created
tickets.
filter_tag
is a Phabricator tag should identify all tasks automatically filed;
new tasks filed will have this tag assigned, and existing ones will be searched
using this tag. tags
defines any additional tags that should be added when
opening a new ticket (but these are not used for any other purpose).
GitLab
|
|
This sets up integration with GitLab issues, for automated issue creation.
/PATH/TO/GITLAB/CONFIG
should be the path to a python-gitlab configuration
file,
and GITLAB/PROJECT
should be the GitLab project where issues are stored (e.g.
infrastructure/apertis-issues
).
The gitlab-issues
section controls the behavior of issues that are
automatically created by QA Report App from LAVA failures. filter_label
is a
GitLab label that should identify all issues automatically filed; new issues
filed will have this label assigned, and existing ones will be searched using
this label. labels
defines any additional labels that should be added when
opening a new issue (but these are not used for any other purpose).
Note that gitlab-url
is additionally used for configuring OAuth logins and
thus may be set even if GitLab issues integration is not set up.
Environment for docker-compose
In addition to the standalone configuration file, a separate file .env
,
providing environment variables to docker-compose, can be created in the
qa-report-app directory. These variables are used to configure hosting aspects,
rather than the behavior of the application itself.
The following variables are available to be set, as well as their default values:
# The port the application will be exposed on.
QA_REPORT_APP_PORT=28080
# The user ID and group ID the application and database will run as. In
# particular, this results in the database files also being owned by this user
# and group.
RUN_USER=1000:1000
LAVA Authentication
QA Report App uses shared secrets to authenticate legitimate LAVA callback submissions. In order to set up a new shared secret, you need to:
- Create a new token in the LAVA UI (
API
→Authentication Tokens
) for the user submitting thenotify
callback, using the name set in thenotify
→callback
→token
property of the LAVA job definition YAML. - Place the token in the
./secrets/token
file in QA Report App’s working directory.