Table of Contents:

QA Report App is the Apertis tool for reporting and viewing the results of both manual and automated (via LAVA) QA testing.

Primary Configuration File

QA Report App’s configuration is handled via the file secrets/config.yaml.

General Options

QA Report App needs to be able to generate links to the test cases and images:

1
2

app-url: https://qa.apertis.org
image-root: https://images.apertis.org

app-url should be the URL where QA Report App is running at, andimage-root should point to the web location where images downloads are available at.

Debug Logging

Debug logging from the service can be enabled via:

1

debug: true

LAVA Authentication

QA Report App uses shared secrets to authenticate legitimate LAVA callback submissions, which can be set in the config file:

1
2
3
4

lava-callback-tokens:
  - TOKEN-1
  - TOKEN-2
  - ...

Additionally, they can be set in the environment.

In order to obtain a new token, you need to create a new token in the LAVA UI (API → Authentication Tokens) for the user submitting the notify callback. The token’s name should match the name set in the notify→callback→token property of the LAVA job definition YAML.

Open ID Connect user login

Users can login to submit manual reports. To activate authentication, the following configuration options must be set:

1
2
3
4
5

openid:
  client-id: CLIENT-ID
  client-secret: CLIENT-SECRET
  well-known-url: WELL-KNOWN-URL
  always-require-login: BOOL  # (optional)

Both client-id and client-secret are created by the OpenID Connect identity provider. It should also provide the well-known-url.

E.g.: For using a Gitlab instance as OpenID Connect identity provider, an application must be added on gitlab, with the openid scope and the https://<qa-report-app-server>/openid_callback callback URL. For Gitlab, the well-known-url value is https://<gitlab-instance>/.well-known/openid-configuration.

always-require-login is an optional boolean value. When set to true, the website will require users to be logged in to access any page. This is set to false by default.

Defining Supported Platforms and Deployments

QA Report App is designed to take in test case results from a given platform that the test cases were run on, such as fixedfunction-armhf-uboot-public or minimal-armhf-uboot-internal, along with a deployment describing how the OS was installed onto the platform (e.g. apt, ostree, lxc, nfs).

The support platforms and deployments can be configured via the images section of the configuration file:

1
2
3
4
5
6
7
8
9

images:
  architectures:
    ...
  variants:
    ...
  deployments:
    ...
  platform-groups:
    ...

Defining Variants and Architectures

Image platforms as described contain both variants (different versions of the same distribution) and architectures. For example, given the following platforms:

  • fixedfunction-armhf-uboot-public
  • tiny-lxc-amd64-amd64-uefi-internal

Each of these starts with a variant (fixedfunction, tiny-lxc) that is immediately followed by the architecture (armhf, amd64). The supported variants and architectures should be defined at the top of the images section of the config file:

1
2
3
4
5
6
7

images:
  architectures:
    - amd64
    - arm64
  variants:
    - fixedfunction
    - tiny-lxc

Defining Deployments

1
2
3
4
5
6
7

images:
  deployments:
    user-selectable:
      NAME-1: DESCRIPTION 1
    lava-only:
      - NAME-2
      - NAME-3

Deployments that can be selected by the user when manually submitting test results go in the user-selectable section, with each deployment name mapped to a human-readable description. For instance, we may have:

1
2

user-selectable:
  ostree: OSTree

In this case, ostree, is the deployment name, and OSTree is the human-readable description that will be shown in the UI.

If, instead, a deployment should not appear for manual test result submission, but instead only be available for automatically reported LAVA results, the deployment names should be placed in the lava-only section, e.g.:

1
2
3

lava-only:
  - lxc
  - nfs

No human-readable name is required, since these are never shown explicitly in the UI.

Defining Platforms

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

images:
  platform-groups:
    - platforms:
        PLATFORM-1: DESCRIPTION 1
        PLATFORM-2: DESCRIPTION 2
      deployments: [DEPLOYMENT-1, DEPLOYMENT-2]
      supported-by:
        - first:
            release: FIRST-RELEASE
            version: FIRST-VERSION
          last:
            release: LAST-RELEASE
            version: LAST-VERSION
    - ...

Image platforms, such as fixedfunction-armhf-uboot-public, are defined in groups called “platform groups”. Each group contains:

  • A list of platforms defined in this group.
  • A list of deployments supported by the given platforms.
  • (Optional) A list of release/version ranges that support the given platforms.

The purpose of placing platforms into these groups is to be able to define multiple platforms that support the same deployments same conditions in one place, reducing repition.

Example of some basic platform groups:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

platform-groups:
  - platforms:
      tiny-lxc-armhf-uboot-public: Tiny LXC ARM
      tiny-lxc-arm64-uboot-public: Tiny LXC ARM64
      tiny-lxc-amd64-uefi-public: Tiny LXC AMD64
    deployments: [lxc]

  - platforms:
      nfsroot-armhf-uboot-public: Nfsroot ARM
      nfsroot-arm64-uboot-public: Nfsroot ARM64
      nfsroot-amd64-uefi-public: Nfsroot AMD64
    deployments: [nfs]

Platform Support Ranges

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

supported-by:
  - first:
      release: FIRST-RELEASE
      version: FIRST-VERSION
    last:
      release: LAST-RELEASE
      version: LAST-VERSION
  - first:
      release: FIRST-RELEASE
      version: FIRST-VERSION
    last:
      release: LAST-RELEASE
      version: LAST-VERSION

The supported-by key defines a list of ranges of releases that support the platforms in the current group.

The individual ranges inside have two keys, at least one of which must be given:

  • first: the first / oldest version that supports this platform. If not given, it is assumed that the platform has been supported since the very first release.
  • last defines the last / newest version that supports this platform. If not given, it is assumed that the platform is still supported by the very latest release.

Each of these endpoints of the range in turn contains release and version, to set the release and version of the given endpoint. The version is optional and, if not given, all versions in the release are counted. For example, this platform group is valid for all releases from v2021 to v2022 version 20220624:

1
2
3
4
5
6

supported-by:
  - first:
      release: v2021
    last:
      release: v2022
      version: '20220624'

The end ranges can be defined to be exclusive instead of inclusive, via:

1
2
3
4

supported-by:
  - first:
      release: v2021
      inclusive: false

This would normally mean “every release since and including v2021 is supported”, but setting inclusive: false changes it to mean “every release since and not including v2021”.

Multiple support ranges can be given:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

supported-by:
  # Supported from v2019 to v2021 and from v2022dev0 to v2022dev2.
  - first:
      release: v2019
    last:
      release: v2021
  - first:
      release: v2022dev0
    last:
      release: v2022dev2

Some more examples:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# Supported from every release since, and including, v2022dev3, 20210910:
supported-by:
  - first:
      release: v2022dev3
      version: '20210910'

# Supported from every version *after* v2021, 20210910 up until, and including,
# v2022.
supported-by:
  - first:
      release: v2021
      version: '20210910'
      inclusive: false
    last:
      release: v2022

Changing the Test Cases Source

Test cases are, by default, pulled from the Apertis test cases Git repository. The individual YAML files describing test cases are then taken from every branch starting with apertis/, with the text afterwards being treated as the release (e.g. given apertis/v2020, the release would be v2020). If you want to change the URL and the branch prefix (the apertis/), set test-cases-repository:

1
2
3

test-cases-repository:
  url: THE-URL
  branch-prefix: THE-BRANCH-PREFIX

where:

  • URL is a URL pointing to a Git repository containing the test cases.
  • BRANCH-PREFIX is the prefix

Note that the portion of the branch name after the branch prefix is treated as the release/suite.

Bug Tracker Integration

Phabricator

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

bug-tracking: phabricator
phabricator-url: https://PHRABRICATOR-URL/
phabricator-token: PHABRICATOR-TOKEN
phabricator-tasks:
  space: S2
  filter-tag: PRIMARY-TAG
  tags:
    - ADDITIONAL-TAG-1
    - ADDITIONAL-TAG-2
    - ...

This sets up integration with Phabricator, for automated issue creation. PHABRICATOR-TOKEN is a Phabricator Conduit API token, which can be obtained via the following steps:

  • Click your profile picture in Phabricator on the top-right -> Settings.
  • Select Conduit API Tokens in the left sidebar.
  • Click the Generate Token button.

The phabricator-tasks section controls the behavior of tasks that are automatically created by QA Report App from LAVA failures.

space is the Phabricator space that should have visibility into the created tickets.

filter-tag is a Phabricator tag should identify all tasks automatically filed; new tasks filed will have this tag assigned, and existing ones will be searched using this tag. tags defines any additional tags that should be added when opening a new ticket (but these are not used for any other purpose).

GitLab

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

bug-tracking: gitlab
gitlab-url: https://GITLAB-URL/
gitlab-token: GITLAB-TOKEN
gitlab-project: GITLAB/PROJECT
gitlab-issues:
  filter-label: PRIMARY-LABEL
  labels:
    - ADDITIONAL-LABEL-1
    - ADDITIONAL-LABEL-2
    - ...

This sets up integration with GitLab issues, for automated issue creation. GITLAB-TOKEN should be a [GitLab access token] https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#create-a-personal-access-token) (use the api scope), and GITLAB/PROJECT should be the GitLab project where issues are stored (e.g. infrastructure/apertis-issues).

The gitlab-issues section controls the behavior of issues that are automatically created by QA Report App from LAVA failures. filter-label is a GitLab label that should identify all issues automatically filed; new issues filed will have this label assigned, and existing ones will be searched using this label. labels defines any additional labels that should be added when opening a new issue (but these are not used for any other purpose).

Note that gitlab-url is additionally used for configuring OAuth logins and thus may be set even if GitLab issues integration is not set up.

Environment for docker-compose

In addition to the standalone configuration file, a separate file .env, providing environment variables to docker-compose, can be created in the qa-report-app directory. These variables are used to configure hosting aspects, rather than the behavior of the application itself.

The following variables are available to be set, as well as their default values:

1
2
3
4
5
6
7
8

# The port the application will be exposed on.
QA_REPORT_APP_PORT=28080
# The user ID and group ID the application and database will run as. In
# particular, this results in the database files also being owned by this user
# and group.
RUN_USER=1000:1000
# If lava-callback-tokens is not set in the config file, it should be set here.
LAVA_CALLBACK_TOKENS=["TOKEN-1", "TOKEN-2"]