This tutorial explains how to submit a personal LAVA job for an Apertis test, which is very useful during development, either to debug a test, or to check that everything is working as expected with the test before its final integration.
Running a personal test basically consists in adding the LAVA test definition file in a personal repository and submitting a LAVA job that will fetch this file from there to execute the test.
lqa is the tool used to submit the daily automated LAVA jobs for the
Apertis images, so it will also be used to submit the Apertis job in
this tutorial. Before running the instructions in this tutorial, please
check the following link to install and configure
lqa and create a
personal bundle stream: LQA
This tutorial assumes that all the packages dependencies required by the personal test are already available in the Apertis packages repositories, if that is not the case, any missing package dependency should ideally be first integrated into the packages repositories.
Nevertheless, there are ways to fetch external packages from LAVA jobs. The two main ways are:
Add a new test definition to include the package repository where the
package resides during the test execution (check the file
apertis-tests/misc/add-repo.yaml for an example).
Fetch the package directly with some command line utility from the test definition itself and install it manually during the test execution.
The test definition is the YAML file containing the steps to execute the test in LAVA. This file can be fetched by LAVA from different kind of locations, in the Apertis project these files are available from git repositories.
So, once you write the test definition file, you need to make available this file from your own personal git repository from which LAVA can fetch it.
The job definition is a YAML file that tells LAVA how to execute the test job. This file will point LAVA to fetch the test definition from the personal git repository.
repository comes with a set of templates which simplify the job of
creating the job definition. The
common-qemu-boot-tpl.yaml templates define device-specific actions to
boot an image on a real device or in QEMU. The job definition using
these templates thus can be dramatically simplified.
The following is a YAML job template using the abovementioned common code to run a simple test:
Each test is listed as a separate item under
definitions. The test
names should be unique. It is also possible to add another
section, if the separation makes the test set more readable to use. It
makes no difference at all on how the tests are run or the results are
Save the above template as a file named
Since most of the fields in the template should work just fine by default, to tweak values from the common code it is required to edit them directly. You might need to change the timeouts values if the test can take a long time.
Choose Apertis profile (image type)
apertis-tests repository contains most of the Apertis tests files
and all the required files to execute the automated tests in LAVA.
There are several types of Apertis images, and there exist a profile for
each of them that can be used by
lqa to execute that specific image
apertis-tests repository to get the Apertis profile file:
$ git clone https://git.apertis.org/git/apertis-tests.git
Check the file
templates/profiles.yaml to select the profile you want
to run, or just run the following the command to get all the profiles
names from that file directly:
$ grep name apertis-tests/templates/profiles.yaml
Each profile specifies an Apertis image type, containing all the specific values for it.
The next step is just to submit the job to LAVA using the selected profile.
Submit LAVA job
Triggering a LAVA job is just submitting the YAML job file with the
required options using the
As an example, to trigger a job to execute the above example template
my-test-tpl.yaml on QEMU for the Apertis Target AMD64 image
with date 20171125.0 the following command can be used:
$ lqa submit -g apertis-tests/templates/profiles.yaml \ -p apertis-17.12-daily-qemu-target \ -t image_date:20161025.0 my-test-tpl.yaml
-g option points to the profile file from the apertis-tests
repository, and the
-p option selects the profile we want to use.
-t option lets users set or override variables used to parametrise
You need to be in a directory containing
apertis-tests to be able to
execute the command above.
lqa submit has many options to submit jobs, for example, the
combination of the
-v options are very useful to check for
the final generated JSON file before submitting to the server. Check the
complete options with:
lqa submit --help.
Once the command is executed, there should be a LAVA job running for this personal test at: https://lava.collabora.co.uk/
If a job template file is not specified from the command line, it will
trigger lava jobs for all the templates found in the
file for the specific profile.
Details on test job templates
The boot process for non-emulated devices and for QEMU differs, and due to the amount of differences the definitions are split into two separate template files.
common-boot-tpl.yaml contains definition needed to boot Apertis images
on real (non-emulated devices). Since they cannot boot images directly,
the boot process is separated in two stages: flashing the image onto a
device from which the board can boot, and booting into the image and
The first stage boots over NFS into a (currently) Debian stretch image
with a few extra tools needed to flash the image, downloads the image
using HTTP, flashes it and reboots. This stage is defined using
namespace: flash in the job YAML file. In most cases you won't need to
edit bits related to this stage. The second stage is common for both
non-emulated devices and QEMU, despite them having certain differences.
It is used to boot the image itself, prepare the LAVA test runner and
run tests. This stage is defined using
namespace: system. You
normally don't need to edit this stage either. The exception to this
is when you need to load an image from a different source than
Image URLs are defined in the
deploy action. For
common-boot-tpl.yaml, it is necessary to specify URLs to both image
itself and its bmap file, which is used to speed up the flashing
process and avoid unnecessary excessive device wear. For
common-qemu-boot-tpl.yaml, only the URL to the image itself is needed,
as QEMU doesn't support bmap files yet.
The second stage always performs two tests:
basically checks that the system actually works, and
isn't actually a test, and is used to add repositories to
/etc/apt/sources.list on certain devices.
These instructions are written to submit LAVA jobs for ONLY PUBLIC
Apertis images. If you need to submit a LAVA job for a private image,
there are few things that need to be taken into consideration and few
changes need to be made to these instructions:
group visibility should be selected for your jobs.
If you really need to submit a private job, please contact the Collabora QA team.