This guide covers the expectations and processes for Apertis developers wishing to make contributions to the Apertis project and the wider open source ecosystem. These policies should be followed by all developers, including core and third party contributors.

Suitability of contributions

Like most open source projects, Apertis requires contributions are submitted via a process (which in the case of Apertis is defined below) to ensure that Apertis continues to meet it's design goals and remain suitable for it's community of users. In addition to design and technical implementation details, the suitability of contributions will be checked to meet requirements in areas such as coding conventions and licensing.

Upstream First Policy

Apertis is a fully open source GNU/Linux distribution that carries a lot of components for which it is not the upstream. The goal of upstream first is to minimize the amount of deviation and fragmentation between Apertis components and their upstreams.

Deviation tends to duplicate work and adds a burden on the Apertis developers when it comes to testing and updating to newer versions of upstream components. Also, as the success of Apertis relies on the success of open source in general to accommodate new use cases, it is actively harmful for Apertis to not do its part in moving the state of the art forward.

It is the intention of Apertis to utilize existing open source projects to provide the functionality required, where suitable solutions are available, over the creation of home grown solutions that would fragment the GNU/Linux ecosystem further.

This policy should be taken into consideration when submitting contributions to Apertis.

Upstream Early, Upstream Often

One mantra that can be often heard in Open Source communites is “upstream early, upstream often”. The approach that this espouses is to breakdown large changes into smaller chunks, attempting to upstream a minimal implementation before implementing the full breath of planned features.

Each open source community tends to be comprised of many developers, which share some overlap between their goals, but may have very different focuses. It is likely that other developers contributing to the project may have ideas about how the features that you are planning may be better implemented, for example to enable a broader set of use cases to utilise the feature. Submitting an early minimal implementation allows the general approach to be assessed, opinions to be sought and a concensus reached regarding the implementation. As it is likely that some changes will be required, a minimal implementation minimizes the effort required to take feedback into account.

Taking this approach a step further, it can often be instructive to share your intention to implement larger features before starting. Such a conversation might be started by sending an email to the projects devel mailing list saying:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
Hi,

I'm attempting to use <project> to <task> for my project.

I'm thinking about doing <brief technical overview> to enable this usecase.

I'm open to suggestions should there be a better way to solve this.

Thanks,

<developer>

This enables other experienced developers the chance to suggest approaches that may prove to be the most efficient, saving effort in implementation and later in review, or may point to missed existing functionality that can be used to solve a given need without needing substancial development effort.

Extending Apertis

Adding components to Apertis

Apertis welcomes requests for new components to be added to the distribution and can act as a host for projects where required, however the open source focus of Apertis should be kept in mind and any proposed contributions need to both comply with Apertis policies and present a compelling argument for inclusion.

Additional components can be categorised into 3 main groups:

  • Existing upstream component available in Debian stable (with suitable version)
  • Existing upstream component, not available in debian stable
  • New component on gitlab.apertis.org

There is a maintenance effort associated with any components added to Apertis, as any components added will need to be maintained within the Apertis ecosystem. The effort required to maintain these different categories of components are very different. Prepackaged Debian components require a lot less maintenance effort than packaging other existing upstream components. Developing a new component on gitlab.apertis.org requires both the development and packaging/maintenance to be carried out within Apertis, significantly raising the effort required.

When looking for ways to fullfil a requirement there are a number of factors that will increase the probability of a solution being acceptable to Apertis.

  • Component already included in Debian stable: As Apertis is based on Debian and already has processes in place to pull updates from this source. The cost of inclusion is dramatically lower than maintaining packages drawn from other sources, as a lot of the required effort to maintain the package is being carried out within the Debian ecosystem.
  • Proven actively maintained codebase: Poorly maintained codebases present a risk to Apertis, increasing the chance that serious bugs or security holes will go unnoticed. Picking a solution that has an active user base, a developer community making frequent updates and/or is a mature codebase that has undergone significant “in the field” testing makes the solution more attractive for inclusion in Apertis. It is understood that, whilst extensive, the Debian repositories are not all encompassing, if proposing an existing open source component that isn't currently provided by Debian, being able to show that it is actively maintained will be important.
  • Best solution: In general, there exists more open source solutions than there exists problems. To be in with a good chance of having a component included in Apertis it will be required to explain why the chosen solution represents the best option for Apertis. What is “best” is often nuanced and will be affected by a number of factors, including integration/overlap with existing components and the size/number of dependencies it has (especially if they aren't currently in Apertis). It may be that whilst a number of existing solutions exist, none of them are a good fit for Apertis. This may suggest a new component is the best solution, though adapting/extending one of the existing solutions should also be considered.

The Apertis distribution is supported by it's members. As previously mentioned, in order to ensure that Apertis remains viable and correctly focused, it is important that any additions to the main Apertis projects are justified and can be shown to fill a specific and real use case. Maintaining the packaging, updating the codebases of which Apertis is comprised and performing testing on supported platforms is a large part of the effort needed to provide Apertis. As a result, it will be necessary to either be able to provide a commitment to support any packages proposed for inclusion in the main Apertis projects or gain such a commitment from an existing member.

The Apertis development team commit to maintaining the packages included in the references images. Packages may be added to the main package repositories but not form part of the reference images. Such packages will be maintained on a best effort basis, that is as long as the effort remains reasonable the Apertis team will attempt to keep the package in a buildable state, however runtime testing will not be performed. Should the package fail to build or runtime issues are reported and significant effort be required to modify the package the original or subsequent users of the package may be approached to help resource fixing the package. Ultimately the package may be removed if a solution can not be found. Likewise, should a different common solution for Apertis be chosen at a later date, the package may be deprecated and subsequently removed.

Proposals for inclusion of new components are expected to be made in the form of a written proposal. Such a proposal should contain the following information:

  • Description of the problem which is being addressed
  • Why the functionality provided by the proposed component is useful to Apertis and it's audience
  • A review of the possible solutions and any advantages and disadvantages that have been identified with them
  • Why the proposed solution is thought to present the best way forward, noting the points made above where relevant
  • Whether any resources are to be made available to help maintain the component.

An alternative to adding packages to the main Apertis project is to apply to have a dedicated project area, where code specific to a given project can be stored. Such an area can be useful for providing components that are highly specific to a given project and/or as a staging area for modifications to core packages that might later get folded back into the main area, either by changes being submitted to the relevant Apertis component or after changes have been upstreamed to the components main project. A dedicated area will allow a project group to iterate on key components more rapidly as the changes made do not need to work across the various supported hardware platforms. It must be noted that whilst a dedicated project area would allow some requirements with regard to platform support to be ignored, packages in such areas would still be required to comply with other Apertis rules such as open source licensing. It should be expected that the Apertis developers will take a very hands off approach to the maintenance and testing of packages in such areas. If packages in such areas require work, the project maintainers will be contacted. The Apertis maintainers may at their discresion help with minor maintenance tasks should a package be of interest to the Apertis project. Packages that become unmaintained may be removed.

Requests for dedicated project areas are also expected to be made in a form of a written proposal. Such a proposal should contain the following information:

  • Description of the project requiring a dedicated project area
  • Preferred name to be used to refer to the project
  • Expected use of the dedicated area
  • Expected lifetime of the project area
  • Contact details of project maintainers

Such submissions should be made via the devel mailing list.

The submission should be discussed on the mailing list and must be agreed with the Apertis stakeholders.

Extending existing components

Apertis carries a number of packages that have been modified compared to their upstream versions. It is fairly typical for distributions to need to make minor modifications to upstream sources to tailor them to the distribution, Apertis is not different in this regard.

Whilst Apertis does accept changes to existing components, it needs to be acknowledged that this increases the effort required to maintain the package in question. It may be requested that an attempt be made to upstream the changes, in line with the upstream first policy, either to the packages upstream or Debian. More guidance is provided in the upstreaming documentation.

Adding designs to Apertis

Another way to contribute to Apertis is with design documents. A design document contains the description of all relevant aspects of a feature or of a requirement. The current design documents can be found in the Concepts Designs section. These documents cover topics that have been researched but not necessarily implemented. They should provide a good understanding of the impact of the technology that forms the basis of the concept, what it is, how it works, what are the threat models, the required infrastructure, how it would be integrated with Apertis and anything else that is deemed relevant.

Such designs should be updated when implemented to explictly cover the final implementation and moved to a suitable section of the site, typically the Architecture or Guides section.

Project-wide impact is the metric used to decide if a contribution will be handled as a component or as a design. If the impact of the contribution on the Apertis project goes beyond the additional maintenance effort, it is likely to require a design document before the component contribution.

As an example we will consider a proposal to provide tools and workflows for process automation by including the Robot Framework in the Apertis Universe. The Robot Framework is a generic open source automation framework that can be used for automation of tests and processes. Robot Framework is released under Apache License 2.0. However we do not expect to ship Robot Framework components on Apertis target images.

The first important consideration is the state-of-the-art for addressing the goals of the design. In our example the Robot Framework is preferred due it's maturity, unique and simple to use descriptive language, and it's active development community. However a strong argument in favor of the Robot Framework is it's user base. Adding the Robot Framework to the Apertis Universe is expected to bring Robot Framework users to Apertis.

The next important consideration are how the design is expected to work and the potential impact on Apertis. The Robot framework has a layered architecture. The top layer is the simple, powerful, and extensible keyword-driven descriptive language for testing and automation. This language resembles a natural language, is quick to develop, is easy to reuse, and is easy to extend. On the bottom layer of the architecture is the item to be tested, or the process to be automated.

The middle layer is what makes the Robot Framework extensible: libraries. A library, in Robot Framework terminology, extends the Robot Framework language with new keywords, and provides the implementation for these new keywords. Each Robot Framework library acts as glue between the high level language and low level details of the item being tested, or of the environment in which the item to be tested is present.

Adding the Robot Framework to the Apertis Universe has potential to impact:

  1. Development workflow: Apertis encourages the use of continuous integration and the use of shared infrastructure resources instead of resources that are private to specific developers.
  2. Testing Apertis images: Apertis encourages the use of environments that are as close as possible to production environments, meaning that ideally, the Apertis images under test are not instrumented for testing, and are only minimally modified.
  3. Testing infrastructure: Apertis uses LAVA for deployment of operating system and software in hardware, and for automated testing. The two main constraints are LAVA being asynchronous and non-interactive. While both developers and CI pipelines can submit jobs to LAVA, they cannot interact with a job while it is running. The LAVA workflow is: submit a job, wait for the job to be selected for execution, wait for the job to complete execution, and download test results.

Addressing the benefits of the new design proposal is also important. As mentioned, adding tools and workflows for process automation with the Robot Framework will extend the Apertis projects and we expect to attract more users by doing so. Adding real-world use cases can illustrate the value with a good level of details.

The proposal should also describe how to address the integration with Apertis taking into account the constraints of the Apertis development workflow, of testing Apertis images, and of the Apertis testing infrastructure.

The design proposal can also include a high level description of the estimated work. For example, adding Robot Framework to Apertis will involve developing and/or modifying Robot Framework libraries; and developing a run-time compatibility layer for LAVA to keep testing environments as close as possible to production environments, and to adapt the execution of Robot Framework tests to suit the LAVA constraints.

And finally it could contain a high level implementation plan. In our example, one possible way to integrate Robot Framework is to adopt it in stages:

  1. Add Robot Framework to the Apertis SDK to enable developers to use the Robot Framework locally
  2. Robot Framework Integration development: Adapt libraries and create the run-time compatibility layer for LAVA
  3. Deployment on the Apertis infrastructure

This section describes general topics, but it may not be complete for all designs. Regarding the level of details the design document should be complete enough to describe the design and surrounding problems to developers and project managers, but it is not necessary to describe implementation details.

As a rule of thumb start with a lean design document and submit it for review as early as possible. You can send a new design for review to the same channels used for a component contribution.

Development Process

The process for contributing code to Apertis is the same as for many other open source projects: check out the code, make changes locally, then submit them for review and merging. Apertis uses its GitLab instance as a code repository and review tool.

This guide assumes good working knowledge of git (see Version control for specific guidelines) and an account with commit rights on GitLab (see Getting commit rights for guidance).

Fork the repository to private namespace

Before commencing any work, the developer should fork the repository into their namespace

Clone

Clone the forked repository

  • Check out the git repository:
1
git clone https://gitlab.apertis.org/[group name]/[repository name].git
1
git commit -i -s
  • Every commit must have an appropriate Signed-off-by: tag in the commit message.
  • Add a Fixes: APERTIS-<task_id> tag for each task in the proposed commit messages (as explained in the section “Automatically closing tasks” below or in the envelope message of the merge request itself) in order to link the merge request to one or more tasks in Phabricator.
  • Note: The tag will ensure that Phabricator tasks are kept up-to-date with regard to the status of related merge requests, through the creation of a new comment with the link to the merge request every time a merge request is created/updated/merged. This syntax has been chosen for the tag because it is already supported by gitlab.

Merge Request

Review

  • Notify maintainers on the devel mailing list when submitting or updating a merge request.
  • The repository maintainer or another developer will review the merge request.
  • Reviews should happen on GitLab.
  • Reviewers can propose concrete code snippets that the submitter can decide to integrate in their patch series or not:
    • Reviewers can use code blocks in comments, and the submitter can copy and paste them in their patches when updating the merge request
    • Reviewers can edit the submitted branch with the GitLab Web IDE or checking it out manually, stacking new commits on top of the submitted patches: it's up to the submitter to cherry-pick or reject the suggested changes and squash them appropriately with the original commits before the request can be merged
    • If submitters do not agree with the suggested snippets they need to explain why they are rejecting the proposed changes
  • If changes are needed, a force-push with the changes is required to the submitted branch to update the commits that are part of the merge request (always remember to use --force-with-lease instead of --force).
  • Questions, request to clarify and any other kind of discussion should be handled via comments on the merge request.
  • Repeat these review steps until all the changes are accepted.

Merge

  • Merge request must have no thumbs down to be merged and all the discussions should be resolved.
  • Reviewers that approve the general approach but still want to discuss some details can add a thumb up, so other reviewers know that once all the discussions are resolved the merge request can be approved
  • Once all comments have been handled, the reviewer hits the merge button or otherwise pushes the submitted commits to the appropriate branch.

Extra Actions

  • If other actions need to be manually taken when commits are landed the developer accepting and merging the changes is responsible for ensuring that those actions are taken, either by doing it themselves or by asking someone else to do that.
  • If the merged commits need to be backported to other branches through cherry-pick or merges, those should go through merge requests as well: however, only changes related to the backport process are allowed, unrelated changes should be applied with a separate merge request, possibly landing in master first.

Release Commits

The process for releasing a package is documented in the packaging guide.

Other important bits

Sign-offs

Like the git project and the Linux kernel, Apertis requires all contributions to be signed off by someone who takes responsibility for the open source licensing of the code being contributed. The aim of this is to create an auditable chain of trust for the licensing of all code in the project.

Each commit which is pushed to git master must have a Signed-off-by line, created by passing the --signoff/-s option to git commit. The line must give the real name of the person taking responsibility for that commit, and indicates that they have agreed to the Developer Certificate of Origin. There may be multiple Signed-off-by lines for a commit, for example, by the developer who wrote the commit and by the maintainer who reviewed and pushed it:

1
2
Signed-off-by: Random J Developer <random@developer.example.org>
Signed-off-by: Lucky K Maintainer <lucky@maintainer.example.org>

Apertis closely follows the Linux kernel process for sign-offs, which is described in section 11 of the kernel guide to submitting patches.

Privileged processes

Pushing commits to gitlab.apertis.org requires commit rights which are only granted to trusted contributors (see “Getting commit rights” for how to get commit rights). All commits must have a Signed-off-by line assigning responsibility for their open source licensing.

Some admin steps on the periphery of packaging and releasing new versions of Apertis modules as Debian packages may require access to build.collabora.co.uk (OBS). These are issued separately from commit rights, and are generally not needed for the main development workflows.

Submitting automated test runs on lava.collabora.co.uk requires CI rights, which are granted similarly to packaging rights. However, CI results may be viewed read-only by anyone.

Getting commit rights

Commit rights (to allow direct pushes to git, and potentially access to the package building system, build.collabora.co.uk) may be granted to trusted third party contributors if they regularly contribute to Apertis, with high quality contributions at the discretion of current Apertis maintainers.

If you wish to acquire an account on the Apertis GitLab instance or other Apertis infrastructure, please send an email to account-requests@apertis.org including:

  • Your full name
  • The email address you prefer to be contacted through
  • The nickname/account name you wish to be known by on the Apertis GitLab