Apertis is an Open Source project which has been growing sustainable during the past years. This growth also made it spread across different projects and teams, requiring Apertis to improve the tools it uses. In this regard, one issue that prevents Apertis to be really open is the fact that the bug tracking system is only open to maintainers making it hard for the community to report new bugs or to keep track of them.

Since everyone can have access to Gitlab, the most reasonable approach would be to use Gitlab issues for the bug tracking. The current document describes the plan for the migration.

Current status

Nowadays Apertis uses Phabricator to track bugs but it is only accessible to users with privileges, such as maintainers or developers. One additional drawback is that Phabricator is an open source tool which is no longer supported upstream.

Organization

Currently bugs are treated as Phabricator tasks with a bug related tag such as bug, bug (warranty), bug (not-warranty). The meaning of these tags are only important for the management team inside Apertis, and has no value for the community.

Bugs can also have additional tags to relate them to projects, such as infrastructure, licensing, releases such as v2021 and can also have tags to relate them to sources, such as test-failure.

The current workflow relies mostly on the warranty/not-warranty tags and on the release ones (v2021, v2022pre, etc.). Other tags are not used on a day-to-day basis.

Fields in Phabricator

Phabricator supports special fields that help both developers and the management team to keep track of the progress. The list of fields are:

  • Priority: Defines a priority for the issue, possible values: highest, high, normal, low
  • Status: Keeps track of the progress of the work, possible values: unconfirmed, confirmed, in progress, proposed, submitted, resolved, wont fix
  • Visible to: Used to configure some tasks only visible to a group of people
  • Tags: Project defined tags
  • Subscribers: List of people that get a notification on changes

Templates in Phabricator

Currently Phabricator presents a pre-formated bug template page to create a new bug task. This helps developers to keep a consistent format across bugs, as well as avoiding missing important information.

QA Report App

Apertis performs automated tests of all its images on LAVA which are launched on image generation. Test failures automatically generate either a new bug or append new entries if a previous open bug for the same test case is found.

Apertimes

Management team inside Apertis uses apertimes to track progress of the project. This tool retrieves task information from Phabricator to generate reports.

Proposal

As previously mentioned, given all the Apertis related projects are already hosted in Gitlab, the use of Gitlab issues is natural. For this, Gitlab provides a per project issue tracking system which is the best approach to handle development. Gitlab also support incidents, but those are focused on service disruption.

However, since Apertis is an Open Source Distribution, the mindset is different, each individual upstream project usually has its own bug tracking system, while Apertis focus on tracking issues from a distribution’s integration point of view.

Also, requiring contributors to first find the right project to report the issue will not be user friendly, and as consequence some users will avoid reporting the problem or possibly report it to the wrong project.

For these reasons, the recommendation is to create a new single project for bug tracking, which will also help the management team, since there will be a new different issue id for every new issue. This is the same approach Debian uses in its bug tracking system.

This approach has also the advantage of making transition from Phabricator easier as issues will be imported to only one project.

During the import open bugs in Phabricator will be added to Gitlab with title, description, priority, status and tags set. In the description, a link to the Phabricator task will be added to more easily check the history. Importing the history is not recommended since there could be sensitive data, additionally comments also wouldn’t have a correct datetime set. For closed bugs, since there is no value on importing them without the history the recommendation is to keep them only in Phabricator.

The use of tags in Phabricator can be emulated in Gitlab with labels, making the transition almost transparent for end users.

However the use of tags for private management is discouraged since all information is made public. This topic will be covered in more detail in the Management data and view section.

Fields in Gitlab

Gitlab issues does not provide the same fields, however, similar functionality can be implemented using labels. To do so, for each field a set of labels should be created for every possible value, providing also additional flexibility.

For example, to implement the priority field, a set of labels priority: highest, priority: high, priority: normal and priority: low should be created. The same approach can be used for status since it also has a defined set of possible values.

However the visibility field does not make much sense on this approach since the plan behind this task is to openly show the bug tracking system to the community, in the same way other Open Source projects do. In case of an exception, Gitlab supports the use of confidential issues which has visibility set to only team members.

Tags can be easily implemented by labels since they have the same behavior, providing a very flexible way of connecting issues.

Finally, subscribers can be handled using the notification system of Gitlab. Issues templates can be configured to notify people about new activity by sending an email allowing all team members to be informed. Gitlab supports different [notification levels] providing a way for users to configure them according to their needs.

Templates in Gitlab

Gitlab templates are used to have a common pattern to report bugs. Gitlab supports creating templates using markdown and thus allowing an easy way to import the current Phabricator templates.

The management of templates is straightforward since they are file located in the .gitlab/issue_templates folder, so the creation, modification and deletion can be handled thorough merge requests, which also helps traceability.

Permissions

As described in permissions, Gitlab supports different roles in a project and since bugs will be a new one, permissions can be easily adjusted as needed.

In order to provide public access to reported issues, contributors need to be assigned the role of guest to the project. In this way they can create issues but won’t be able to change labels or assign them after creation.

Management data and view

Gitlab issues is a powerful tool to keep track of different tasks, in this case bugs. However, due to the private nature of some aspects of these tasks some information should be placed in a different location.

On the public project useful information can be included to help the management team, as example new labels based on bug-type should be used:

  • bug-type:apertis: To denote a bug that appeared in Apertis due to work done inside Apertis project
  • bug-type:upstream: To denote a bug that appeared in Apertis due to a bug also present upstream

Any private information should be placed in a different location, such as a private GitLab project or an external tool, and the mapping between issues and management data should be performed manually.

The easiest way to implement this is to keep private information in Phabricator as it is the tool currently used, reducing the complexity of the solution. With this approach bugs will be handled on Gitlab issues but a Phabricator task will be created and linked to track private management information. To make this possible a tool should be created to create a Phabricator task based on a Gitlab issue with the same title and a description referencing it. The management team will use this task to track only private data. Lastly, since management data will remain available in Phabricator, apertimes tool doesn’t require any update, making the transition easier.

For public data, to have a high level overview of bugs, Gitlab support boards which provide a kanban style interface. Boards columns can be configured based on labels, so the initial approach could be to have a board which shows bugs grouped by priority, which should help tracking the most important issues. It is also possible to create multiple issue boards per project, to highlight other aspects, such as bug-type.

Summary

The table below shows a summary for the mapping between Phabricator and Gitlab features/information:

Phabricator Gitlab
Organization Tasks with tag bug Gitlab issues on new project
Priority field Internal field Label with prefix priority
Status field Internal field Label with prefix status
Asignee field Internal field Internal field
Templates Supported Supported
Permissions Supported Supported
Management view Workboard Gitlab boards
Information type Public and private Public information only

Export/import

A key point during the migration is the export and import task, which should end up with all the open bugs in Phabricator available as Gitlab issues.

The desired data to be migrated is:

  • Title
  • Description with a link to the original Phabricator task
  • Priority
  • Status
  • Tags

In this regard there are some tools already available such as import from csv and import from Phabricator. These tools only allow to import title and description, so importing additional information requires building a tool based on the Phabricator and Gitlab API. Creating such a tool will likely be rather simple though.

Migration

The migration should be planned with enough time before a quarter ends in order to have a stable system previous to the release dates. It is a pre-requisite that the QA Report App is updated to support Gitlab issues and a set of tools are developed, one to create and mirror Phabricator tasks based on Gitlab issues, and another to [Export/import] Phabricator tasks into Gitlab issues.

For the actual migration, first a clean up of the current open bugs list should be performed, to avoid importing useless information. After this point, a freeze period should be started to guarantee that no modification to the bugs is made on Phabricator during the migration. In this context the export/import operation can be triggered, which should end with a production test to confirm that all bugs have been migrated and tools work as expected. The freeze period should be minimal, a first estimation is a few days, which should be enough to run tests and fix trivial problems.

Steps

  • Create test environment on Gitlab
  • Enhance QA Report App to use Gitlab issues instead of Phabricator
  • Enhance Apertimes to use Gitlab issues instead of Phabricator
  • Create script to export bugs
  • Cleanup bug list
  • Freeze bug report/update on Phabricator
  • Export bug list
  • Import bugs list
  • Perform production test
  • Unfreeze bug report/update on Gitlab