Pagure Dist Git Interactions With Release Engineering Scripts

Release Engineering Scripts (as the name suggests) contains scripts necessary for the Fedora Linux release processes.

Possible Interactions

Documentation

The repository houses the following set of documentation that details on the interaction that the Release Engineering team has with the Pagure Dist Git platform.

Remove dist-git branches

The documentation details on the process that is required to remove branches from the source control management repositories when they are requested for by the maintainers. The said documentation can be found here.

Mass branching

The documentation details on the process of branching the pending release away from devel/ at the alpha freeze and how many elements (including Pagure Dist Git) are involved in making it happen. The said documentation can be found here.

Fedora Layered Image Build System

The documentation describes how the Fedora Layered Image Build System works and how many elements (including Pagure Dist Git) are involved in making it happen. The said documentation can be found here.

Adding a New Release Engineer

The documentation elucidates about the onboarding process for a new release engineer and how they must be provided access to various elements (including Pagure Dist Git) to be able to fulfill their purposes. The said documentation can be found here.

Mass Rebuild of Modules

The documentation explains about how periodical mass rebuilding of modules are performed in rawhide and how many elements (including Pagure Dist Git) are involved in the said process. The said documentation can be found here.

Unretiring a Package Branch

The documentation elaborates about how certain retired packages can be brought to an unretired state and how many elements (including Pagure Dist Git) are involved in the said process. The said documentation can be found here.

Update Critical Path

The documentation explicates how certain applications can be added, updated or removed from the Fedora Linux release’s critical path and how many elements (including Pagure Dist Git) are involved in making it happen. The said documentation can be found here.

Troubleshooting

The documentation demonstrates the behaviour of many elements (including Pagure Dist Git) can be diagnosed and corrected in case those do not work how they are supposed to. The said documentation can be found here.

Scripts

retire_packagers.py

This script helps retiring a packager by orphaning all the packages that the said packager is the main admin of, removing the access to the source control management repository for the said packager and resetting their watch status.

The Pagure Dist Git API is used here and the base URL of the production deployment is mentioned of in the variable dist_git_base. The said script can be found here.

modulepkg.py

This script helps track modules that contain a specific package optionally limited to its branch involving various systems like Product Definition Center, Pagure Dist Git and many other such elements.

The Pagure Dist Git API is used here and the base URL for /modules/ of the production deployment is mentioned of in the variable MODULES_SRC_URL. The said script can be found here.

give-package.py

This script helps transfer a package in Pagure Dist Git from one user to another or to orphan the said package by transferring the package to the orphan user using a privileged Pagure API access token.

The Pagure Dist Git API is used here and the base URL for the REST API of the production deployment is mentioned of in the variable PAGURE_URL. The said script can be found here.

utilities.py

This script helps manipulate Pagure Dist Git. It can retry creation of a session with the Pagure Dist Git API and transferring packages from one user to another by using one of the helper scripts mentioned above.

The Pagure Dist Git API is used here and the base URL for the REST API of the production deployment is mentioned of in the variable PAGURE_URL. The said script can be found here.

sig_policy.py

This script helps to enact the FESCo SIG policy. More about the FESCo SIG policy can be found in this documentation.

The Pagure Dist Git API is used here and the base URL for the data elements of the production deployment is mentioned of in the variable PAGURE_DIST_GIT_DATA_URL. The said script can be found here.

find_unblocked_orphans.py

This script helps discover orphaned packages in Pagure that are unblocked in Koji adnd to show what assets depend on those orphans. This script works across various releases including EPEL and Fedora Linux repositories.

While a direct affiliation with Pagure Dist Git could not be drawn, the object cache depends of a dogpile cache database named dist-git-orphans-cache.dbm. The said script can be found here.

insert_sla.py

This script helps choosing SLAs from a set specified by the Release Engineering on the branches available on Pagure Dist Git to avoid having arbitrary EOLs by working with the Product Definition Center.

While a direct affiliation with Pagure Dist Git could not be drawn, the changes made as a consequence of the execution of this script can lead to changes on Pagure Dist Git. The said script can be found here.

check-retired-branches-pdc.py

This script helps collect all the packages from the Pagure Dist Git to check if they are retired and queries Product Definition Center to confirm they are not listed as active there.

The Pagure Dist Git API is used here and the base URL of the production deployment is mentioned of in the variable dist_git_base. The said script can be found here.

orphan-listed-packages.py

This script helps orphaning listed packages in bulk when they fail to install or fail to build without being addressed for some time. This is a better solution than to perform the process once per package using the scripts.

The Pagure Dist Git API is used here and the base URL of the production deployment is mentioned of in the variable BASE_URL. The said script can be found here.

check-unretirement.py

This script helps finding and listing the contributors of a Fedora Src Package and check their SLAs from Product Definition Center to assert if the file dead.package exists in the package for unretirement confirmation.

The Pagure Dist Git API is used here and the URL of the production deployment for contributors and states are mentioned of in the variables contributors_url and state_url respectively. The said script can be found here.

orphan-all-packages.py

This script helps to orphan all the packages belonging to a certain set of users if they wish to no longer package software on Fedora Linux. The package is either transferred to a new owner from the committers or orphaned.

The Pagure Dist Git API is used here and the base URL of the production deployment is mentioned of in the variable PAGURE_URL. The said script can be found here.

fedretire

This script helps to retire packages that have either been orphaned for more than six weeks, depended on a retried package that was orphaned or because the said package was built and published.

The Pagure Dist Git API is used here and the base URL of the production deployment is mentioned of in the variable PAGURE_URL. The said script can be found here.

It is important to note that this script is not used anymore and in its existing state the script probes into an HTTP API endpoint /api/0/packages that no longer exists now. The script was also last updated over a couple of years back.

find_failures.py

This script helps discover failed builds in a given tag by delivering an output in rough HTML format for a given package belonging to a certain packager.

The Pagure Dist Git API is used here and the URL for the owner data of the production deployment is mentioned of in the variable ownerdataurl. The said script can be found here.

follow-policy.py

This script helps confirm if a package follows policy by asserting its installability and other checks, and in case it does not, it follows through with orphaning the said package belonging to a certain packager.

The Pagure Dist Git API is used here and the URL for the package owner data and confirming package orphaning of the production deployment is mentioned of in the variable pkg_owners and function update_bugs. The said script can be found here.

Changes proposed

In order to switch out Pagure Dist Git for an alternative that provides a relatively compatible API pattern, the following changes must be made in the places mentioned below.

Necessary

  • Release Engineering scripts heavily rely on the Pagure Dist Git API and hence it is important to find a replacement alternative that mimics the API scheme of Pagure Dist Git and replace the URL to the production deployment in the usages of the same in the scripts and documentations.

  • In case the above is not possible, a translation layer (a meta service of sorts) needs to be created and hosted on the same location where the Pagure Dist Git is deployed and that can translate these requests into the API schema that can be comprehended by the replacement alternative.

  • In case the above is not possible, all the aforementioned scripts and documentation will have to be rewritten to play well with the API schema of the replacement alternative chosen and checked thoroughly to confirm that the inter-working of the scripts are as robust as they are at the moment.

Unnecessary

  • None