Dist Git Move

Definition

Dist Git is a generic Git forge platform based on Pagure with additional data storage. The Git forge part of the platform holds contents like codebases, documentation and RPM specfiles and the data storage part of the platform holds binaries like source tarballs. The project primarily consists of three main components - Git repositories, Lookaside Cache to store source tarballs and scripts to manage both. More information about the project can be found here.

Purpose

The objective of the potential initiative is to move repository contents (including but not limited to source codes, Packit configurations, RPM specfiles) from Pagure Dist-Git to another platform and confirm that the associated tooling and services (including but not limited to FMN, Datanommer, COPR, Toddlers, FMN, CI, Monitor-Gating, Packit, Bodhi, Fedpkg) work well with the newer platform. The investigation aims to be as agnostic as it can be regarding the destination platform to help ideate a general solution for the compatibility of the associated tooling and services.

Background

With Pagure’s development having less activity for a while now, we need to ensure that the workflow changes are documented associated tooling and services are changed to continue working the way they did when Pagure Dist-Git is decomissioned and the repository contents are moved elsewhere, while confirming that we are not locked down to the features that are specific to the chosen destination platform.

Requirements

  • Study the interactions of toolings/services with Dist-Git

  • Include additions to Datanommer to support interactions with another platform

  • Include additions to Fedora Notifications to support interactions with another platform

  • Include additions to COPR to support interactions with another platform

  • Include additions to Toddlers to support interactions with another platform

  • Include additions to Continuous Integration to support interactions with another platform

  • Include additions to Release Engineering to support interactions with another platform

  • Include additions to Monitor Gating to support interactions with another platform

  • Include additions to Packit to support interactions with another platform

  • Include additions to Bodhi to support interactions with another platform

  • Include additions to Pagure to support interactions with another platform

  • Include additions to Anitya/The New Hotness to support interactions with another platform

Interactions

Index

Estimate

To adequately complete the proposed work, 3-4 engineers for approximately 2 quarters is recommended. 2-3 developers and 1 sysadmin should suffice the need of both building the said solution as well as creating the deployment.

Roadmap

  • Step 1 - Discussing on the Git Forge and deciding on a specific one to use

  • Step 2 - Introducing the newer API compatibility to the internally maintained applications

  • Step 3 - Adding Git Large File System support for the source tarballs and binary files

  • Step 4 - Developing the compatibility metaservice for the chosen Git Forge API

  • Step 5 - Making a staging deployment for the Git Forge on a subdomain (say, SUBDOMAIN A)

  • Step 6 - Pointing the Dist Git’s staging URL (say, SUBDOMAIN B) to the compatibility metaservice

  • Step 7 - Following up for the externally maintained applications to switch over to the newer API

  • Step 8 - Improving the project on the community feedback received on the staging deployment tests

  • Step 9 - Making a production deployment for the Git Forge on a subdomain (say, SUBDOMAIN C)

  • Step 10 - Pointing the Dist Git’s production URL (say, SUBDOMAIN D) to the compatibility metaservice

  • Step 11 - Announcing a time limit for the planned obsolescence for the compatibility metaservice

  • Step 12 - Confirming that all the applications have switched over to the newer API

  • Step 13 - Decommissioning of the compatibility metaservice after its utility is outlived

  • Step 14 - PROFIT?

Appendix

  • Internally maintained applications - Projects maintained by the Red Hat CPE team (eg. Monitor Gating, The New Hotness, Releng Scripts, Toddlers, Notifications and Datanommer)

  • Externally maintained applications - Projects maintained by other teams (eg. Fedpkg, COPR, Fedora CI, Bodhi and Packit)

  • Newer API compatibility - Making pre-releases with the compatibility changes included as the said Git Forge is not yet deployed

Technologies

  • Git - For Source Code Management

  • Git LFS - For source tarballs and binary files in lookaside cache

  • FastAPI - For the compatibility metaservice backend

  • Vue - For the Git Forge frontend