From ebc8fea34d63e25c1a155e6b1d44a8c26d3cb849 Mon Sep 17 00:00:00 2001 From: Simone Pelosi Date: Mon, 4 Mar 2024 18:37:17 +0100 Subject: [PATCH 1/3] Add documentation for git hosting --- .wordlist.txt | 5 ++ reference/index.rst | 8 +++ reference/services/git-hosting.rst | 90 ++++++++++++++++++++++++++++++ 3 files changed, 103 insertions(+) create mode 100644 reference/services/git-hosting.rst diff --git a/.wordlist.txt b/.wordlist.txt index bc8bdc2..03d193f 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -4,6 +4,7 @@ APIs balancer bugfix callables +cgit Charmhub CLI conf @@ -18,14 +19,17 @@ dropdown EBS EKS favicon +filesystem FIXME fmt formatters garbo +git Grafana IAM innerHTML installable +haproxy Jira JS JSON @@ -55,6 +59,7 @@ RSS RTD sampledata scss +SSL stylesheets subdirectories subfolders diff --git a/reference/index.rst b/reference/index.rst index 578d4d2..ae0abe6 100644 --- a/reference/index.rst +++ b/reference/index.rst @@ -50,6 +50,14 @@ Ubuntu-related services/mirror-prober.rst +Git-related +~~~~~~~~~~~~~~ + +.. toctree:: + :maxdepth: 1 + + services/git-hosting.rst + Possibly out-of-date -------------------- diff --git a/reference/services/git-hosting.rst b/reference/services/git-hosting.rst new file mode 100644 index 0000000..177388f --- /dev/null +++ b/reference/services/git-hosting.rst @@ -0,0 +1,90 @@ +Git hosting +=============== + +Description +----------- + +Launchpad has direct Git hosting support alongside its existing Bazaar hosting support. +The major subsystems involved are: + +* The git client +* Hosting service +* The underlying object model +* The web application +* Repository source code browser + +Git client +---------- + +This is what users install on their systems to use Git. +It is also installed on the hosting backend to support the ``receive-pack`` and ``upload-pack`` protocols. + +Hosting service +--------------- + +The outside world connects to the Git hosting service using one of several protocols: + +* ``git://`` +* ``git+ssh://`` +* ``https://`` + +The Git "dumb" HTTP transport will not be supported, as it has largely been considered +deprecated since the provision of "smart" HTTP transport in Git v1.6.6 (released January, 2010). + +SSL is terminated by haproxy for HTTPS, and all the other protocols also pass through +haproxy for load balancing (although at the moment we only have a single backend). +The underlying protocol endpoints live in `lp:turnip `_, +which invokes ``git upload-pack`` and ``git receive-pack`` to implement the git protocol itself. + +``turnip`` also provides an internal API used by the Launchpad web application to manipulate and inspect repositories; +and in turn the Launchpad web application provides an internal XML-RPC interface used by ``turnip`` to translate logical +repository paths to filesystem paths and to notify Launchpad of repository changes. + +``turnip`` is deployed using Juju, using the `turnip charm `_. + +See more at `turnip documentation `_ + +The web application +------------------- + +Code that is executed as part of the Launchpad web application. Major features: + + * general information + * listings for various registry objects - people, teams, projects, packages + * default repositories for projects and packages + * privacy + +Repository source code browser +------------------------------ + +Launchpad uses the external `cgit `_ project to provide a web view of the repository contents. +This is invoked by ``turnip`` when it receives HTTPS requests that don't correspond +to git smart HTTP protocol, and ``turnip``` deals with configuring ``cgit`` appropriately for each repository. + +Log files +--------- +See `FreshLogs documentation `_. + +Production +~~~~~~~~~~ + +* ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-access.log`` +* ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-api-access.log`` +* ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-api-error.log`` +* ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-pack-backend.log`` +* ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-pack-frontend-git.log`` +* ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-pack-frontend-http.log`` +* ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-pack-frontend-ssh.log`` +* ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-pack-virt.log`` + +Staging +~~~~~~~ + +* ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-access.log`` +* ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-api-access.log`` +* ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-api-error.log`` +* ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-pack-backend.log`` +* ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-pack-frontend-git.log`` +* ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-pack-frontend-http.log`` +* ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-pack-frontend-ssh.log`` +* ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-pack-virt.log`` \ No newline at end of file From 726bdc4b4569ad2befc1e3cc5d9716248df81bda Mon Sep 17 00:00:00 2001 From: Simone Pelosi Date: Fri, 22 Mar 2024 15:37:32 +0100 Subject: [PATCH 2/3] Address comments --- reference/services/git-hosting.rst | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/reference/services/git-hosting.rst b/reference/services/git-hosting.rst index 177388f..8d3b025 100644 --- a/reference/services/git-hosting.rst +++ b/reference/services/git-hosting.rst @@ -8,10 +8,10 @@ Launchpad has direct Git hosting support alongside its existing Bazaar hosting s The major subsystems involved are: * The git client -* Hosting service +* Hosting service (turnip) * The underlying object model -* The web application -* Repository source code browser +* The web application (Launchpad) +* Repository source code browser (cgit) Git client ---------- @@ -31,8 +31,10 @@ The outside world connects to the Git hosting service using one of several proto The Git "dumb" HTTP transport will not be supported, as it has largely been considered deprecated since the provision of "smart" HTTP transport in Git v1.6.6 (released January, 2010). -SSL is terminated by haproxy for HTTPS, and all the other protocols also pass through -haproxy for load balancing (although at the moment we only have a single backend). +SSL is terminated by haproxy (haproxy has two units haproxy/1* and haproxy/0) for HTTPS, and all the other +protocols also pass through haproxy for load balancing (although at the moment we only have a single backend). +See the full charm configuration at `launchpad-mojo-specs _`. + The underlying protocol endpoints live in `lp:turnip `_, which invokes ``git upload-pack`` and ``git receive-pack`` to implement the git protocol itself. @@ -40,9 +42,9 @@ which invokes ``git upload-pack`` and ``git receive-pack`` to implement the git and in turn the Launchpad web application provides an internal XML-RPC interface used by ``turnip`` to translate logical repository paths to filesystem paths and to notify Launchpad of repository changes. -``turnip`` is deployed using Juju, using the `turnip charm `_. +``turnip`` is deployed using Juju, using the `turnip charms `_. -See more at `turnip documentation `_ +For more details, see the `turnip documentation `_ The web application ------------------- @@ -52,7 +54,7 @@ Code that is executed as part of the Launchpad web application. Major features: * general information * listings for various registry objects - people, teams, projects, packages * default repositories for projects and packages - * privacy + * privacy settings Repository source code browser ------------------------------ @@ -77,8 +79,8 @@ Production * ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-pack-frontend-ssh.log`` * ``rless turnip-pack-{1,2,3,4}.lp.internal::turnip-logs/turnip-pack-virt.log`` -Staging -~~~~~~~ +Qastaging +~~~~~~~~~ * ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-access.log`` * ``rless turnip-pack-{1,2}.qastaging.lp.internal::turnip-logs/turnip-api-access.log`` From 1fca0df99b43e316c8a77ace30dfdce5df6dd6b5 Mon Sep 17 00:00:00 2001 From: Simone Pelosi Date: Fri, 22 Mar 2024 16:09:54 +0100 Subject: [PATCH 3/3] Address comments --- reference/services/git-hosting.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/reference/services/git-hosting.rst b/reference/services/git-hosting.rst index 8d3b025..77fa659 100644 --- a/reference/services/git-hosting.rst +++ b/reference/services/git-hosting.rst @@ -9,7 +9,6 @@ The major subsystems involved are: * The git client * Hosting service (turnip) -* The underlying object model * The web application (Launchpad) * Repository source code browser (cgit) @@ -33,7 +32,7 @@ deprecated since the provision of "smart" HTTP transport in Git v1.6.6 (released SSL is terminated by haproxy (haproxy has two units haproxy/1* and haproxy/0) for HTTPS, and all the other protocols also pass through haproxy for load balancing (although at the moment we only have a single backend). -See the full charm configuration at `launchpad-mojo-specs _`. +See the full charm configuration at `launchpad-mojo-specs `_. The underlying protocol endpoints live in `lp:turnip `_, which invokes ``git upload-pack`` and ``git receive-pack`` to implement the git protocol itself.