Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update mirrorbrain #250

Merged
merged 7 commits into from
Oct 14, 2024
Merged

Update mirrorbrain #250

merged 7 commits into from
Oct 14, 2024

Conversation

benoit74
Copy link
Collaborator

@benoit74 benoit74 commented Oct 20, 2023

⚠️ Do not merge before kiwix/mirrorbrain#1 is ready and released. 🦺

Fix #244
Fix #251

Changes :

  • update base image from httpd:2.4.43 to 2.4.57
    • this includes a move to Debian Bookworm
    • and hence an update to Python 3 (Python 2 is dropped in Bookworm)
  • replace python2 packages by python3 ones
  • move to MaxMind GeoIP2
    • remove geoip-api-c and mod_geoip2
    • add libmaxminddb (C library), geoipupdate (tooling to fetch / update MaxMind DBs) and mod_maxminddb
  • remove obsolete tools geoiplookup_continent, geoiplookup_city and geoip-lite-update
  • copy new MB assets to a /var/www subfolder
  • modify mirror list generation so that we have to pass a custom HTML header template (which will reference the proper base URL where assets will be located, probably https://download.kiwix.org)
  • created a mirrorbrain/dev folder containing a minimal docker-compose setup to test Mirrorbrain locally

@rgaudin rgaudin marked this pull request as ready for review October 14, 2024 11:11
@rgaudin
Copy link
Member

rgaudin commented Oct 14, 2024

I've updated the versions of the base image and the libraries.
I've put it on a separate tag (:bookworm).

As discussed with @benoit74, there's no real way to properly test this outside of production. The goal is thus to try this live, monitor it and quickly revert to latest on issues (and iterate).

It is acknowledged that we it should not disturb downloads delivery much:

  • apache module code has not changed beside switching to MMDB and this has been tested.
  • apache API is very stable so upgrade should not have impact

The other tools we use being:

  • mirrorprobe: changes were syntax only but it does import mb
  • mb scan (via update_mirrorbrain_db.sh): heavy work is deffered to a perl script but a lot still done in python script.
  • mb mirrorlist: this was tested
  • geoipupdate: that's the the maxmind binary. We need to ensure this executes properly and its results are properly used by mod_mb (we use default paths)
  • mb makehashes: we'll need to be careful about this working properly. hashes can easily be tested but btih are also computed there

@rgaudin rgaudin merged commit c585008 into main Oct 14, 2024
6 checks passed
@rgaudin rgaudin deleted the update_mirrorbrain branch October 14, 2024 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mirrorbrain is not reporting IP location appropriately Update MirrorBrain base image
2 participants