Add a belated changelog update with recent OCaml Infrastructure updates #2744

shonfeder · 2024-10-03T23:34:43Z

Requested by @rikusilvola and @tmattio, and in accordance with our discussion today about improving the cadence and regularity of changelog updates.

This is an unusually long entry, relative to past entries and what you can expect from us in the future. That is only because we are making up for a lapse in external communication.

In ocaml/infrastructure#159 we will be announcing the plan to shift to using the ocaml changelog as our main venue for updates.

It's my first time writing one of these, so please let me know if the content or tone should be adjusted.

cc @mtelvers, @cuihtlauac (since I cannot request you are reviewers :)).

punchagan · 2024-10-04T06:03:28Z

data/changelog/infra/2024-10-02-updates.md

+
+# Catching up
+
+[Taride](https://tarides.com/)'s OCaml Infrastructure team has been hard at work


s/Taride/Tarides (I don't know if that's a typo or that how Tarides's is supposed to be written.

Thomas decided that any possessive of Tarides should be Tarides'

punchagan · 2024-10-04T06:13:00Z

data/changelog/infra/2024-10-02-updates.md

+  [opam-ci-check](https://github.com/ocurrent/opam-repo-ci/tree/master/opam-ci-check),
+  a command line utility aiming to make opam repo CI's logic local-first. It is
+  still in the early stages and far from finished, but it is enabling faster
+  development iteration already, and it will soon be fit for early adopting


I think we can already call it fit for early adopters to test, given that we have added instructions to reproduce locally. (to be deployed soon)?

tmattio · 2024-10-04T07:56:23Z

Thank you for preparing this update on the OCaml infrastructure work! It's great to see all the progress that's been made. I have some feedback to help align this more closely with the OCaml Changelog's intended format and purpose:

Remember that the OCaml Changelog is a blog, not a raw changelog of changes. We aim to publish announcements that are user-centric, focusing on changes that have a direct impact on users.
The current PR reads more like a detailed changelog, with credits and links to PRs. While this is valuable, it might be better suited for the changelog field in the YAML frontmatter at the top of the file.
Consider grouping the changes into more specific user-relevant categories. Based on the updates you've shared, here's a suggested grouping:
- Platform Support
  - Added support for OCaml 5.2 and 5.3
  - Added support for GCC 14
  - Improved Windows support
  - Dropped support for Debian 10 and 11
- Opam Repository Publishing Experience
  - Implemented linting for package maintainer email addresses
  - Mitigated solver timeout errors in opam repo CI
  - Fixed uncaught exception in opam repo CI linting checks
  - Improved error reporting and local lint check instructions in opam CI
- Security and Reliability
  - Updated systems to address CVE-2024-6387
  - Maintained opam repo CI at 99.9% availability for the last 2 months
Some updates might not be directly relevant for users and could be omitted from the main blog post or mentioned only briefly if they lead to user-facing improvements:
- Added continuous deployment for service configuration
- Generated documentation for CI test platforms and deployed services
- Developed opam-ci-check utility for local-first CI logic

Here's an example of what I mean, feel free to discard it, this is more to give a concrete example of the format we could target for the infrastructure updates.

---
title: OCaml Infrastructure: Enhancing Platform Support and User Experience
authors: [Taride OCaml Infrastructure Team]
date: 2024-10-04
changelog: |
  - Added continuous deployment for service configuration (by @mtelvers, https://github.com/ocurrent/ocurrent-configurator)
  - Implemented linting for package maintainer email addresses (by @punchagan, https://github.com/ocurrent/opam-ci-check/pull/30)
  - Mitigated solver timeout errors in opam repo CI (by @mtelvers and @shonfeder, https://github.com/ocaml/infrastructure/issues/147)
  - Fixed uncaught exception in opam repo CI linting checks (by @shonfeder, https://github.com/ocurrent/opam-repo-ci/pull/341)
  - Updated systems to address CVE-2024-6387 (by @mtelvers, https://github.com/ocaml/infrastructure/issues/140)
  - Added support for OCaml 5.2 and 5.3 (by @mtelvers, https://github.com/ocurrent/opam-repo-ci/issues/268 and https://github.com/ocurrent/opam-repo-ci/issues/363)
  - Improved Windows support (by @mtelvers, https://github.com/ocurrent/docker-base-images/issues/241)
  - Added support for GCC 14 (by @mtelvers, https://github.com/ocurrent/docker-base-images/issues/279)
  - Dropped support for Debian 10 and 11 (by @mtelvers and @shonfeder, https://github.com/ocurrent/ocaml-dockerfile/pull/220 and https://github.com/ocurrent/ocaml-dockerfile/pull/210)
  - Generated documentation of CI test platforms and deployed services (by @benmandrew, https://github.com/ocurrent/opam-repo-ci/blob/master/doc/platforms.md and https://github.com/ocurrent/ocurrent-deployer/blob/master/doc/services.md)
  - Developed opam-ci-check utility for local-first CI logic (by @punchagan and @shonfeder)
  - Improved error reporting and local lint check instructions in opam CI (by @punchagan, https://github.com/ocurrent/opam-repo-ci/issues/359 and https://github.com/ocurrent/opam-repo-ci/issues/360)
---

[Taride](https://tarides.com/)'s OCaml Infrastructure team has been hard at work
over the last months, focusing on improving the stability, reliability, and user experience of the OCaml ecosystem's core infrastructure. We are preparing for more
substantive changes to improve the reliability and scalability of essential
OCaml ecosystem infrastructure. We will have more fruits from this work to share
soon, but updates on our efforts are overdue and we wanted to get you all caught
up.

## Platform Support

We've expanded and updated our platform support to keep pace with the evolving OCaml ecosystem:

- As part of the compiler release readiness, we've added support for OCaml 5.2 and the upcoming 5.3, currently in alpha.
- We're now publishing Windows Server 2022 Docker images to the OCaml Dockerhub. See for instance [windows-server-msvc-ltsc2022-ocaml-4.14](https://hub.docker.com/layers/ocaml/opam/windows-server-msvc-ltsc2022-ocaml-4.14/images/sha256-cfe98cb048514e9eace3c4d4e607b7e7fce5abba3a49e9a07318180d6213eee2?context=explore)
- Added support for GCC 14, which brings support for Arch, Debian {unstable, testing}, and Fedora 40
- Dropped support for the post-LTS distro versions Debian 10 and 11, focusing our resources on more current platforms.

These updates ensure that our systems continue to support OCaml developers on a broad range of systems, in particular by testing package publication to the opam repository and publishing Docker images.

## Opam Repository Publishing Experience

We've made several improvements to enhance the experience of publishing packages to the opam repository:

- Implemented linting for package maintainer email addresses, ensuring package maintainers can be reliably contacted when needed.
- Mitigated solver timeout errors in the opam repo CI system, reducing frustrating delays in the package publishing process.
- Fixed an uncaught exception that sometimes occurred during linting checks in the opam repo CI.
- Improved error reporting and added instructions for running lint checks locally, making it easier for package authors to identify and resolve issues before submitting to the repository.

We hope these changes will make package publication experience smoother. We're committed to improving the experience for newcomers and existing repository contributors alike, stay tuned for more improvements!

## Security and Reliability

Maintaining the security and reliability of the OCaml infrastructure remains a top priority:

- Updated our systems to address [CVE-2024-6387](https://nvd.nist.gov/vuln/detail/CVE-2024-6387).
- Maintained 99.9% availability for the opam repo CI over the last two months, providing a stable and reliable service for the OCaml community.

## Forging ahead

Much has been done but much more is in the works! Expect updates more regularly
going forward, and don't hesitate to reach out on
[ocaml/infrastructure/issues](https://github.com/ocaml/infrastructure/issues) or
[discuss.ocaml.org](https://discuss.ocaml.org/tag/infrastructure) if you have
questions, requests, or ideas!

christinerose · 2024-10-04T11:50:27Z

data/changelog/infra/2024-10-02-updates.md

+tags: [infrastructure]
+---
+
+# Catching up


Suggested change

# Catching up

# Catching Up

Suggested title case correction

christinerose · 2024-10-04T11:51:17Z

data/changelog/infra/2024-10-02-updates.md

+
+# Catching up
+
+[Taride](https://tarides.com/)'s OCaml Infrastructure team has been hard at work


Suggested change

[Taride](https://tarides.com/)'s OCaml Infrastructure team has been hard at work

[Tarides'](https://tarides.com/) OCaml Infrastructure team has been hard at work

christinerose · 2024-10-04T11:52:24Z

data/changelog/infra/2024-10-02-updates.md

+over the last months, focused on sustaining the existing systems and
+regenerating our team's capacities. This work is in preparation for more
+substantive changes to improve the reliability and scalability of essential
+OCaml ecosystem infrastructure. We will have more fruits from this work to share
+soon, but updates on our efforts are overdue and we wanted to get you all caught
+up.


Suggested change

over the last months, focused on sustaining the existing systems and

regenerating our team's capacities. This work is in preparation for more

substantive changes to improve the reliability and scalability of essential

OCaml ecosystem infrastructure. We will have more fruits from this work to share

soon, but updates on our efforts are overdue and we wanted to get you all caught

up.

over the last few months, focused on sustaining the existing systems and

regenerating our team's capacities. This work is in preparation for more

substantive changes to improve the reliability and scalability of essential

OCaml ecosystem infrastructure. We will have more fruit from this work to share

soon, but updates on our efforts are overdue and we wanted to get you all caught

up.

Minor language usage corrections

christinerose · 2024-10-04T11:52:44Z

data/changelog/infra/2024-10-02-updates.md

+
+Since we last published an announcement, we have been...
+
+## making the systems more stable


Suggested change

## making the systems more stable

## Making the Systems More Stable

christinerose · 2024-10-04T11:53:36Z

data/changelog/infra/2024-10-02-updates.md

+  in the opam repo CI (by [@shonfeder](https://github.com/shonfeder), see
+  [ocurrent/opam-repo-ci#341](https://github.com/ocurrent/opam-repo-ci/pull/341).
+
+## keeping the systems up to date


Suggested change

## keeping the systems up to date

## Keeping the Systems Up to Date

christinerose · 2024-10-04T11:53:58Z

data/changelog/infra/2024-10-02-updates.md

+
+- Dropped support for the post-LTS distro versions Debian 10 and 11 (by
+  [@mtelvers](https://github.com/mtelvers) and
+  [@shonfeder](https://github.com/shonfeder) see


Suggested change

[@shonfeder](https://github.com/shonfeder) see

[@shonfeder](https://github.com/shonfeder), see

christinerose · 2024-10-04T11:54:09Z

data/changelog/infra/2024-10-02-updates.md

+- There was a profusion of solver timeout errors impacting the opam repo CI
+  system, and we have managed to mitigate these (by
+  [@mtelvers](https://github.com/mtelvers) and
+  [@shonfeder](https://github.com/shonfeder) see


Suggested change

[@shonfeder](https://github.com/shonfeder) see

[@shonfeder](https://github.com/shonfeder), see

christinerose

Mostly minor formatting, capitalisation, and grammar fixes...so apologies for the noise! I usually edit little things like this directly, but I don't have write access on this. If you'd like to give it to me, I can make these changes without disrupting the conversation too much. 🙂

christinerose · 2024-10-04T11:54:33Z

data/changelog/infra/2024-10-02-updates.md

+  and
+  [ocurrent/ocaml-dockerfile#210](https://github.com/ocurrent/ocaml-dockerfile/pull/210)).
+
+## making it easier to understand the systems


Suggested change

## making it easier to understand the systems

## Making It Easier to Understand the Systems

christinerose · 2024-10-04T11:54:58Z

data/changelog/infra/2024-10-02-updates.md

+[services]: https://github.com/ocurrent/ocurrent-deployer/blob/master/doc/services.md
+[platforms]: https://github.com/ocurrent/opam-repo-ci/blob/master/doc/platforms.md
+
+## improving the functionality and usability of the systems


Suggested change

## improving the functionality and usability of the systems

## Improving the Functionality and Usability of the Systems

christinerose · 2024-10-04T11:55:21Z

data/changelog/infra/2024-10-02-updates.md

+## improving the functionality and usability of the systems
+
+- We have been developing
+  [opam-ci-check](https://github.com/ocurrent/opam-repo-ci/tree/master/opam-ci-check),


Suggested change

[opam-ci-check](https://github.com/ocurrent/opam-repo-ci/tree/master/opam-ci-check),

[`opam-ci-check`](https://github.com/ocurrent/opam-repo-ci/tree/master/opam-ci-check),

christinerose · 2024-10-04T11:55:35Z

data/changelog/infra/2024-10-02-updates.md

+  and
+  [ocurrent/opam-repo-ci#360](https://github.com/ocurrent/opam-repo-ci/issues/360)).
+
+# Forging ahead


Suggested change

# Forging ahead

# Forging Ahead

shonfeder · 2024-10-04T12:58:10Z

@tmattio Thanks for the review.

it might be better suited for the changelog field in the YAML frontmatter at the top of the file

What effect does this have?

tmattio · 2024-10-04T14:05:04Z

What effect does this have?

It displays a "See full changelog" button at the bottom of the post, which displays the content of the field, see for instance https://ocaml.org/changelog/2024-07-31-ocaml-lsp-1.19.0

shonfeder · 2024-10-04T15:02:47Z

I see, thanks!

I would note, IMO generating documentation of what we are testing and deploying is likely to be of direct value to users.

Also, I suggest expanding the contributing section with the kinds of guidelines you've shared here, as I consulted that before opening this PR, and all it says is

The OCaml Changelog is a feed of the latest releases and feature highlights for official OCaml projects.

Stipulating the stylistic and content constraints there (along with providing a template somewhere) would help with the process we discussed.

shonfeder · 2024-10-04T15:38:59Z

I adopted @tmattio's preferred presentation with minor changes and additions. I also tried to preserve all of @christinerose's fixes.

I don't have write access on this

I don't have permissions to grant you write access to ocaml.org, sorry. But your suggestions and corrections are most welcome and not noise.

christinerose · 2024-10-04T15:39:51Z

Thank you for saying so! ❤️

tmattio · 2024-10-04T16:11:04Z

Thanks a lot! I'll add these guidelines to the contributing guide in a separate PR

shonfeder · 2024-10-04T16:57:58Z

Thanks for review, feedback, guidance :)

Unfortunately, it looks like this was deployed despite the CI check on this branch failing (https://github.com/ocaml/ocaml.org/actions/runs/11182816793/job/31089951473) and as a result the same (inscrutable) error has broken the deployment https://deploy.ci.ocaml.org/job/2024-10-04/161841-ocluster-build-376c36

I would suggest enabling branch protection rules on this repo (and just in general).

(I'm trying to figure out why the post broke the build and how to fix it.)

Fixes several errors in the meta-data from ocaml#2744 . These broke the build with uncaught exceptions such as ``` Ood_gen.Exn.Decode_error("data/changelog/infra/2024-10-02-updates.md : error calling parser: mapping values are not allowed in this context character 0 position 0 returned: 0") ``` and ``` Ood_gen.Exn.Decode_error("data/changelog/infra/2024-10-02-updates.md : data/changelog/infra/2024-10-02-updates.md : date2024-10-04\n") ``` Also added the infrastructure tag that had been lost during review.

Fixes several errors in the meta-data from #2744 . These broke the build with uncaught exceptions such as ``` Ood_gen.Exn.Decode_error("data/changelog/infra/2024-10-02-updates.md : error calling parser: mapping values are not allowed in this context character 0 position 0 returned: 0") ``` and ``` Ood_gen.Exn.Decode_error("data/changelog/infra/2024-10-02-updates.md : data/changelog/infra/2024-10-02-updates.md : date2024-10-04\n") ``` Also added the infrastructure tag that had been lost during review.

shonfeder added 2 commits October 3, 2024 19:18

Note inclusion of Infrastructure in the changelog

6b36ae4

changelog: add belated updates for Infra work

3b7c1c0

shonfeder requested a review from christinerose as a code owner October 3, 2024 23:34

punchagan approved these changes Oct 4, 2024

View reviewed changes

mtelvers approved these changes Oct 4, 2024

View reviewed changes

christinerose reviewed Oct 4, 2024

View reviewed changes

Adopts Thibaut's changes

448e05e

shonfeder requested a review from christinerose October 4, 2024 15:39

christinerose approved these changes Oct 4, 2024

View reviewed changes

tmattio merged commit ede2f21 into ocaml:main Oct 4, 2024
0 of 3 checks passed

shonfeder deleted the infra-changelog-update branch October 4, 2024 17:22

shonfeder mentioned this pull request Oct 4, 2024

Fix metadata in Infrastrucure changelog update #2749

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a belated changelog update with recent OCaml Infrastructure updates #2744

Add a belated changelog update with recent OCaml Infrastructure updates #2744

shonfeder commented Oct 3, 2024 •

edited

Loading

punchagan Oct 4, 2024

christinerose Oct 4, 2024

punchagan Oct 4, 2024

tmattio commented Oct 4, 2024

christinerose Oct 4, 2024

christinerose Oct 4, 2024

christinerose Oct 4, 2024

christinerose Oct 4, 2024

christinerose Oct 4, 2024

christinerose Oct 4, 2024

christinerose Oct 4, 2024

christinerose left a comment

christinerose Oct 4, 2024

christinerose Oct 4, 2024

christinerose Oct 4, 2024

christinerose Oct 4, 2024

shonfeder commented Oct 4, 2024

tmattio commented Oct 4, 2024

shonfeder commented Oct 4, 2024

shonfeder commented Oct 4, 2024 •

edited

Loading

christinerose commented Oct 4, 2024

tmattio commented Oct 4, 2024

shonfeder commented Oct 4, 2024 •

edited

Loading


		# Catching up

		[Taride](https://tarides.com/)'s OCaml Infrastructure team has been hard at work

	[Taride](https://tarides.com/)'s OCaml Infrastructure team has been hard at work
	[Tarides'](https://tarides.com/) OCaml Infrastructure team has been hard at work


		Since we last published an announcement, we have been...

		## making the systems more stable

	## keeping the systems up to date
	## Keeping the Systems Up to Date

	[@shonfeder](https://github.com/shonfeder) see
	[@shonfeder](https://github.com/shonfeder), see

	## making it easier to understand the systems
	## Making It Easier to Understand the Systems

	## improving the functionality and usability of the systems
	## Improving the Functionality and Usability of the Systems

	[opam-ci-check](https://github.com/ocurrent/opam-repo-ci/tree/master/opam-ci-check),
	[`opam-ci-check`](https://github.com/ocurrent/opam-repo-ci/tree/master/opam-ci-check),

Add a belated changelog update with recent OCaml Infrastructure updates #2744

Add a belated changelog update with recent OCaml Infrastructure updates #2744

Conversation

shonfeder commented Oct 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tmattio commented Oct 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

christinerose left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shonfeder commented Oct 4, 2024

tmattio commented Oct 4, 2024

shonfeder commented Oct 4, 2024

shonfeder commented Oct 4, 2024 • edited Loading

christinerose commented Oct 4, 2024

tmattio commented Oct 4, 2024

shonfeder commented Oct 4, 2024 • edited Loading

shonfeder commented Oct 3, 2024 •

edited

Loading

shonfeder commented Oct 4, 2024 •

edited

Loading

shonfeder commented Oct 4, 2024 •

edited

Loading