OCaml.org blog

tarides — 2023-07-14T05:47:25-00:00

OCaml-CI started with the goal of making a better continuous build system for OCaml projects. When we began in 2019, the goals were clear: it should provide a zero-configuration experience for OCaml projects using opam and Dune, and it should use an incremental architecture to avoid expensive recomputation of builds. We're delighted to announce that we achieved these goals, and OCaml-CI is currently tracking over five hundred repositories and processing over a hundred thousand jobs daily. This is inspiring news to those already using OCaml-CI or developers looking for a CI solution for their OCaml project.

+https://ocaml.org/feed.xmlOCaml.org blog2023-07-14T05:47:25-00:00https://tarides.com/feed.xmltarides

Throughout 2022, the Tarides CI team worked on rennovating OCaml-CI, focusing on improving the usability of the website, adding build history for branches, supporting new platforms, and launching experimental build support. We will cover all of those things in this blog post and hope you find them useful.

There is also a Discuss thread on CI Best Practices.

What is OCaml-CI?

diff --git a/data/planet/0branch.xml b/data/planet/0branch.xml index 33f6dd7250..f910b32583 100644 --- a/data/planet/0branch.xml +++ b/data/planet/0branch.xml @@ -1,5 +1,5 @@ -https://blog.0branch.com/rss.xml0branch2023-07-13T05:47:28-00:00https://blog.0branch.com/rss.xml0branch

+https://blog.0branch.com/rss.xml0branch2023-07-14T05:47:25-00:00https://blog.0branch.com/rss.xml0branch

vile 9.8u

diff --git a/data/planet/ahrefs.xml b/data/planet/ahrefs.xml index 39a5e9894c..20cd6b1637 100644 --- a/data/planet/ahrefs.xml +++ b/data/planet/ahrefs.xml @@ -1,2 +1,2 @@ -https://medium.com/feed/ahrefs/tagged/ocamlahrefs2023-07-13T05:47:24-00:00https://medium.com/feed/ahrefs/tagged/ocamlahrefshttps://medium.com/media/aef5aa9cb9b71aaafb0a07cb48cb312d/href

Transcript

David: [00:00:00] Hello, my name is David. I run EmelleTV. It’s a talk show about OCaml, ReScript, and Reason. I often bring guests from the community to talk about them and meet them and asking a lot of questions about the language or what they’re working on, and of course having fun with Hindley–Milner type system. That’s part of the show. I work for Ahrefs, it’s actually this company.

Today I’m interviewing a coworker, so it’s going to be a little bit a branded stream. Hope you understand. It’s a lovely company. Apply if you’re looking for a job to work on OCaml or Reason. Aside from that, I maintain styled-ppx and implementation of React on server, but that’s just enough about me, and I’m going to introduce our guest, Louis. Hello, Louis. How are you?

Louis: [00:00:59] Good, and you?

David: [00:01:01] Good. Very good. You obviously work at Ahrefs. At what team do you work?

Louis: [00:01:10] This is recently changing, but I have been in the back-end forever, since like seven years ago and I still have some of the projects that I had when I joined. This was stable. I’m in this new team called middle-end. Ahrefs is not very good with naming. We say that it’s the hardest thing in computer science. We have front-end that is actually full stack, then we have a back-end, which is more like data, and now we have middle-end, which is somewhere in the middle. I’m supposed to lead this new middle-end team.

David: [00:01:48] Nice.

Louis: [00:01:49] We’ll see how it goes.

David: [00:01:49] Nice. Today I think we’re going to answer a few questions about Ahrefs, I think has been a mysterious company, if you look at it from the outside. When I joined, I think you helped me understand a lot of things that I didn’t know about Ahrefs. I might just fire the same questions that I did, just recorded so everybody can understand them. Aside from from Ahrefs, who are you and can you present a little about yourself?

Louis: [00:02:21] This is a tough question. Who am I? I grew up in France. I’m French, still I’m French, but I’ve been living in Singapore for seven years, with Ahrefs for seven years. I’ve been working in OCaml my whole life basically because my first job was in OCaml, and Ahrefs which is my second job, is in OCaml too. I cannot say that it’s better than the rest because I never tried the rest. I’ve been involved with OCaml, like the OCaml meetup in Paris for some time.

I’m on the online community. I’m part of the OCaml Code of Conduct committee, which is an effort that was started last year, I think. So far we don’t have a lot of work, so that’s good. I’m one of five doing this. Then outside of that, I’m a pretty normal person. I’m 31 years old, and that’s about it, I would say.

David: [00:03:35] That’s the whole idea. You have been been writing OCaml for a long, long time. That’s fair to say.

Louis: [00:03:40] I’ve been writing OCaml since I’m 16 or 17 was when I wrote my first line, like 14 years ago, something like this. There was no Merlin at the time.

David: [00:03:52] There was no LSP. The first question is, 15 years, this is a long time, but how do you see the evolution of entire language? Would you split it in chunks? How have you seen the progress of the language?

Louis: [00:04:08] It’s hard to say. When I joined the OCaml world, it was because of people who nowadays are fairly important like Gabriel Scherer, who’s working in Inria, I think, but he’s one of the main maintainer for OCaml. He was in this French forum, pushing very hard people to try OCaml and I got convinced. I started just writing a few lines here and there, and then I just stick to that for a long time. I’ve been mostly a user like this. My usage has extended over time, but I’ve never been called a contributor. My point of view is more as a user.

The biggest difference is the size of the community, I would say. It’s much more dynamic than it used to be. We used to install packages like OCaml libraries, using Debian packages. It was apt install something. There was no opam, there was no Merlin, there was no vscode, right?

David: [00:05:25] Right.

Louis: [00:05:25] LSP didn’t exist, so way less library. I think you can see today how it was in the past because you can see, we have 10 different libraries to do HTTP 1.1. We have 10 different standard libraries, and it’s legacy of what OCaml was in the past. We used to have all those smart people, but they had no way to collaborate. There was no opam, there was no way to share your work. Everyone was smart enough to rewrite —

David: [00:06:04] Build your own library for http.

Louis: [00:06:05] Yes.

David: [00:06:06] Right. That’s always interesting for me, how OCaml got so many different things that are hard to create, like standard libraries. I think recently, Containers reached 10 years, so it’s like what? [chuckles] It doesn’t make much sense. If you look at now, it doesn’t make much sense, but if you look at historically, it does make sense.

Louis: [00:06:35] When you have no choice, you do it. You don’t know that it’s harder, you just see “Oh, I can do it,” and you do it.

David: [00:06:42] Yes. You need to think it’s easy. You need to start a standard library or http library saying, “Oh, it’s easy.” Then, you start a little bit, and eventually, you create something. Last week — Oh, sorry, go on.

Louis: [00:06:57] No, go ahead.

David: [00:06:59] Last week, I tweeted that OCaml suffers a lot from the Python Paradox. The Python Paradox, I think somebody write it, I think it was, I don’t remember the name of the guy, but somebody write it in 2004, that when you use Python for a company, and you get the smartest people that they want to innovate, and you have the pioneers of the language. Then, by default, you try to hire people that are in love with software, so eventually, they create good software or they are willing to suffer from getting out of the comfort zone and create software. I believe the combination is the same spot. Can you see, is that true? Do you agree?

Louis: [00:07:49] I don’t know if that’s true. Partially, it’s a strategy of Ahrefs, so I have to say it’s true.

David: [00:07:55] [laughs]

Louis: [00:07:58] Yes, I think it’s partially true, but it’s not completely correct. For OCaml, at least it’s a bit different from Python, because OCaml has this strong academic influence, so a lot of people are actually researcher. There is a big benefit that they have, not free time, but they manage their time, they manage what they work on, and they decide what is important. They got all this time to actually write this complicated code many times because actually, it’s part of their job to just redo the same thing in better ways. It’s normal you have to explore a subject.

It’s okay to do it multiple times. It’s a combination of those people are working in the right place, they have the right time, and they have the correct background. A lot of people were working on subjects that allowed them to do it. Some, it’s because they were very strong in writing languages. Some, it’s because they have this strong Unix background, I would say.

David: [00:09:05] Right. It’s very unixy. The start of OCaml is very unixy. That’s true.

Louis: [00:09:14] Xavier Leroy wrote LinuxThreads, I think that was used in Linux forever, so there is this background. Probably, it’s a bit different in Python because it grew more in the industry rather than in an academic setup. For sure, if you try to target people who live in a niche, you find people with a different interest from the main programming community, I would say. At the same time, I think some of the best meetup or conference I’ve been to were Java meetup.

David: [00:09:59] All right.

Louis: [00:10:00] They know that their language is boring. The language is the same since 20 years or 30 years. There are some changes but they don’t really care about the language too much. It’s a huge, huge community. Basically, everything already exists. There is no big bragging, everyone can do everything. There are 10 versions of everything, whatever. The benefit is that they are super open-minded.

Oh, something is new. Something is different. Okay. Let’s see that. You go to that meetup and they will not talk only about all the fancy new feature in Java. It’s like, “Oh, I saw this new git tool. Oh, it’s funny. Okay, let’s try to use this.” A new way to do web development. Okay, let’s study the subject. It’s not about Java itself because the language is not interesting enough. It’s about other programming stuff. It’s very fun to attend.

David: [00:11:03] That’s a good one. Actually, many people that now are starting to hear OCaml for the first time, after they leave or they got disappointed with the Rust policy drama. These people cannot get into OCaml because some influencer wanted to bash on Rust. They started exploring all the languages and of course Ocaml was one of those. Aside from Zed or whatever you compare it with, low level programming languages. Do you see those influencers move people to actually try the language and deploy it into users and doing serious stuff, or it’s more like vain marketing?

Louis: [00:11:56] I’m not a big Twitch person. I don’t know [crosstalk] modern influencers.

David: [00:12:02] That’s true.

Louis: [00:12:03] I think it exists in two forms. In the past it existed in two forms. You had influencers, you had Rob Pike and- who’s the other person? The two person who are behind Go. They do not make a good language but they are influencers. They were like, “Oh, yes. We did UTF-8 and a Plan 9 in the past and we work at Google. Oh, it’s going to be amazing.” No, it’s a crappy language, but they are influencers. They move people.

David: [00:12:32] [laughs] Okay. Would you say that Go is crap?

Louis: [00:12:38] Go is a language. I haven’t used Go 2 extensively.

David: [00:12:42] This is recorded. This is not a beer in Singapore. This is recorded. You can obviously bash go, that’s part of the game.

Louis: [00:12:50] Let’s say Go is not the most modern language there is.

David: [00:12:54] Right. Thank you. This is just for the headline. We don’t want the headlines- because Ahrefs is going to be like — No, I’m joking. [crosstalk] Yes, go on. Sorry.

Louis: [00:13:10] On the same topic of influencer, we saw it with Reason. When the Reason comes, it’s not just a random person creating Reason. It’s Jordan and he comes with a React background, and he comes with followers. He is not doing videos online but it’s the same idea. I think yes, it has definitely an influence and OCaml grew a lot when Reason started.

David: [00:13:38] Yes, that’s true.

Louis: [00:13:39] I definitely think it has an influence.

David: [00:13:43] That’s true. From the community, how have you seen the Reason creation and adoption from your point of view? You can bash Reason if you want.

[00:13:43] [crosstalk]

Louis: [00:13:56] At that time I think the OCaml community was one IRC channel. It was a bit different from now. I think what I was not super convinced by when Reason to syntax arrived, I think the original claim by Jordan was he’s making a better syntax. I was not super convinced that the syntax was better. It was developed independently from OCaml.

By experience I already knew at the time that if you start to fork or develop on your side and don’t integrate fairly quickly with upstream it’s actually never going to be integrated with upstream.

David: [00:14:53] Right.

Louis: [00:14:54] I don’t know why exactly, but it has happened a few times. Then there is a question of bucklescript because if you write Reason it’s two sides. There is a syntax which I only partially understood too because I was not a web developer, I’m still not a web developer. I did not know about JSX. I did not know how powerful it was and I think React was not as big at the time too, but I think JSX is a nice idea and there are a lot of things in the syntax that are nice, like parentheses around arguments is a problem but it has some benefits, too.

David: [00:15:42] Yes, I think some trade-offs from OCaml, or at least some edge cases from the syntax from OCaml got resolved in Reason just by adding more- like the parentheses or the braces or the semicolons. But yes, the others are, can remove some problems from the syntax. Not problems, but just the edge cases from the cleanness for OCaml.

Louis: [00:16:10] Yes. Even sometimes it’s not edge case but it’s nice to see very clearly for example, when you apply a function, where are the arguments? Where it starts where it ends? There are benefits, obviously, like the OCaml syntax or Haskell syntax is lighter, we will say, have some benefits. The other one is nice, too.

David: [00:16:36] Yes, definitely and you mentioned BuckleScript?

Louis: [00:16:40] Yes. BuckleScript, they didn’t have —

David: [00:16:42] That was not so well received.

Louis: [00:16:47] Yes, I think because there was Js_of_ocaml idea. There was js_of_ocaml and so again, it was like yes, I do something different and- I think Bob developed it fully inside Bloomberg at the time. Basically, he came out and he had, “Oh, yes, I have a new project and it’s working already.” He didn’t start to develop it in public. The community was much smaller, too so every time you split efforts like this, it’s kind of costly. People will say, “Yes, we will try to collaborate. We’ll try to make the two projects work together,” or whatever and it never works. Never works. I don’t think I understood all the trade-off. I’m happy that I invited Bob to the OCaml meetup in Paris which retrospectively, it was a good thing to do.

David: [00:17:50] Yes, for the record, Louis was running the OCaml Paris Meetup, I think. Yes, go on with the story.

Louis: [00:18:00] Yes, so when I moved to Singapore, I still organized one meetup, even though I was in Singapore and I invited Bob to present BuckleScript. At the time, it was a bit controversial, because many people were a bit unhappy with what he was doing, but I’m happy that I did it. I didn’t understood what I was doing exactly but at the end, I think it was the right thing to do. Even if the project died later on, you have to give such projects a chance.

David: [00:18:35] Yes, I think I wouldn’t say that BuckleScript died. It’s more like BuckleScript has been working for seven years, I think.

Louis: [00:18:42] Yes, no, even if it was a failure, which it was not, but maybe like six months later, it could have died and disappeared. Yes, I think when people have a drastically different approach, usually they have a reason. It’s worth listening. A lot of what Bob defended, I’m not sure I completely agree with it. He wants a very stable compiler, for example. He said, in Bloomberg, they are using GCC 3 or 4, I don’t remember, since forever.

So they don’t need to upgrade the compiler, the GCC compiler, for example. He thought the same idea can apply to OCaml, we don’t need to follow the upstream compiler all the time.

David: [00:19:27] Right. Yes.

Louis: [00:19:29] Most companies actually they don’t want to change compiler version. They want something stable. They want no surprise, which has some value, or the stability has some value.

David: [00:19:40] Yes, that’s true but I think when he mentioned about the compatibility with the compiler, I think it’s mostly OCaml has been very stable since, what 6, 7 years ago, I think. I think there were some small changes or some addition features, but nothing really break, but mostly the syntax. Then he complained about the parsing, like the AST modifications, those were present, those were changing between versions. He wanted to not- because BuckleScript is a fork of the common compiler and embedded into ReScript now.

Yes, he was complaining about the AST transformations because every version changes a lot. There are migrations. You could write some logic to migrate from one to another. It’s painful if you maintain a fork of this, you might suffer a lot from updating from one compiler to another.

Louis: [00:20:41] Yes, and I think for him, even as an end user, the stability has some value. It’s interesting for him to have a stable compiler and even for his target, the people he’s targeting, the stability has some value too.

David: [00:21:00] After you mentioned that people were installing or sharing libraries through Debian packages, which maybe- I’m as old as you, but maybe I’m too young to see how those package managers could work with apt get. What’s the position of the tooling? Right now I think we are in a state where we have two bigger players such as opam and dune, as Package Manager and Build Infrastructure, we’d call it, I don’t know. Now dune is exploring installing packages. How do you see the tooling these recent years?

Louis: [00:21:48] It’s amazing. It’s completely incredible. Then people will have different opinions on is opam perfect or whatever. If you compare to what it was before, it’s incredible. I think even if you compare to other languages, it’s a fairly solid experience now. Opam is working well. You just need to learn the UI, but it’s working fairly well. Dune is relatively fast, easy to use. The LSP is pretty magical. Merlin is a very solid tool. It was one of the first, I think I would say, like a small language with a tool as powerful as Merlin.

It’s not only powerful, it’s avantgarde. It understood already that you had to be able to do error recovery and that you had to change the way you parse files to be able to work with something that is half broken.

David: [00:22:57] Yes, that’s true.

Louis: [00:23:00] The people behind Merlin are super smart. In a way, it’s not a surprise.

David: [00:23:05] Right. You actually contributed to the LSP and dune, to both projects, I saw your contributions.

Louis: [00:23:13] Yes, I have commits on many small- it’s mostly small contributions, but I have commit on everything, I think at some point.

David: [00:23:19] Right.

Louis: [00:23:21] LSP, I participated in putting some ppx deriving stuff and I wrote a bunch of commands. I implemented some Merlin behavior inside LSP. If you hover multiple times on the same value, the type will be more and more verbose. I took this behavior back to LSP. Dune I have mostly bug fixes, probably small documentation, nothing big.

David: [00:23:58] One of the things that you mentioned as well, I think we talked about this before. When OCaml was very young, all features that got added into the language were PhD projects, where it’s a student that is very passionate or maybe just his guidance is OCaml fan, he just explored with a language in the theory on academia. Then he worked on a paper and eventually it gets released as part of the language. That was the times where maybe Jane Street not even started using OCaml seriously. Do you see that now? Do you see that those features or academic features got into language? Do you think it’s a weird mix now or they compose well together? How do you see language after these contributions?

Louis: [00:25:04] I was looking today at the OCaml change log because I was wondering when was the release of OCaml 4, and that was 11 years ago because OCaml 4 is- before to OCaml Multicore is the last time there was a big change which was GADT. In the meantime there was mostly small changes. I don’t think the language changed much. If we look what were the big features we could say like the objects in OCaml, GADT, OCaml Multicore.

They all were developed by people in a research setup and somehow it seems to work. I’m not a maintainer on OCaml. I think it works also because they don’t have a lot of energy to integrate a lot of new features, they are very, very picky on what they actually accept in the compiler. Only the most solid implementations will get in.

David: [00:26:36] Yes, that’s true. I think the quality is something that everything core team members says all the time that all these things would be amazing to do but our quality bar is very high. Yes, you need to work on it much more to let us just even look at it. Yes, that’s true.

Louis: [00:26:54] Then there are things that do not compose super, super well. There are part of the module language and part of the object language that do not compose very well. You can make the compiler more or less blow up or the compilation time will become crazy. Actually, those are parts that I don’t know very well. I very seldom combine first-class modules and objects.

David: [00:27:21] Objects, yes. That’s something I haven’t done yet. I think the only experience with that combination might be ppxlib maybe, because you have the traversers. Yes, you use them. You instantiate the traverser. You don’t do anything with internal states of anything. Good point. One thing that maybe it’s worth saying is that right now you work at Ahrefs for seven years.

At the beginning when Ahrefs pick OCaml or Igor, our CTO came with OCaml in the back, there were not many companies working on- using it, using OCaml. Now we have Tezos, Tarides, Ahrefs of course, LexiFi, Bloomberg, BeSport. Many companies that have- even some of them have their own forks of OCaml that they are experimenting and deploying it or whatever. Seven years ago, do you think it’s a risky decision? The second question is how can you convince your boss about using OCaml?

Louis: [00:28:35] For sure, I think picking OCaml at the time was a risky choice because who do you hire? It’s like there were five OCaml developers. In Paris, you can find people. In Paris, you can find students. You go to the OCaml meetup and socialize and you can more or less build a company, which is what the previous company I was in called Cryptosense was doing. This is how BeSport came to life. BeSport just picked a few people around Vincent Balat and then you meet people. You steal one or two person from the OCaml meetup and you tell them, “Oh, join my company,” and now you have enough people to push a project forward. How do you do this from another country? Even today, I think it’s not an easy choice.

David: [00:29:35] Somehow risky, yes. That’s true.

Louis: [00:29:38] Today, you can hire, but even if you have, I don’t know, 2,000 packages on opam, the tooling is still- the libraries are not, there are not libraries for everything like you have in some other languages.

David: [00:30:00] Right. It’s big enough, but it’s not populated with everything.

Louis: [00:30:07] I don’t know if we have full support of GRPC. I’m not sure that we have complete support of http2 or 3. It’s not that small, but many things like this. I would say, today I would say it’s a risk. How would I convince my boss to move to OCaml? I would —

David: [00:30:34] Would you do it? Maybe you would not do it. Maybe you say, “It’s fine, we can do with whatever,” with Java you said that you enjoy the Java meetup, then you join your company writing Java. Would you be happy writing Java? Would you be fine? Or you would say “Oh, OCaml here makes sense, let’s try to change it.” How would you do it?

Louis: [00:30:57] I think if I was in a small company, it would definitely make sense to use OCaml. It’s interesting because in a small company you could say all the Java tooling has more value than in a big company, but at the same time you have less hands. You need to be more productive per person and you have less time for maintenance, and those are two things for which OCaml is very strong. You can write few lines of code that do many things, so it’s very expressive. At the same time, it’s solid enough that when you write your code, you can launch it in prod and you can leave it there for some time and hopefully nothing breaks.

The language is stable, the compiler is stable, so there will be no big surprises. I think that’s very valuable, and then you compare, what are the alternatives today? Rust is incredibly hard. It’s very, very hard language to use. You can do fancy stuff, you have incredible community but it’s a super hard language to use. You have what, Python, but then you are losing all the type safety. You have Go, which is a bit in between those. You have a fast Python I would say. Then you have Java. Java which has a huge community, and is a fast language.

In a way I think OCaml is closer to Java. It’s one easy language to use, solid, no surprises. The feature set is not incredible but it’s working well enough and you can do more or less what you want with it. You can do work in the back end, work in the front end, it’s approachable. To me it’s a replacement to Java. It’s a light Java.

David: [00:33:04] I’m mostly front-end. Now I’m doing some back-end stuff but I mostly am experienced from front-end. You are experienced from back-end of course and when I’m talking to back-end persons from OCaml, every time I talk with a back-end person who only writes OCaml they mention then the runtime. From the front-end, it’s a problem that I have never, ever thought. I know that the problem exists because I studied computer science and all these things, but it’s something that in the front-end I never think about it.

How could you describe to me that- I know a little bit about the memory presentation and about the stack, the heap, how memory works, even the O(n) notation, O big notation. How can you describe the runtime of OCaml, from someone who doesn’t know much about runtimes, so has nothing else to compare, rather than notes, for example. That’s my experience.

Louis: [00:34:08] Yes, I’m not an expert either, but it’s an interesting point, actually, because if you go on, for example, the real-world OCaml, there is a whole chapter on the runtime. I think it’s important for the OCaml people because of their background. We have those unixy people, so they have experience with C before and because in C you need to know what is a representation in memory of everything you manipulate, they took that from C and bring it to OCaml. Those people, they like to know, when I have an integer, it’s going to be nowadays, 64 bits.

David: [00:34:56] 63 right? That’s the —

Louis: [00:34:59] Yes, one bits for the right GC, and then we have 63 bits for the value.

David: [00:35:05] Right. Yes, people love the runtime. I think it’s like those things when — You guys started talking about the front-end. For me, I love CSS. I can talk all the time about CSS, but if you never have experience with a language or with designing the UI, CSS means nothing. You understand what they are saying because the thing makes sense, but semantically it doesn’t. When you talk about the run time at the beginning, for me it felt like I have never, ever thought of this.

Louis: [00:35:41] I guess it’s two sides. There is the technical side, how it’s actually implemented that when you allocate a value, where do you put it in memory? What is the representation of that value in memory? For example, we said that the int are 63 bits actually, that when you allocate the value, you allocate by words in OCaml. You have one word, for example, if you allocate the values that is on the heap, you have potentially two words. You have one word, which is a pointer to the actual value, and then the values, which is like a number of words afterwards.

You have the GC, so when is it triggered? Actually, the GC can be running every time you can allocate a value which means that you can write code that will not trigger the GC. It means you can write code that is very fast because there will be no interruption, and I think that’s critical for companies like Jane Street. Then yes, the other side is the runtime from a user perspective. I see it two way. I see one way that it’s like no one knows about the runtime because it’s very, very simple in OCaml. You don’t need to deal with the runtime very often.

You just know that you pass values by reference, so you don’t make many copies, and then the GC is fairly fast and will not stop for too long. That’s probably what 99% of the normal OCaml people know about the runtime. Then an interesting fact that comes with that is that the OCaml compiler is bad by modern standards, that it’s not doing any kind of optimization or very little optimization, yet the native code is fairly fast. The native code that is generated for an OCaml program is fairly fast. I think if you look at the benchmark it’s not too far away from C++, which is surprising, and it means that the language is pushing you to write code that by default is fairly efficient.

David: [00:38:06] That’s exactly my experience.

Louis: [00:38:09] The types that are offered and the functions, the APIs that are offered, somehow allow you to write code that is not too, too bad. I think it’s a miracle, but it’s an interesting one.

David: [00:38:23] Yes, that’s exactly my experience. At Ahrefs, the formula of the Coca-Cola of Ahrefs is like the crawler, the thing that navigates the internet and saves data. After that, we have storage and all of these pieces that are complex. What can you explain about the secret sauce of Ahrefs? What are they from the outside? Many people would never, ever write a crawler or a very dummy one, but for one that indexes 9 billion pages, 1 trillion? I don’t know the numbers but insane amount of numbers. What can you explain?

Louis: [00:39:10] I guess the first question is what is a crawler?

David: [00:39:13] Yes, yes, because you read the webpage, you scrap a webpage, that’s fairly simple. You can do it in any language, but then what do you extract about this page, and more importantly, how you navigate to the next one. I think that these are the two main questions.

Louis: [00:39:35] What you extract, depends. Ahrefs, we care about the links. What is Ahrefs building is more or less a map of the internet. The crawler is running all the time. It’s downloading, I don’t know, like 4 million pages per minute or something like this. There is a counter. Every minute we crawl 5 million pages. We have been talking for 40 minutes. You can count how many pages we have downloaded in a period of time. We download those pages and then we extract the links. That is the main information we care about. This is not the only information.

First is how do you parse HTML and how broken is HTML on the internet? This is horrible. The internet is broken. You have to extract all the links in a page and then you have to store all those links. When you store links, because — What is a crawler exactly? Where does it start and where does it end? Is it only the part that is downloading the html? Or is it actually the parsing too, and it’s influencing how you are storing your data, because — Let’s say you download a page and you have a 100 links in it, you do at least two things with those 100 links that you want to reuse them in your scheduler to decide what do I crawl next.

You also want to update counters, because you want to update your map of the internet. You downloaded a page, you know that there are links and you want to update the map. How do you update the map, because you have a 100 new links? What do you do? You update a 100 small counters, a 100 small integers. Then can you do it 5 million times per minute? Then can you do it in many direction, because it’s a graph.

David: [00:41:48] You would loop. If you don’t do it properly, you would loop forever.

Louis: [00:41:55] You have links between pages, but then you want to also count links between domains and you want to count the links inside the domain. Then how do you decide it’s an interesting link or not? Then when you index a link, what do you index? You need to index the link itself, but you want to index the text that is attached to the link, maybe the paragraph that is around that specific link. You could look where it is in the page. Is it visible or not? You have — It’s an open question.

That’s an interesting question, because there is no one that can say, “I’m doing a crawler and this is the right way to do it.” Even big companies like Google, they make tradeoffs. They decide, “We do it in one way.” Then they gather information they can gather. They cannot download every page on the internet all the time. They cannot download and process stuff fast enough. There is more content that is created than content that can be downloaded.

David: [00:43:00] It’s interesting, because if you think about fixing a bug on the crawler, it’s usually when you have a database, you can run migrations or you can get out data. You can store data broken or whatever. You can fix it. If you have the history of internet, that’s another source of data. It is life. I don’t know the right metaphor, but it’s — If you have to fix a bug on the crawler, that means that you stored information wrongly. That can affect the next version of your map, because it’s not only a map, it’s a map and a timeline. You can just look it up. It’s an archive as well. I think internet archives, they don’t have a crawler. I think they don’t have a crawler.

It’s the idea of — You can improve the crawler. Something that you didn’t look before, now you are going to look it up now. I don’t know, when — I think at some point we — At the beginning, either we started indexing, I think videos was — I don’t remember. Some media, I don’t know. That, of course, blowed up immensely, everything- they complicated everything.

Louis: [00:44:14] Yes. That’s an interesting question. Actually, Because, it is right that your database is very big so you cannot just migrate stuff. The big data page says we have 170 trillion rows in the database, so we cannot just push this to somewhere else.

David: [00:44:35] First of all, what technology is that running on?

Louis: [00:44:40] I think that’s a combination of different technologies. That will be a ClickHouse and then some internal database. Custom stuff.

David: [00:44:52] At Ahrefs, correct me if I’m wrong, we like to build our own things mostly. When I have it in other companies, you would use Sentry for reporting, or you would use PagerDuty for live crashing, or you would use whatever tool that you- or a web server, like a framework that runs your server. I think we implemented all of this by ourselves. That sounds both crazy from the outside, but when you join Ahrefs, if you ever join the company, you understand perfectly why has it been done like that. Yes, we have our own database. It’s scary.

Louis: [00:45:41] It’s not completely our own database, it’s more like a wrapper around existing database. It’s partially because we have no choice. The problem is large enough that you don’t have a ready made solution. Google was like this for a very long time. They had MySQL, I think that they used very extensively MySQL and it’s just that they used it in a way that was working for them. They don’t have a giant MySQL database, but probably they just sharded the problem.

They have one small database per server and they have a smart way to send the tasks to the right server to retrieve the data they want. Because you have to build on top of something, we are a small company. The total number of employees, I don’t know, it’s 100 plus now, but the back-end team is still 15 people or something like this. We don’t have too many hands.

David: [00:46:43] Yes. That’s insane.

Louis: [00:46:47] You ask what you do when there is a bug in the crawler and it affects how you conceive the programs because you know that something will run forever. The strategy becomes, I don’t want to fix bug by hands. It’s, you have an auto healing index. You crawl a page for the first time, and let’s say you make a mistake. The number of links you counted is off by one. You know it was like this for three days because you deployed, it was broken. Three days later, you notice it, and you cannot go back in time. It’s already too late.

Instead, what you do is that you fix your crawler. The way you store the data, you make sure that the next time you crawl the page, it overrides the previous version with something that is correct now. You have to have those auto healing processes, and you cannot attend to every small detail by hand, and the full rebuild of the index will be the last resort. Only if you have absolutely —

David: [00:48:01] Did that ever happen?

Louis: [00:48:06] It partially happened. Not everything, but there are things that were rebuilt once in a while. We were storing two things because we download pages, we download the HTML that we store, and then we have two counters. We have the counters we extract from the page. Let’s say you have one link that you see twice in a page. You have this link and the number two attached to it, and then you have diff. You store diff, let’s say, because you downloaded that page that belongs to the domain ahrefs.com. Now you see that that specific URL, for example, has three links that were not present before.

You store somewhere plus three, and later on you will aggregate all those plus three, plus one, minus one together. There are two different things. You have the absolute numbers and then you have those diffs. Once in a while, we have a bug that we didn’t compute the diff correctly. Then we will rebuild the diff from scratch. We will go back to those absolute numbers, process them altogether and then restore it. Then when it happens, it can take a month, but it hasn’t been done in a long time. It’s a long process.

David: [00:49:28] Right. That’s interesting.

Louis: [00:49:32] This is where OCaml is shining, too, because it’s very easy to have multiple versions of the same type, for example. If you store data, with a version number in the database, you have a variant.

David: [00:49:48] Yes, you treat it differently or?

Louis: [00:49:50] It’s fairly automatic and- yes.

David: [00:49:59] Yes, as well we have diffing on HTML. One of the big features that we did, I think that was last year, that we have diffing for the content of the page, or the diffing of links, we have as well diffing of content. That’s very good.

Louis: [00:50:18] We have a lot of small funny features. We are one of the first company after Google, obviously, to render pages at scale. We have hundreds of servers running chrome, and as much as possible, when we download the HTML of a page, which is the raw HTML, then we will put it in Chrome, let it run for a while and then get the rendered version of that. Which is incredibly expensive time-wise, because it’s much harder work than just downloading the HTML. We couldn’t do this if there was no project like Chrome that would be open source and usable for free. We are standing on the shoulders of giants for this.

David: [00:51:08] Imagine trying to create a web engine just to see a page from the server. That’s very good. What’s the favorite part of working at Ahrefs?

Louis: [00:51:26] To me, is the people. I’m not a SEO expert, and I’m not a SEO fan either. I didn’t join Ahrefs because I love to study the internet, that was not my goal, and I’m not a marketing person, so I don’t have a big use for SEO by myself. At first when I joined, the technical challenge was fun, but nowadays to me, the value is more the people. You get to meet smart people who work on complicated projects. I spend a lot of time dealing with interns, for example, which is super gratifying, I feel. I try to spend a good amount of time sharing with other people, working on the tooling or stuff like this. I really like that part, you can see the influence you have on other people when you make their life easier.

David: [00:52:32] That’s good. What’s your favorite part of OCaml?

Louis: [00:52:37] Of OCaml?

David: [00:52:38] Yes. You can say the people.

Louis: [00:52:41] When it compiles, it works. That’s the key point. Unlike Go, we have some types.

[00:52:41] [laughter]

Louis: [00:52:59] If we are a bit serious, maybe not the language itself, but the LSP is super, super good nowadays. It’s definitely a very good experience. We have to thank Tarides for all the work they are doing on the tooling over the past three years now, because it’s crazy.

David: [00:53:21] I think Tarides is carrying most of the boring work. Not boring work, but work that is always hidden. That you don’t get any fame, but you will only get the trash. When the toolings don’t work, you complain, and when tooling works, you just don’t celebrate it. Tarides is behind, for sure. You have been going to ICFP, ICFP is international conference for functional programming, for quite some time. One of the verticals, or one of the parts of ICFP is OCaml.

Last year we outgrow our neighbors, our language neighbors, I think it was Scala, maybe, Haskell. I think we outgrow them. What do you think about the conference?

Louis: [00:54:20] Same thing, it’s amazing to go there and meet the people actually, because ICFP, it has multiple parts. The main track is a bit more academic. Even though some people who work at Ahrefs, they published there, but they were students when they did it. It’s a bit more academic. Then you have all the workshops that are a bit more approachable, at least for me, but you spend one week with smart people who are very excited by what they’re doing. This is the amazing part. Once a year, everyone is super happy to meet each other. This is very much a good experience, but it’s- about conferences, I think the ReasonML ones were very, very nice too.

Again, I’m a back-end person, but it was super cool to attend a conference on a different topic where people have different interests, and it was the early days of the language for people who were super interesting, the people who attended were curious and wanted to see something new. They had different ideas. I think that was super good.

David: [00:55:36] Yes, those conferences were very good. I haven’t attended any, actually, but yes, I heard Javi saying amazing things about them. Almost everybody who attended said good things. Why there’s no OCaml conference?

Louis: [00:56:00] This is a multifold answer, because it’s actually a question that was asked. There is this, how is it called, the OCaml Software Foundation, because if we can explain how OCaml, the management of OCaml is that —

David: [00:56:23] Please do. As core contributor of the code of conduct, please do.

Louis: [00:56:27] There is the core group of contributors for the language, and out of that before there was something called the OCaml Consortium, I think, where companies could pay few thousand USD a year and it will give them a license to use OCaml not as an open source project. You could get the compiler and do changes on it and you didn’t have to publish the changes again. It was also a way to just sponsor the OCaml development. They took it to a different level. They created this OCaml Software Foundation that is pushing some efforts around OCaml.

One question was, do we want to have an OCaml conference or do we want to have OCaml- maybe not conference, but smaller events but that could happen more often. Where will we put those events in the world? You need people with time, you need people with money, and you need to find the right place for the right people to attend. I think no one has all those resources, including the mental space to build fancy ideas on what to put in a conference. I cannot provide a definitive answer because I’m not the one deciding on all those things, but I think it’s a combination of all those that makes the ICFP the place to be.

David: [00:58:05] Right, because this year is on Seattle. Every year it changes the location.

Louis: [00:58:13] Yes.

David: [00:58:16] Last year we released we- I didn’t but yes, OCaml released multicore and effects or handling effects. We chatted a bit, a lot at work about this, and I think you said multicore was not something that needs to happen, but you are not very excited. On the contrary, you said the effects are a big deal. Thinking about the person that doesn’t know a bunch about what effects are, could you do a short summary and then explain why those are exciting?

Louis: [00:59:01] I’m not a specialist with effects either but to me, a parallel will be to talk about Rust. In Rust, you have those ways to, how is it called? Borrow checker. You have a way to know to who one value belongs. It heavily affects how you are writing code because then you need to architecture your code in a way that is safe. Do you know that, for example, that value can only be used by one bit of code at a time?

David: [00:59:45] Right. Otherwise, you would have crazy bugs. Data corruption

Louis: [00:59:56] That would be C. Rust came in and was provided this safety. That’s a bit the same idea in OCaml. It’s like you come with effects and I think it has many usages that I do not completely understand, but some of them allow us to change the way we do concurrent or parallel computations, and it makes it safe. Like the borrow checker makes the Rust code safe. This is definitely affecting the way you write code because now you have one more tool to express your ideas. I think this is definitely changing the way the language will be used.

While the multicore, it’s just in the background, it is happening, but this is not the tool. This is just a mean, so it’s like, how do we do fast computation? Do we need to split stuff on different cores and how do you do it? Either you fork or you do multicore. In a way, it could be completely hidden behind a magic API and I will not know if it’s fork or multicore, and it’ll be fine to me as a user. If you have something like the borrow checker in Rust, this is actually a language feature, and this is something I see day to day and it is affecting how I can think and what I can express.

David: [01:01:36] Right. I see, I see. Yes, because right now one of the features that Jane Street, the famous company that does the Wall Street and whatnot and pushes OCaml for the next level, they have a team working on the OCaml compiler, and one of the big fears that they want to work on, I think, they call it locality or local, global variables. Would that express, I have no idea about those rather than watching Stephen Dolan at the presentation on last ICFP, but would that allow some of the users of OCaml that they do care about the memory layout or the owner of the variables to express those different changes regarding using multicore?

Louis: [01:02:34] I don’t know exactly — My own — I mean, I have a light understanding of that. To me, I think it would be interesting even if there was no multicore, the stuff they’re doing local/global because we already had concurrency with lwt or stuff like this. It has benefits because you control your allocations too. You can decide what is allocated on the stack versus what is allocated on the heap. It can have big performance implication.

This is an exciting feature, but this is maybe where you see my C background that when I was in uni, the first year was just writing some C code and we had to rewrite Bash. We had our own version of Bash. We spent two months writing in Bash or stuff like this. We had to deal with many of the small like — You are launching a bunch of processes together and you have to manage your memory or whatever, or we had to rewrite malloc, so we —

David: [01:03:53] Okay.

Louis: [01:03:56] I know little bits about memory management and how to deal with pointers because I did those projects in the past. These local, global things seems appealing, but at the same time, it’s probably not critical. It’s not going to change the vast majority of the code that is written in general. All my small personal projects or even most of the code that is running at Ahrefs, performance is not key. I care more about the fact that the code is readable and stable rather than performance, I would say.

David: [01:04:42] Yes. Only in somewhere else. Yes, I see your point. Usually, code, for example, just like a web API were way, way fast enough. There’s no point to optimize the endpoints. Most of our endpoints I think we have 500 endpoints. That’s optimizing one by one or optimizing 10% of them, they would not change absolutely anything.

Louis: [01:05:10] We spend so much time doing queries to different databases or http query to gather whatever we need to gather before to answer a request. This is so expensive compared to what we do most of the time.

David: [01:05:28] Right. That’s true. That’s true. Why do you think Ahrefs is such a different company? When I have experience with the — It’s because of the culture maybe? Here we don’t have real management, we don’t have product owner. We don’t have many things that when you come from working on the SaaS companies that are from the culture of US maybe, or some Europe companies. In Ahrefs we don’t have anything like that. Can you say that it’s good or bad and why?

Louis: [01:06:07] It’s good and bad. There are definitely some downsides. Why it’s like this, is also because the company is young and small too. It’s what? 10 years old so it takes time. Every time you want to make a change company-wide, it probably takes two years to actually make change happen.

David: [01:06:33] Okay.

Louis: [01:06:35] This is not the only company with a structure that is not well defined.

David: [01:06:44] Right.

Louis: [01:06:44] What happens is that there is a structure, it’s just people don’t have the title because actually when you have been in the company for a long time, you own some bits of codes and there are people who are expert on the subject. Then there are people that you trust for something and people that you trust for something else. Even though there is no direct management, there are people taking decisions, so who is taking the decision, right?

David: [01:07:16] Right.

Louis: [01:07:17] It’s good to be flexible and it allows more or less anyone at some point to take a decision if they want to and if they dare to. The downside is that sometimes you don’t know if you can take the decision or not, and you don’t know who you should talk to and then there are some hidden politics because some products, some features there, they belong to someone. You don’t want to offend that person so you can’t go and touch this or stuff like this.

David: [01:07:51] Well, I need to interrupt here. I think you have been way too long at Ahrefs to realize what politics — What it means because in Ahrefs there’s literally zero politics or not politics, but battles or discussions for the sake of discussion, it’s nearly zero. I think that’s one of the things that at the beginning of like, are we not talking about this and somebody said no need to. It’s the culture of getting very direct and very technical focus.

I think when you work in a company that you can be weeks without knowing what to do or just months working on so many processes that are close to useless from your point of view or maybe very beneficial from an individual contributor, you feel like you are losing your time. In Ahrefs I don’t think I have been noticing a layer I thought I’m losing time because of the company it’s the other way around. Oh, my peer is asking me to implement something that needs to be done and I haven’t finished yet. That’s more the feeling of the work, right?

Louis: [01:09:09] I guess it’s not politics looking for power because there is no power to gain.

David: [01:09:16] Exactly.

Louis: [01:09:17] What do you want to own? There is nothing to own. You can try, but there is nothing to win at the end.

David: [01:09:26] Getting to the last questions now, but have you been following a little what Javi and Antonio and a little bit of myself having worked in Melange. What’s your opinion about Melange?

Louis: [01:09:43] I know what Melange is.

David: [01:09:46] Definitely.

Louis: [01:09:47] Okay. What do I think about Melange? Again, it’s a question that is hiding other questions. Let’s say technically, for example, this is pretty impressive. What the four of you have been able to do in a few months is amazing. Because just to give some context, it’s like moving — Okay, Melange was not super, super alive, six months ago. The project was moving but slowly. There was no Dune support, there was not much stuff happening. Then six months later, you have the whole Ahrefs front-end, which is like, hundreds of thousands of lines of code that are written by what? 30 people maybe now.

It’s completely moved to Melange. This is amazing. I’m able to compile all this code in one comment. I go in the repo, I do “make dev” and everything works.

David: [01:10:56] These and many more advances, but yes, that’s the part that it’s funny.

Louis: [01:11:04] Yes, it’s amazing. It automatically works and it didn’t break the experience of anyone so it’s compatible with what was bucklescript or rescript beforehand. It’s compatible with native code at the same time. It’s amazing. What do I think about the project? Another side of the question will be, was it the right thing to do to fork rescript? Or, is it the right way to do it? Is it good to have a fork of the compiler inside of Melange to achieve that project? I don’t have a strong opinion on it.

I don’t have enough experience. After all those years of seeing Reason and Bucklescript evolving, I believe that the experience of the end user, so the developer that is using these tools, is more important than the technical implementation. Is it the best way to do it? I don’t know. Does it give a good end user experience? Yes, then that was the right thing to do.

David: [01:12:21] On those tools, you would always prioritize the developer experience, rather than technical merits? How would you choose —

Louis: [01:12:33] As a user or as a developer of those tools?

David: [01:12:36] As a developer of those tools.

Louis: [01:12:39] As a developer of those tools, given the target, given what I see of how you build a community and it’s like the early days of Melange, I would prioritize user experience. I think, for example, all the efforts that have been put into making Dune work, I think the target was the user experience at the end. Because we couldn’t make it work another way. If we didn’t have this, I’m not sure that we would have moved to Melange, for example.

David: [01:13:19] I see.

Louis: [01:13:20] What’s the downside was that for example, this is not the fastest implementation there is. I think there is some many different calls to this Melange compiler that are not the fastest way to do it, but the UI is good so we still use it.

David: [01:13:44] What I would like to feature, knowing that at the start Bucklescript got born, even though they were like js_of_ocaml. Now, I think, eight or nine years past, Rescript got its own path, but then Melange is trying to again, be part of the OCaml to Javascript compilation, or Reason to Javascript compilation. How do you see the future? Because eventually, nobody wants to have two ways to Javascript.

Louis: [01:14:16] I’m not sure that’s true. Why would people not want many ways to do the same thing? It’s like if you look at other languages, actually, many of them have different ways to do the same thing. Why not OCaml? As long as the projects don’t die, it’s not like Melange is attacking jsoo or jsoo is attacking Melange. It’s like, people don’t hate each other. They are not fighting for users; I think the targets are a bit different.

David: [01:14:53] You would want different ways of combining to Javascript? Because the sane competition? That’s true-

Louis: [01:15:03] To me it’s not the sane competition. It’s more that I think it targets different audience. It tries to do different things. One example will be during one of the Reason conferences, we wanted to do a workshop and we wanted to show atdgen which is a tool we’re using a Ahrefs lots to parse and write JSON. It’s protobuf but for JSON.

David: [01:15:34] Yes, it would give us type safety from front end all the way down. Sorry, backend all the way down.

Louis: [01:15:43] Yes.

David: [01:15:43] Sorry, go on.

Louis: [01:15:46] You have as with protobuf or with Graphql too you have a definition, you have a file with type definitions and from the definition you derive OCaml code or Python code or TypeScript code. It supports multiple languages. To do so you need an atdgen binary. In the Reason conference, you have people using Linux, Windows, Mac, different version, whatever so how do you give a binary that everyone can use? In two minutes, I just went into the atdgen repo and I enabled js_of_ocaml compilation inside Dune and now my binary is actually JS file that I can run in node JS.

David: [01:16:34] Right.

Louis: [01:16:35] I don’t think that Melange aims to do that. Because then —

David: [01:16:41] I think that’s the magic. Yes, I agree.

Louis: [01:16:44] In Melange you will have one file per module or something like this, which means I will need to run through webpack or something like this later on.

David: [01:16:51] Yes, you could but you would face a few problems. Marshall for example, that it’s the encoding/decoding on bytes, that doesn’t work in Melange.

Louis: [01:17:01] Well, it doesn’t work in js_of_ocaml I think too.

David: [01:17:05] Yes, but I think you can stub it, right? I think you can —

Louis: [01:17:10] But I would say most of the time actually you don’t care because it’s corner cases, it’s just that the UI they provide is good enough for OCaml people and Melange, it provides the nice, what? FFI, for example, to interact with the JavaScript code. The way it outputs code is closer to the JavaScript way too; I would say so it’s easier to make webpack or other tools like this work together

David: [01:17:39] Yes, I agree that those are different targets. It just my point of view was more like, okay, js_of_ocaml the crazy thing is that you have entire project in OCaml. You add one line say in Dune compile to node JS please and then you have a single file that is compiled to JavaScript. That’s insane so if you have, for example, a compiler written in Menhir it’s a language to write compilers in OCaml, you can compile it to JavaScript in one line or any library, even drivers, even anything that you can imagine. That’s the valuable position or thing that gets people to try js_of_ocaml very fast. But on the contrary, the documentation is very bad.

It’s the classic OCaml project that you need to understand 50% of the project to even start it so that’s like — For people like me, I invest a lot of time trying js_of_ocaml and even try to write bindings to React and succeed but I did not succeed convincing people in Ahrefs, front end of Ahrefs to try js_of_ocaml. For me, that was the — That technology is not good enough for prime time or not good enough to convince my team, then yes, there’s no way to convince any other.

On the opposite, Melange fits together the low barrier to try and good documentation and at some point, it gets complex but the ease of experience I think it’s much better. But yes, you don’t have a one line — You need to meddle it a bit on building the integration with your front end or your pipeline but yes, once it is done, it works. But yes, you would never do that with atd. The experience in atdgen that’s not going to happen in Melange.

Louis: [01:19:41] It’s funny how you say it and it’s true that it’s easier. Many things in Melange are easier to experiment with and at the same time it’s more complicated. For example, in js_of_ocaml you have a clear separation between OCaml types and JavaScript types. String that is an OCaml string is a different type from JavaScript string.

David: [01:20:10] You have like a wrapper, right?

Louis: [01:20:12] It’s very explicit and it’s good for the OCaml person because then you know when this is a part of the language you are comfortable with and then when it starts with JS, it’s okay, be careful because you don’t know what you are doing. This is easy and in js_of_ocaml. Because it’s very easy. You see JS dot and then you know now I have a JavaScript value. In Melange it’s your string is what? And you have to deal with the encoding. What is the actual encoding of a string in Ocaml?

David: [01:20:57] Do you remember that I said that every time that I’ll talk with a backend person, they always mention the runtime. Exactly that moment. You always think about the runtime.

Louis: [01:21:06] Actually, I’m not sure.

David: [01:21:08] It’s not the runtime itself, but the encoding. In Melange for example, of course, all the types not of course, but all the types that you have in language are the same representation as a JavaScript value. For example, a string is in a string, integration is a number, float is a number and so on and so forth. Variant is an object; a record is an object. Melange maps perfectly or as good as possible to JavaScript values. It’s cool that you said that when every time that you work with js_of_ocaml, once you see JS dot, whatever this is the namespace and you know that you’re treating with things that come from the client.

For example, that’s a barrier for people that tried rescript or tried Melange in the first place because they don’t understand why do I need a wrapper? Why do we need a generic for at type that already have? Because it’s the mentality of why do I need to care about the runtime?

Louis: [01:22:17] Yes, basically you pay a cost but at a different time, like in js_of_ocaml, you pay the cost very early because as soon as you write code you need to make the difference between the two words. In Melange you will only pay the cost if you write FFI and you need to care about the representation. It’s if there is a string with something weird in it, you don’t know the encoding of the string, for example, then you need to be careful.

The experience by default is much easier. It’s just that when you are dealing with the boundaries then things can be a bit more implicit and probably you need to know the language better to do things the right way. It’s easier and it’s actually more complicated in some bits.

David: [01:23:08] Yes, I think if you look now, js_of_ocaml and Melange are very drawn line. You can draw a line between the tradeoffs. One side is very clear, one side is very clear. Now, I would say that I’m comfortable saying that both are balanced for the users, even rescript now. For me now, I have a feeling the three of projects are in the right column. You can classify them perfectly now. If you get into, “Oh, I want to try this ML or like OCaml, whatever language as a whole,” you can choose — Based on your team or your decision. You can choose clearly one another.

Louis: [01:23:57] Yes. Actually, you said you couldn’t sell js_of_ocaml to Ahrefs, but we can probably talk a bit about what was the discussion, what happened, because Javi and you, you actually tried to do something so that it could happen. You work on the React bindings and then you try to show that it could work. In a way, I think that js_of_ocaml, it could fit what we do because we don’t depend on a lot of external code. One very interesting thing in Melange is that the FFI is very good. It’s easy, convenient, to interface with other existing JavaScript libraries.

In Ahrefs, we have bindings to what, React, and then maybe one or two library to deal with the timestamps and charts. We don’t have millions of bindings. We have maybe five big libraries we have bindings for, and then a bunch of smaller stuff. We don’t bind to so many things and we don’t need FFI that is amazing. It’s not a priority. js_of_ocaml could have worked.

David: [01:25:14] Could have worked. I agree.

Louis: [01:25:16] The fact that even in this perfect setup for js_of_ocaml it fails is interesting. You find the right company with many OCaml people, many people who understand js_of_ocaml, and you don’t need one of the best features of Melange and still, this is not actually the tool that won at the end.

David: [01:25:42] Yes, that’s true. The experience, I think that’s exactly what you said, it could work, theoretically if you look at the direct from the outside or even if you look at far from the front end, it makes a lot of sense. Once we were working on this, I was working on this middle-end team, before it was not called middle-end. I was working on the middle-end, and most of my assumptions were like, “It’s going to work perfectly.”

Because of what you said, right? Then when we try to — How can we write React, we are married to React. I think we like the model of components. We like the model of data; we like the composition. We’re not going to change React. Let’s bind it to React, so we create the same PPX and the same library to React. I think that was how Javi started and then we end up finishing.

did the emotion binding, so I know the CSS, everything worked and we felt like js_of_ocaml was very mature, but there were a few problems that you could not solve easily at the time. At the time js_of_ocaml didn’t have Unicode support. Now they have some Unicode support or the parsing, I haven’t followed that closely, but you would need another library to run to get the Unicode support that in Melange or Bucklescript at the time was natively. That was an issue. The other issue or biggest issue that you can’t bypass is that js_of_ocaml, you compile it in one file, one gigantic file.

Incremental migrations were very hard or very difficult to iterate over time. You could migrate parts of the app, but then you would need to compile everything in both, have two duplicated apps. It was definitely not — The migration plan was impossible. We could try. I think we tried in one of the small apps, I think we could try wordcount, is one of the verticals we have at Ahrefs, with js_of_ocaml, and once we were trying those, we find the wrapper, it was very hard to sell.

The wrapper is like the Js.t that we call it in Recript, in js_of_ocaml I think it’s JS.object. It’s unsafe. You have JS.unsafe. There are many, yes, many constructions you can track with JavaScript differently from what we do with the bindings. That part was — With these three things that I said Rusty, which is one of let’s say the only Tech Lead at Ahrefs, like the only person that — He’s the CTO in the frontend, how I call it.

He was the person who we would need to convince to migrate to the frontend. He was definitely not on board with the idea. I think that’s the main reason. He would chat with our people and people would say, “Yes, fine, if Javi and David are happy, then we are all happy,” but even though we migrate one small app, the experience was worse. The user experience of iterating over React components was worse or even the data was worse because you had this wrapper.

Louis: [01:29:23] There were too honest in the way they named functions in the API. For example, all those unsafe functions, they exist in every FFI, it’s just not called unsafe, but because it’s called unsafe, people are like, “They’re not going to use this, you are not supposed to use it.” Yes, you’re supposed to use it. Just be careful when you do it.

David: [01:29:47] I think you explain to me that anecdote, is that somebody asked Xavier Leroy the creator, the author of OCaml, they ask, “What do you think about Objec.magic?” Right? Object.magic is the method of OCaml that you can, like unsafe, coerce any variable, right? You can light the typechecker and say, “Trust me, this is whatever, an array and it’s a list or whatever”. His answer was, it is like when you are working in the street, would you inject — How is that called? I don’t remember the thing, but would you become a junkie?

You get a syringe, I don’t know how to say in English, but would you inject some random thing on the street? That’s not part of the language. I think you’re explained me the anecdote, or maybe it’s Javi, How do you see the purity of OCaml? Do you think that the OCaml is very pure or has some pragmatism on safety? Because of course it’s type safe, of course, the compiler when it compiles it works, but you can bypass it from time to time. What’s your opinion?

Louis: [01:31:10] I don’t think it’s pure in any way, shape, or form.

David: [01:31:17] You can write pure code, right?

Louis: [01:31:20] Yes. You can write pure code. But for example, you have exceptions that are very pregnant, that are everywhere and you don’t have any way to know if a function can raise an exception or not.

David: [01:31:37] Right.

Louis: [01:31:38] Okay, it depends what program you write, but basic things, you run your program, like a CLI that is running — I don’t know, downloading something and you press control C like you want to stop your program. There is — It’s a signal, and in OCaml it’ll raise an exception that you need to — You can catch and you can do something with it, right? At any point in time, the user of your CLI can come and interrupt the program, right? Which means at any point in your program, you need to be able to deal with this interruption.

David: [01:32:18] Right.

Louis: [01:32:19] It’s like as soon as you have these where is the purity, what is — You have no good way to protect yourself against all these issues. At the same time, I’m probably biased because I have been using the language for long. It provides you what is good enough. There was some improvements because, I don’t know if you remember, but at some point, the strings were mutable in OCaml.

David: [01:32:51] Yes.

Louis: [01:32:53] By defaults, the strings were actually what is called Bytes nowadays. It has been a big change. People had to fight to turn Bytes into string, because it was breaking code, obviously. There was more mutability. It was not as pure as it is nowadays, I would say that the balance is not too bad. Could it be more? Probably. There are some things that we can’t really express in OCaml, like ownership of a value.

Like you open connection to a database, you have a handler or something like this that you want to use only at one point in time and you don’t want to share. You have no way to express it. Then you can’t really protect yourself against the steal. The code can take that value, put it in a global reference, and it can be suddenly reused elsewhere. This is where, for example, the local/global stuff —

David: [01:34:06] Yes, solve exactly that issue.

Louis: [01:34:09] Yes. This kind of issue. Is it a problem? Yes. Is it a problem that we face at work? Yes. For example, we see like — We have one problem where people can open the connection to a DB using one of those — A common pattern in the ocaml to do like “with_db” for example. Then you pass a continuation, you pass a function, and then this with_db function will create a DB handler and pass it to your function later on.

Inside your function, you can do one more with_db. This is something that you probably want to forbid because you don’t want to open connections after connection after connection when there is already one available.

David: [01:34:57] Right.

Louis: [01:35:00] For now, how do you fix this? This is an actual problem and you have no good solution. But maybe you write different code than I do. You write code that is in the browser or just behind the browser so maybe you have different views. You have to deal with more mutability than I do, for example the whole DOM, before React. Yes, but before React no one assumed that anything was immutable in a browser. Everything could be changed at any point in time.

David: [01:35:43] Yes. That’s why in the browser many APIs were pushing for observables, right? You accept mutability into all your values and then you say any value can change in any time and you need to subscribe to — Listen to the changes or not and that’s the trend of — I think that was one of the biggest inclusions of ES4 that didn’t get published and they tried with ES5 and they didn’t get to the language neither that are like these observables concept. I think they come from React JS and they come from reactive programming from, I don’t know, 30 years ago where sometimes reactive is very useful. Before React, I would say that not many people did — The immutability was not part of their fashion of writing code.

We are very far from those problems. We do immutability in a few places, for example, we have a global theme, a CSS theme, right? You can have a dark or light theme. We interact with the browser directly. We opt out from React to do that because the performance is better. You can load that at the beginning, you can then allow React later. But the way it’s just very self-contained, right? You will never want to write your data reactive.

Maybe you want, but for example, for Ahrefs it doesn’t make any sense because our data is you have tabular data that never changes on your session, right? It’s not live data. It’s like you open a report and the report is the moment time that you request. There’s no live thing. Nothing is very reactive in nature so yes, for us it’s just like a perfect sense.

Louis: [01:37:41] You would be happy with more purity?

David: [01:37:45] Would we be happy with more purity? No, I think —

Louis: [01:37:49] Would you wish to have a language that is closer to Haskell that is like —

David: [01:37:54] No, I don’t think so. I think no, because purity makes — Purity in some places makes your life so much better, right? But often you want the tools to be pure so like libraries that you create or you consume need to be pure but your application needs to do all sort of things, right? Your application or when you are a product engineer, you want to just ship fast and if something gets you in your way and you store it globally and deal with it later or store it globally and be safe and then forget about it.

You need to do things perfectly and draw the line and architect things that slows you down insanely. I think the line — OCaml is very well position where you can opt out, do your life easy and then move back and run fast. But yes, my tools to be pure or libraries that I’m using or even — I know I’m working on styled-ppx so making types safe, like your styles. I think that’s something that I’ve been pushing but yes, you want that tool to be type safe.

You don’t want to do all your things on your app perfectly, to demand it perfectly mostly because on the web, everything is changing all the time. On the backend it’s a little bit different but on the web, iterations are just much more common than in the backend

Louis: [01:39:31] I find it interesting that everyone is pushing for immutable stuff. At least my impression is that in the front end React maybe didn’t create this trend but made it popular. My understanding is that you deal with the DOM as an immutable object. You never manipulate the DOM directly anymore. You do it through React. You have an immutable object, more or less, which goes against many things that happened, historically in a browser, the way the DOM is implemented is completely not like this. It has some interesting benefits.

You can have any extension in your browser that are changing part of your page. I’m using one daily. I’m using Dashlane to store my passwords. It does stuff for — If there is an input field, it’s creating a popup and I can click, input my password in that specific field.

To do this, it has to inject HTML in the page actually. But it breaks. Some apps are crashing because of this. Some apps, some websites, they’re not crashing. They will just see that there is a change coming from my extension and they will just discard it and rerender without my stuff.

It goes against many things that happened for 20 years, more or less. You have to make both of those worlds still somehow work together.

David: [01:41:18] I think React did — I think not even React, Meta did that all the time. They pushed for a solution that is way better in some areas but destroys previous effort insanely. For example, with GraphQL, I think it’s happened the same. They say, “You’re going to have one endpoint. It’s going to be through POST.” You would call this endpoint all the time, which goes against completely about REST what we were doing before.

Of course, with all the tradeoffs, if you go 10 times — 10 years back and you say to a person, “No, we call all the time same endpoint.” You will say, “You guys are stupid.”

Louis: [01:41:58] RESTful was a trend.

David: [01:42:00] Exactly. The RESTful, you will need to add the link to go to the next resource. Of course, all the people would scream at you. Similarly, it happens now with server components. I don’t know if you are following the thing. Again, they are pushing for a new concept that they have mined in their business and works well. The rest of the people are like, “No, that’s just insanity.” I think React, the first concept is you — The insert and update are the same operations.

There’s no create the DOM and then update the DOM. It’s always like, “Do the thing.” It’s just rerender. Just because they push for that approach and they delay, of course, the first load is going to be slower. You don’t have serialization. Then later updates are going to be faster.

Louis: [01:42:58] Actually, even the later update are slower, because you need React to do this diff between the two version and to only update the relevant part of the DOM. React is doing what the browser was actually doing. You are duplicating the work and you are doing it in JavaScript, which is slow. The browser was doing it in a very optimized C++ code

David: [01:43:26] Fair point, but the DOM is fast enough. I think I read a lot of articles about anti-React that the DOM is fast enough. I think that’s — These benchmarks are nonsense. Those benchmarks — Even the benchmarks that — I would call it micro-benchmarks. Even microbench like — I don’t know, work faster, implement some charting library mutable to DOM and using whatever, Vanilla JavaScript or using React. Of course, you can outperform React but at what cost.

The cost of creating two charts, two components of a chart, the API nicely, blah blah blah blah rather than mutating the DOM all the time. That’s super expensive. When you’re working, for example, for Ahrefs, I think we have — I don’t know, 2 million lines of code in reason. I don’t know, 5,000 components. Some insane amount of number of components. If you do that in mutable or maybe not mutable but maybe just going to DOM and listening to DOM and hoping that everybody is a good citizen. It’s just you would slow down development so hard that it doesn’t make any sense.

I think React draw this line where virtual DOM is like if you know how to create the structures, know how to trigger the renders, I think can be as good as the DOM, of course. In general terms, I think it’s good enough. I think the balance is very OCamly. [laughs] You know that Jordan was behind, when — Jordan is an OCamler that knows this balance, that —

Louis: [01:45:23] It’s funny, because you could say that for example Facebook has enough resources to make it work, right? They could have decided it is going to be a more troublesome for our developers, but we are going to offer a faster experience for the customers, the people actually visiting the website. But even them with their infinite resources, they decided to go in a different direction. Is it because it’s a tech company? I feel that some big tech company, they run the way they do, because they are led by technical people, so they can make technical choices instead of business choices sometimes.

David: [01:46:08] React is that, exactly that case. I think I saw the documentary about React, and it’s more the idea just spreaded everywhere. I think they have that competing library; I don’t know how, I think it’s called Jacks or something. I don’t the name, but they have a competing library that was written with PHP and XML, and the whole stack on meta back in the days. The idea of React spreads everywhere. The point that you made before of developer experience leads everything in the early days. I think that it just applied exponentially, because at some point they released publicly, everybody hated JSX. Then after one year it was the most famous library used. Then from now on, the monopoly went for, I don’t know, six, four years, I don’t know. So many years that now everybody has the component model, the hooks, the state. It spread the idea everywhere. It’s interesting, but I think it’s just like [unintelligible 01:47:11] , so.

Louis: [01:47:13] How smart do you need to be to Jordan and be correct about React was the right thing to do, and to do it properly, and then Reason was the right thing to do, and to do it properly?

David: [01:47:25] How smart need to do. I don’t know. Uncountable, I would say. Hard to quantify smartness. It’s hard to quantify smartness, but I think it’s even harder when you look at what Jordan has been doing. I think Jordan is the kind of person when you speak with him, he talks, or he says things that doesn’t make sense at the beginning. It’s his way of thinking. He’s thinking three, five years ahead, and when he explains the idea to you, you are like, “I got the sense that I didn’t understand anything.” After a few months you start saying, “Oh, right, it made sense.” I think everybody on the React team only says how brilliant Jordan is. So, yes.

Louis: [01:48:18] It’s interesting that you interviewed Rudi, the author of Dune, and he said that even him, for example, originally when he saw the Reason syntax, he was like, “meh,” like, “What is this thing?” Like, “Yet again?” He say, “Yes, at the end it was right to make it more approachable.” This was from a user experience perspective. It’s a clear benefit.

David: [01:48:44] Yes. I love that interview, because Rudi said exactly that, like, “Oh, at the beginning Reason felt like a toy, but then we were doing tooling for OCaml, and the language and a lot of work on the actual language. Reason was the thing that bring more people on OCaml, than we never, ever did. Even though it’s not a competing language, even though the person that created has so much power into the frontend people, even that the number of people that got into OCaml community, it was bigger than any other effort that we made. [laughs] Which for me is super funny at the end of the day, because most of us came from the JavaScript and end up doing OCaml and mixing everything. That was the idea.

Louis: [01:49:40] It’s funny how you need a little bit of luck for all those things to work. You need, for example, Rudi to decide early on, “Okay, I don’t really trust this thing, but I still I’m a good citizen, so I will add the support inside Dune”

David: [01:49:56] Yes. That’s very noble.

Louis: [01:49:58] It’s like if you don’t have those people who are able to compute this is maybe good or maybe bad, and are able to balance their opinion, versus the community thing. They have to do it early enough, at the right time. It’s interesting that somehow —

David: [01:50:24] Of course, not what everything Jordan says is correct, I think in the sense of, of course, he created stuff that was definitely not on shape or not on the area of success of React, of course. I think he created React and then React native. Of course, they both are insanely successful. Reason I would call it successful as well, but actually, push the idea of Esy. Esy is the package manager that is still somehow used, and some people love it. Even myself, I have a lot of respect for Esy, and use it from time to time. That you can consume JavaScript libraries, npm packages as well as OPAM packages.

This project has been suffering for long, that it’s definitely not the right solution, or at least, it didn’t create these ideas, this sudden idea to the rest of people to continue pushing for it.

Louis: [01:51:27] This is one of those tools where I think the technical implementation was good, but the UI was not great. The output is just not nice, for example. You run it, and then it displays some-

David: [01:51:44] You mean, the actual UI, the CLI?

Louis: [01:51:46] Yes, because opam, which is not the most fancy tool ever, but still when you opam install, it has some colors. It doesn’t display one line production. It’s some kind of somehow clean output that Esy doesn’t have. You had to learn this weird JSON syntax to put your package, and it outputs ugly text after that. The idea was very good, but the UI was not completely working.

David: [01:52:24] I think Esy has some, people call it state-of-the-art ideas, the end goal of our package manager, what do you want to do, or we want to use. The efforts of maintaining the overrides or being on top of all the libraries or even compiles Esy, there’s a lot of maintenance that needs to get done. At some point, we have a team of, I think, six, seven persons working on it, and that experience was very good, but from when Reason got a little bit lost, this team, those people —

Of course, when the blockchain companies started hiring all of them to work for Web3 and paying them insanely amount of money, then the project got a little bit carried over, got a little bit less maintenance then is likely in a stale mode that you can use, but you could get not as good as opam.

Louis: [01:53:35] This exactly why there is no Rust code inside Ahrefs, because all the developers got stolen by the blockchain companies.

David: [01:53:45] That’s fair. That’s fair. I think, Louis, we are running out of time. For me is daytime, I can do stuff, but for you is definitely night time. I can work, talk with you for hours and hours, but I think the show is reaching to a point to finish. It was a pleasure to have you, of course.

Louis: [01:54:07] It’s a pleasure.

David: [01:54:09] If somebody attends to ICFP, please go to Louis, I think he’s the party manager, and as well, a person very interesting to talk to. Please, bother him. I think he’ll be in Seattle on September.

Louis: [01:54:26] Yes, Seattle, September 4th to 9th, I think, something like that. It will be online too, this year. I think it will be online and for free. All the talks, at least all the ML, or maybe not ML, but the OCaml workshop will be online of free. There is no need to travel all the way to Seattle to see the talks, at least to see the OCaml workshop.

David: [01:54:52] Makes sense. In this era of internet, I think that makes sense. Cool. Thanks everybody for being here. You’re having a little bit late day, but that was perfect. Thanks, Louis, to spend time with us.

Louis: [01:55:07] Thank for having me. That was fun.

David: [01:55:10] See you guys.

[01:55:13] [END OF AUDIO]

EmelleTV: Talking with Louis Roché about OCaml and Ahrefs was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Ahrefs is now built with Melange

ahrefs — 2023-05-03T19:06:58-00:00

OCaml, all the way down

Photo by Lance Grandahl on Unsplash

Some history

In 2021, we decided to evaluate Melange as an alternative to ReScript for compiling Ahrefs’ frontend codebase. We wrote about the reasons that led us there, as well as the limitations we encountered at the time, in a previous article.

After this experiment, discussions continued inside the team. Switching to a different compiler, which was in a very early stage, involved quite some risk. But so did the continued use of ReScript, which seemed to be diverging further and further away from OCaml.

Finally, in September 2022 (during ICFP in Ljubljana), we decided to bite the bullet and kicked off a project to deepen the integration between Dune (OCaml’s most used build system) and Melange. This better integration was the key to solve two of the three limitations we had encountered during our initial exploration of Melange:

Build speed would increase due to less work needed to parse dune files, and more efficient rules planning and execution.
Developer ergonomics would get better, as Melange would become a first-class citizen in Dune, with concepts like Dune libraries and other stanzas becoming available to Melange users.

Ahrefs’ leadership backed the project and agreed to financially support the development of this tighter integration. With this support, we set about building a team that included Rudi Grinberg, who maintains Dune as part of its development team, and Antonio Monteiro, who created Melange and is also part of the Dune development team.

Heads down

During the following months, we focused on two tasks, iterating over multiple cycles where the progress on one task would inform the next steps to take for the other:

Evolve Dune to add stanzas, fields, and documentation to support Melange projects.
Migrate Ahrefs’ frontend codebase to use the Melange compiler and Dune, adapt third-party libraries and bindings to Melange, and polish the editor integration, build scripts, and other aspects of the development experience.

We believe that tackling these two tasks in parallel led us to better results, compared to a more waterfall-based approach. As we applied the changes over Ahrefs’ large-ish frontend codebase — it will soon reach 5000 modules — we kept finding and fixing bugs, improving the ergonomics of the Dune and Melange integration, and in general making the solution more robust, real-world ready, and developer-friendly.

Another upside of the way the project was implemented is that we developed it initially in stealth mode, keeping it quite private. By working on it within a tight-knit team, before making a public release, we could make progress faster. We believe that this approach saved future Melange users a lot of churn and burn caused by the multiple changes in Dune stanza options, Melange flags, and other configurations we changed along the way, as we learned more about this integration.

Migration strategies

Initially, our plan was to progressively migrate Ahrefs’ code to Melange. As the frontend codebase is divided into different tools, each being self-contained, we thought we could introduce Melange to build one tool, then another tool, gradually migrating them one by one.

However, this approach turned out to be too complex because configuring a development environment that works on both Melange and ReScript is challenging. As developers could be working on multiple tools during the same week, or even within the same day, we realized that it was unfeasible to reconfigure the environment every time a developer switched from a tool built with Melange to a tool built with ReScript.

Therefore, we changed our minds and opted for a one-shot migration. We would ensure that CI, development, and staging environments were working with Melange and Dune. And we would do this on separate branches, while still using ReScript on our main branch CI and development scripts. Once we were confident everything was building and functioning correctly with Melange, we switched all CI and development scripts to use the Melange and Dune commands. We tried to keep the PR that applied this switch as small as possible, with just a few hundreds of lines of changes so that we could switch back to ReScript if needed. In fact, after a first attempt in March, we had to switch back to ReScript due to some issues on the developer experience side, related to build performance and ergonomics, which took a few more weeks to solve.

In terms of package management and third-party Melange dependencies, we followed a more gradual approach. Dune is quite flexible when it comes to vendoring, so in the initial phase, we downloaded Melange libraries with npm, and had Dune include them in the project as if they were local sources. Now we have started migrating some of these libraries so that we can consume them using opam, the OCaml package manager. This will involve first publishing them in our private opam mirror, but the plan is to have them published in the public opam repository in the future so that other Melange developers can also use them.

Timings

You may be curious about the performance differences between the previous and current approaches. Measuring performance is tricky, but we attempted to measure a few different scenarios with both setups. The results can be seen below.

Keep in mind that Ahrefs frontend setup has specific characteristics, which affect the performance measurements:

Before migration: Dune generated ml files from atd files, then ReScript build tool bsb built all hand-written source files plus the ones generated from atd files.
After migration: everything is built with Dune and Melange.

All measurements were taken on a node with 2x AMD EPYC 7742 cpu @3.2 GHz (nproc=256), 1TB RAM, Debian 11 x86_64 GNU/Linux. The build target is always the entire Ahrefs frontend codebase.

Cold build:

Before: real 0m28.232s, user 9m23.883s, sys 13m33.939s
After: real 1m14.208s, user 10m33.708s, sys 5m45.644s

Warm build, noop (no file is built):

Before: real 0m14.687s, user 3m17.058s, sys 3m57.903s
After: real 0m21.895s, user 0m20.528s, sys 0m1.372s

Watch mode, modifying an “edge” file with almost no reverse dependencies:

Before: 1002ms
After: 1576ms

Watch mode, modifying an “inner” file belonging to a library, with many reverse dependencies:

Before: 7032ms
After: 15394ms

In general, Melange and Dune are slower than ReScript for cold builds in our setup. However, the differences are smaller for warm builds. For watch mode, the difference gets reduced when modifying edge files.

There is room for improvement in the way the Melange and Dune rules are arranged so that cold builds can get faster. For example, delaying some optimizations in Melange might allow to parallelize more work.

Conclusions

The results so far are quite encouraging. These are some of the things that are possible thanks to the deeper integration between Dune and Melange, and its application within the Ahrefs codebase:

The same OCaml compiler is used on both frontend and backend codebases.
Access to all the bug fixes, error improvements, and new features that the OCaml compiler team added between versions 4.06 and 4.14 of the compiler.
A shared developer environment across teams, including editor extensions, OCaml LSP server, etc. No more need to maintain a different set of tooling for backend and frontend.
Removal of hand-written CI checks that were ensuring different tools in the frontend codebase would not access components from other tools. This is now solved by Dune libraries, and the OCaml compiler will complain if logical units try to reach outside their bounds.
Frontend and backend shared dependencies, such as anuragsoni/routes, can now be defined in a single place: an opam file.
Faster rebuilds and better watch mode, as Dune now controls all the build artifacts. Previously, Dune and ReScript were sharing responsibilities, which was leading to unnecessary rebuilds of some artifacts. Or alternatively, rebuilds were not starting when required due to the build system not tracking changes in some subsets of the sources.
Easier PPX maintenance, as there is no longer a need to publish pre-built versions of these tools.
Melange allows to run all ppxs from a single executable file, which has some nice performance benefits.
All the other advantages of using Dune: virtual libraries, watch mode, leverage integrations with tools like odoc…

What’s next?

We are excited about this project becoming a reality, and we believe that the deeper integration between OCaml and Melange through Dune, together with Melange’s ergonomic integration with the JavaScript ecosystem through its bindings, can enable projects that were previously impossible to imagine. For example, full-stack React by hydrating components that are rendered server-side using native OCaml.

Now, we have to make it easier for other people who are willing to try and use Melange and Dune. So our focus is shifting to documenting how Melange works.

There is a section for Melange in the Dune manual that will be included in the next stable release, and that can be consulted today in the latest branch: https://dune.readthedocs.io/en/latest/melange.html.

The next step will be to design and create a site where everyone can read and learn about what is needed to create and maintain a project using Melange and Dune. This site will include a playground, in the spirit of the ReasonML one, so that we can share snippets, see the resulting JavaScript compilation output, and iterate on ideas together.

Besides the above, we have plenty of other things we will be working on in the next months. We will share more information about the roadmap as soon as Dune 3.8 and its respective Melange version are published in the main public opam repository, which should happen in the next weeks.

How to contribute?

If you want to be a part of this, or you want to write or port your libraries to Melange, the best way to do so is by reaching out on the ReasonML Discord. There is a #melange dedicated channel where one can get help and advice on how to get started.

Otherwise, if you are missing features, find bugs, or run into confusing errors, please open an issue in the Melange public repo.

We hope you share our excitement about this update. Our journey to integrate our frontend stack more naturally within the incredible language and ecosystem of OCaml will be well-documented. Stay tuned for further updates in the future!

Ahrefs is now built with Melange was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

How to get started with OCaml in 2022

ahrefs — 2022-10-31T16:13:10-00:00

Intro

There has never been a better time to learn OCaml, one of the premier statically-typed functional programming languages used in industry. We at Ahrefs have used it on our backend since the early days of the company, and since 2018 have even used it extensively for our frontend code. Today we’re very excited to share with you our favorite recommendations for getting started with OCaml!

Best resources for learning OCaml

If you have little functional programming experience or are even a beginner to programming, the place to start is OCaml from the Very Beginning, a book that is now free thanks to generous funding from the OCaml Software Foundation (which Ahrefs is a sponsor of). The book can be viewed directly from its website and can also be downloaded as a PDF. While advertised as being approachable even to new programmers, this doesn’t quite seem to be true, at least based on feedback we’ve received. The text itself introduces concepts in a structured way, but the exercises require a little background in programming to complete. Such background info can be obtained by watching the opening videos in this series of lectures from Cornell’s OCaml programming course. Speaking of the Cornell course, their official online textbook OCaml Programming: Correct + Efficient + Beautiful is an excellent resource for learners with some programming experience under their belt (specifically, you should have written some code using a mainstream imperative language like Python or Java).

How to install OCaml

The official installation instructions are entirely adequate. It shows you how to install the compiler and some useful dev tools like dune (build system), utop (interactive read-eval-print loop), and ocaml-lsp-server (useful for editor integration).

Actually, if you are working through OCaml from the Very Beginning, you do not need to install OCaml during the first several chapters, as you can execute code snippets directly on the Try OCaml page, or create a notebook at Sketch.sh, an interactive OCaml notebook site maintained by Ahrefs through our monthly Open Source Friday program.

Editor support

For most beginners to the language, we recommend the official OCaml Platform Visual Studio Code extension. The official OCaml installation guide has a good section on setting it up. There is also great support for users of emacs and vim.

Tips

Even early on, it’s a good idea to start saving your code into .ml files and learning how to run it. The easiest way to run a simple program is to start up utop and inside of it run #use "name_of_your_program.ml" as described here.

When creating a new notebook in Sketch.sh, the default syntax is ReasonML (it’s not a different language, just an alternate syntax that more resembles JavaScript). Click on ML in the top left corner to switch to the original OCaml syntax.

Conclusion

OCaml originated from French academia more than 25 years ago, and from there spread to elite universities and forward-thinking companies around the world. Now the OCaml community has produced high quality learning material that is both free and easy to access. So take the initiative and learn you some OCaml!

How to get started with OCaml in 2022 was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Monorobot: a Slack bot for monorepos

ahrefs — 2021-12-09T15:19:04-00:00

Monorobot: a notification bot for monorepos

Monorobot enables configurable directory tree notifications for your monorepo.

A few years ago, we decided to move most of our code into a monorepo. Many advocates have highlighted its upsides, which include better cross-project coordination and simpler dependency management.

But one problem remained: none of the available GitHub integrations for Slack work nicely with monorepos. Slack is vital for day to day communication among Ahrefs’ globally distributed team, so a Slack integration was a must-have feature. We needed a service that could map activity from our various subprojects to their corresponding Slack channels — something existing solutions didn’t offer.

That’s why we built our own integration: Monorobot, a notification bot for monorepos. We’ve improved it iteratively since the monorepo transition, incorporating real time feedback from our engineers over time. Today, Monorobot is an active member of Ahrefs’ Slack workspace, dutifully routing GitHub activity notifications to different channels based on the relevance of each activity.

And now we’re open-sourcing Monorobot, for anybody to use in their monorepo setup! The package is available via OPAM:

opam install monorobot

Read on for more details about the motivation, an overview of the main features, and what’s in the pipeline.

Existing Slack integrations lack monorepo support

Within a monorepo, multiple projects have their code located in separate, nested directories. Correspondingly, each project’s Slack channel is only interested in activity from that part of the overall repository. The issue with most GitHub-to-Slack integrations is that once you subscribe a Slack channel to a GitHub repository, the channel receives all activity from that repository.

Suppose we operate various camel-related services, and we’re planning to launch a new camel ride-sharing app called Camel Ride. The directory structure could look like this:

monorepo/
| frontend/
| | camelride_ui/
| | | mobile/
| | | web/
| | cameldance/
| backend/
| | camelride/
| | | routing/
| | | pricing/
| | camelfood/

As you can see, both the frontend/ and backend/ directories contain code for our fictitious ride-sharing service, along with code from other projects.

If we were to connect the GitHub for Slack integration to this repository, notifications for activity from all projects would be sent to the same channel. Even if I were only interested in activity from the Camel Ride project, I’d need to sift through notifications from the other, unrelated projects. Imagine the volume of notifications this would create for a larger monorepo with dozens of projects. What a mess!

Enter Monorobot

Monorobot, hard at work.

Monorobot enables more granular control over where notifications from the same repository get routed, depending on the type of activity. This routing behavior can be defined in a configuration file named .monorobot.json, which should be committed to the root of the monorepo. Once you create a GitHub webhook from the repository to a running instance of Monorobot, it will use the configuration file to route notifications to relevant channels based on the webhook event payload:

For pushed commits, it checks the path prefixes of the files modified in the commits.
For activity related to PRs and issues, it checks their labels.
For status updates on pushed commits (e.g., CI builds), it uses the same path prefix logic as pushed commits.

Additionally, Monorobot supports unfurling GitHub links shared in Slack.

Path prefix routing for commit push notifications

Continuing with our example, suppose we want to route all commit activity related to the Camel Ride project to a Slack channel called #camelride. Our configuration file might look like this:

{
  ...,
  "prefix_rules": {
    "rules": [
      {
        "match": [
          "frontend/camelride_ui/",
          "backend/camelride/"
        ],
        "ignore": [
          "frontend/camelride_ui/images",
        ],
        "channel": "camelride"
      }
    ]
  }
}

Each rule “matches” a file path to a channel. Whenever someone pushes a commit touching files with either of these prefixes, the #camelride channel will be notified. All other commits will be ignored.

If a file prefix appears in a rule’s optional ignore field, the rule won't be matched even if the prefix is is also in the match field. In the above snippet, the Camel Ride frontend team has decided to silence notifications for activity in the images/ subdirectory.

Now, let’s say the project’s price optimization team is growing, and they’ve decided to create their own separate Slack channel called #camelride-pricing. We can simply commit an update to the .monorobot.json file, and Monorobot will detect the configuration change:

{
  ...,
  "prefix_rules": {
    "rules": [
      {
        "match": [
          "frontend/camelride_ui/",
          "backend/camelride/"
        ],
        "ignore": [
          "frontend/camelride_ui/images",
        ],
        "channel": "camelride"
      },
      {
        "match": [
          "backend/camelride/pricing/"
        ],
        "channel": "camelride-pricing"
      }
    ]
  }
}

Since Monorobot will match the rule with the longest matched prefix, only commits related to the price optimization aspect of Camel Ride will notify #camelride-pricing, and all other general Camel Ride commits will notify #camelride.

There are additional configuration options for prefix rules (and for label rules discussed in the next section) that aren’t mentioned here. Visit the repository for the full details.

Label-based routing for PRs and issue notifications

For activity related to pull requests and issues (opening, closing, merging, commenting, and reviewing), Monorobot uses labels to determine routing. The format is largely the same as for path prefix routing:

{
  ...,
  "label_rules": {
    "default_channel": "notifications",
    "rules": [
      {
        "match": [
          "Camel Ride"
        ],
        "channel": "camelride"
      },
      {
        "match": [
          "Price Optimization"
        ],
        "channel": "camelride-pricing"
      }
    ]
  }
}

Here, all PRs and issues with the “Camel Ride” label will have activity sent to #camelride; those with the “Price Optimization” label to #camelride-pricing; and those with both labels to both channels.

The default_channel field provides an option to fall back on a channel if no rule is matched; this option is available for prefix rules as well.

Status notifications

Monorobot also supports build status notifications for CI pipelines. When it receives a status update for a pushed commit, it routes it to the relevant channel(s) by applying the prefix rules to the commit associated with the build. Further filtering based on status (e.g., ignoring canceled builds, and only notifying for a successful build when preceded by a failed one) is also possible.

Link unfurling

Finally, Monorobot can unfurl links to GitHub repositories shared on Slack (including private ones, if a personal access token is provided). This applies to commit, issue, and pull request URLs.

What’s next

Monorobot is actively used at Ahrefs today, but there are lots of promising future directions it could take. Here, we list a few.

Unifying GitHub and Slack identities

It would be useful to allow GitHub user IDs to be mapped to Slack ones. This would enable more personalized features for Monorobot, such as direct messaging a user when their review is requested or when a CI build fails on a feature branch they authored.

Consolidating notifications

Sometimes, a collection of multiple GitHub webhook events makes sense to be grouped and delivered as a single Slack notification. For example, pull request reviews generate discrete webhook events for each review comment, but it would make more sense to pool them together, so as not to spam a channel with many notifications.

Better status notifications

A CI build failure can have multiple potential causes:

A bad commit
A previous bad commit that has yet to be fixed
An issue with the pipeline itself (this is out of scope for Monorobot)

It can be quite tricky to discern between the first two causes from the GitHub webhook event alone. Cause 1 is handled well by our current approach of using path prefix routing on the commit associated with the build. But with cause 2, that same approach doesn’t always send the build failure notification to the channel where it is actually relevant. In that case, the originator of the initial bad commit won’t be nagged about all subsequent failures, and Slack channels with no relevance to the cause of failure will get polluted with unnecessary notifications.

How can we best determine whether a status notification is “relevant” to a Slack channel? This is still an open question, but one possible direction is to track build state per build step rather than per status, and route notifications based on that. For example, if an overall build fails due to a backend build step failure, then it could be sent to a channel where the frontend team won’t be notified.

Wrapping up

The overall goal of Monorobot is to make Slack notifications more relevant for all teams in a large monorepo environment, using the information available from GitHub webhook events. We’ve had fairly positive results with our own internal usage, and now we hope others find it useful as well.

Monorobot is written in OCaml. We welcome your feedback and contributions on GitHub!

P.S. If anyone does make an actual ride sharing service for camels, do let us know…

Thanks to Feihong, Igor, and Louis for feedback on this post.

Monorobot: a Slack bot for monorepos was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Building Ahrefs codebase with Melange

ahrefs — 2021-05-18T15:24:20-00:00

Photo by Jens Lelie on Unsplash

At Ahrefs, we have been using BuckleScript and ReasonML in production for more than two years. We already have a codebase of tens of thousands of lines of code, with several web applications that are data intensive and communicate with backend services written in OCaml, using tools like atd.

Given our investment in these technologies, we have been following closely the recent changes in ReScript, with its rebrand and renaming, and the split with the ReasonML project, explained in the project blog post.

ReScript: becoming its own language

We are excited about the way ReScript is unifying the experience and making it easier for developers who are getting started to find documentation in a single place, as well as continuing its strong focus on performance and readable JavaScript output.

On the other hand, we are trying to figure out the implications of this change in the mid- and long-term, especially regarding the integration with the OCaml ecosystem. And more importantly, what this evolution will mean for production users like us who rely on this integration.

ReScript integration with OCaml has historically been seamless, as BuckleScript started originally as a new backend for the OCaml compiler. However, in recent months, there have been several hints that ReScript wants to evolve towards becoming its own language:

It has now its own parser, incompatible with OCaml native applications
Official repository guidelines for technical writing mentions explicitly that no reference to OCaml should appear in docs
Upgrades to the latest version of OCaml compiler, which used to be part of the roadmap, have been deprioritized recently.

So, even if officially ReScript has not announced that they will break backwards compatibility with OCaml, just the fact that it is sticking with an old version of the OCaml compiler poses some challenges for us in terms of tooling. The uncertainty about the future and the pace of changes add some risk to the high-level goals we have for our teams and codebase: we would like to share more code between frontend and backend, not less.

Melange: a fork of ReScript, focused on OCaml compatibility

When António Monteiro announced Melange, a fork of ReScript but with a strong focus on keeping compatibility with OCaml, we decided to try it out and see how it could work for us.

Ultimately, the experiment was successful. We managed to build all our frontend applications with Melange, while keeping the existing bundling setup, which currently uses Webpack.

Throughout this process, we had to modify some parts of the code. We will now go through the most relevant parts of the process:

Upgrade to OCaml 4.12: the most relevant part was the deprecation of Pervasives module to use Stdlib.
Use ppxlib in our ppxs: we had to upgrade the two ppxs that we use in the frontend codebase to the latest compiler version, bs-emotion-ppx and an in-house ppx for internationalization.
Configure esy: we were already using esy to bring the editor tooling into scope of the developer environment, so we just had to make sure melange would also be included in the json configuration.
Upgrade to Reason 3.7.0: a quite simple change too, as the whole process is automated by using refmt. As a side note, we ran into a small bug with some type annotations, that we were able to work around.
“Lift” dune workspace to the root of our monorepo: this is probably the most intrusive change. Because we have shared code between backend and frontend, and Dune needs to have access to all sources under its workspace, we had to “lift” the Dune workspace from the backend directory to the root of monorepo.

The good

This experiment allowed us to experience what a project like Melange could offer for our use case. Here are some of the things we might be able to leverage in a codebase built with Melange:

Recent version of the OCaml compiler: at some point, we could pin compiler version between backend and frontend teams, making upgrades more straightforward as they would happen atomically.
Shared editor tooling: the official OCaml vscode extension works great with Melange, as well as any other OCaml editor integration. Having backend and frontend teams use similar editor setup removes a lot of maintenance work for us.
Consuming ppxs from source: Melange allows to consume ppxs from source, which also removes issues with pre-compiled ppxs (like this issue with the recent M1 Macs).
Melange allows to run all ppxs from a single executable file, which has some nice performance benefits.
Use Dune for atd files generators: ReScript “generators” are unfortunately not documented anymore, but we use them extensively for atd file generation. Being able to share Dune rules in backend and frontend would make our build setup easier.
Access to OCaml documentation tooling: Melange allows to leverage existing tooling for generating documentation, like odoc.
Async syntax: the latest Reason version supports “let op” syntax, which is handy for client-side code.

The bad

While there are many things that are exciting about Melange, there are some other parts that can be improved.

Build performance: We already knew that performance would be far worse than ReScript, as Melange uses Dune in a way that it was not designed for. In our tests, builds with Melange are roughly 1 order of magnitude slower than ReScript ones.
First-class Dune support: if there was a deeper integration between Dune and Melange, we could explore features like shared libraries or shared rules between backend and frontend. As of today, Dune has no knowledge about Melange environment, so it can perform basic rules execution, but there is no access to high level stanzas like library in Melange.
Two-headed goal: finally, we see a more strategic risk in Melange proposition. Right now it has two goals: keep compatibility with both ReScript and OCaml. But we don’t know how long these goals will be feasible. If at some point ReScript decides to move away from the OCaml compiler fully, then Melange users would not be able to consume any updates to the ReScript ecosystem anymore.

Alright, but are you migrating to Melange or ReScript?

With all the information available, the answer is: we don’t know yet. 😄 We want to keep exploring all the available options and have as much information as possible before committing further. So for now, we are upgrading the codebase to recent versions of ReScript, but we are holding up on features that only work one way. For example, we have not migrated our codebase to the ReScript syntax yet, as there is no way to translate back to Reason syntax.

In the meantime, we will keep exploring how far the limitations of Melange can be mitigated. To be continued! 🚀

Thanks to Igor and Feihong for reviewing and improving earlier versions of this post.

Building Ahrefs codebase with Melange was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

One and a half years of ReasonML in production

ahrefs — 2020-07-26T15:19:31-00:00

Photo by https://unsplash.com/@willianjusten

The first Reason application at Ahrefs went online on January 31, 2019. Since then, many more applications have been either rewritten in Reason, are being slowly migrated from React to ReasonReact, or are conceived from the start as Reason projects. It is safe to say that the bet placed on Reason paid off big time. We will never go back to doing pure JavaScript again, with the possible exception of simple backend scripts.

In the past few years, it’s come to light that there are a number of other large Reason/BuckleScript codebases in the wild, but there still isn’t a ton of information out there about what it’s really like to work with Reason in production. To help remedy that, we thought it would be instructive to ask each of our frontend team members what their Reason journey has been like so far.

We gave them the following questions as starting points (but they were free to talk about anything they wanted):

How does Reason compare to other languages you’ve used in the past?
What’s your favorite thing about Reason?
What’s your least favorite thing about Reason?
How does ReasonReact compare to other frameworks you’ve used?
Was it easy to pick up Reason? Why or why not?

Javi

How does Reason compare to other languages you’ve used in the past?

In the past I worked with languages like Java, C, or less known like Pascal or Prolog. But the languages I’ve spent more time with are Objective-C and JavaScript. The main difference between all those languages and Reason is the exhaustiveness that you get from OCaml type checker. This is maybe awkward, but it feels like you stop coding alone and suddenly you have a sidekick always sitting next to you, that is helping you notice the things you forgot about, or found new code that is not consistent with code you or someone else wrote before.

In a world that is moving towards remote work, where many of us spend hours every day coding physically far from our colleagues, it makes the experience much more delightful. Plus, it allows for teams working on different time zones to keep a healthier work-life balance, because there is less need to have synchronous communication than with more dynamic languages, as more assumptions and design decisions are “embedded” into the code.

What’s your favorite thing about Reason?

Can I pick two things? It’s hard to choose only one.

The first one is the exhaustiveness and quality of the type checker, as mentioned above. Sometimes it takes a bit longer to build a feature than what it would in other languages, until the types are figured out. But this is largely compensated by the confidence one has when shipping code to production, or diving into large refactors.

The second one is the speed of the BuckleScript build system, which is built on top of ninja. I had never worked with such fast build system. As an example, we have recently started to use remote machines to develop at Ahrefs. In one of these machines that has 72 cores, BuckleScript takes roughly 3 seconds to clean build all our Reason code: application, libs, decoders… everything. Many tens of thousand lines of code! We thought there were something wrong, but we realized the compiler is just So Blazing Fast™️.

What’s your least favorite thing about Reason?

I guess we’re going through a necessary stage until things stabilize in the future, but there is a lot of fragmentation at the moment between “Reason native”, which tries to stay closer to OCaml, and “Reason web”, which has a goal to become friendlier for JavaScript developers.

I am excited to see what BuckleScript new syntax will lead to, but I would also love to see a “universal” solution that works for the main use cases out of the box, becoming sort of Rails for Ocaml or Reason. sihl is a project that seems to go in that direction and looks very promising.

How does ReasonReact compare to other frameworks you’ve used?

I consider ReasonReact mostly like React + types on top, because the bindings layer is very thin. The thing that I like most about React is that it follows the Unix philosophy: it does one thing and it does it really well. Maybe we have forgotten already today, but having to maintain and mutate UI based on data updates was one of the main sources of bugs in the past. The other nice thing is that there is so much good content about it: blog posts, documentation, etc.

Was it easy to pick up Reason? Why or why not?

It took some time, as with any other language. We have things like syntax or semantics much more ingrained into our brains than we think, so there is always some “rewiring” time that is needed to learn a new language, even if Reason makes an effort to stay close to JavaScript syntax. The most challenging part was probably the bindings one, because coming from JavaScript, there are no previous knowledge that one can use as foundation to build upon, it’s all “new knowledge”. glennsl BuckleScript ffi cheatsheet was a huge help for me.

Ze

I really like working with Reason, and have wanted to do so for a while. I was quite happy to see that working with it matched my expectations.

You get so much support from the type system, and still have a lot of flexibility to represent your domain model. Coming from other languages or paradigms, you don’t feel limited at all in what you can achieve.

The language has such a strong type system that you feel much more comfortable with your coding.

The OCaml type system is there to make sure you code with assurance. This is especially true when refactoring code. You can be sure that everything will work fine after it compiles. If it compiles, it works :)

It’s also very helpful when working on a monorepo. You don’t have to keep reading the source code of everything you use to make sure you don’t have types mistakes. Changes in code in one lib reflect immediately in all the others. This makes the feedback loop much shorter and safer.

The editors integrations with the type system are quite good and help a lot to write code better and faster.

Also, compilation times are super fast.

Last, but not least, ReasonReact is, for me, the hidden gem of ReasonML. The newcomers that have some difficulty with the language should start with it. IMHO, ReasonReact is simpler and has a better developer experience than React itself. It should be the gateway drug frontend developers need to get started with Reason/OCaml 😄

Liubomyr

To me, all those language features boil down to one essential thing, and it’s the easiness of refactoring. New business requirements popups all the time, and often your initial code assumptions are no longer correct. It was such a pain to modify code in a large JS codebase, as you never know how many things you potentially break in the process. With Reason, it has never been easier. If you need to change your data shape or some component API, you just do it, and from there, the compiler will guide you through all the places you broke, and help to fix those.

Coming from the JS world, it feels like the initial development is slower, because of the learning curve, missing bindings, less StackOverflow answers, but in the end, you are getting a stable software which is way easier to maintain and add features to.

Egor

I switched to Reason when I joined Ahrefs team about a year ago, before that I worked mostly with Ruby language.

The first thing that impressed me in ReasonML was code refactoring. Refactoring in language with a strong type system, like ReasonML and OCaml, is much easier than what I am used to. If your program compiles after your refactoring — most likely you did everything right, if it doesn’t compile — you can immediately see what you forgot to change. This can be achieved in languages with a dynamic type system only with a huge amount of code tests (supporting big test suite is a time consuming process as well as code support).

The other thing that I really like about ReasonML codebase — how readable it is. When you just enter into ReasonML world — some things can be unfriendly from the first sight, for example, immutable let bindings, but in the end, you realize that these language decisions help you to write cleaner and simpler code.

Seif

The programming language I used the most in the past is JavaScript. I switched to Reason when I joined Ahrefs a few months ago. From the start, I worked mainly on the code shared by the majority of the tools and I don’t think I would have had the same confidence making changes if I was doing it with JavaScript. I love JavaScript’s developer experience and accessibility. Reason provided me predictability without hurting these very same things I like about JavaScript.

Bryan

Reason (and OCaml) is, by far, one of the easiest languages to work with. Easy in the sense that the compiler helps eliminate an entire class of errors so you don’t have to worry about them. Additionally, in most other web-centric languages, it’s a pain to add features to existing code that you’ve not touched for a long time. With strong static typing, I can usually add the feature I want in either the backend or frontend, and then let the compiler tell me what needs to be updated.

Pattern-matching is one of my favourite features in Reason. To me, it makes more sense to be able to explicitly specify conditions that I’m interested in a clear and concise manner, and let the compiler tell me if I missed out a particular condition. Records go hand-in-hand with this. As software programs are made up of data and instructions, records are the perfect data containers. They are quick to define and query, focusing on data rather than behaviour (think classes and instance methods).

It definitely took a while to pick up Reason mainly because it takes time to become familiar with idiomatic OCaml. But once I crested that learning curve, everything just made sense and all the features of the language that made Reason seemingly difficult to learn — strong typing, the functional paradigm, etc, became assistants that helped me to write better code.

Feihong

ReasonReact is a great library for making complex UIs in a large codebase because you get the familiarity of React coupled with the type safety of OCaml. Having two well-established technologies in its foundation is a big advantage that ReasonReact has over other functional UI libraries/frameworks in the transpile-to-JS universe. I didn’t have any professional OCaml experience before joining, yet the ramp up was made much easier by my existing knowledge of React and the (somewhat superficial) similarity of the Reason syntax to JS. Oftentimes it was possible to correctly guess the intent of existing Reason code without knowing all the syntax, because most React concepts carry over pretty directly. And even though the documentation is incomplete and not perfect, it’s quite usable already and among conceptually-similar frameworks is second only to the Elm documentation.

The compiler errors were difficult to get used to at first. The compiler is fairly good at pointing out the location of the error, but not necessarily as good at explaining the nature or cause of the error. As such, having a REPL would be extremely useful. Actually, OCaml does have its own REPL, but BuckleScript (the compiler used by Reason to translate OCaml to JS) does not at the moment. Nonetheless, the Try Reason page is a really good tool to try out small snippets of code and is extremely useful while learning the language (we will still occasionally post Try Reason links in our slack channel).

Summary

The reality is that Ahrefs has always been an OCaml shop, but in the past OCaml was only used to build the backend. Now that we are also using it on the frontend, we get the benefits that our backend colleagues have enjoyed for many years: the expressiveness afforded by pattern matching, the ease of refactoring in large codebases, the stability of a mature programming language, and the confidence of “if it compiles, it works”. To make a shoddy nautical analogy, it is as if we had built a wooden ship powered by a turbo engine. But now the wooden parts are being replaced with steel and plastic, bringing the exterior of the ship up to modern standards as well. As a result, the ship runs faster and more reliably, making the passengers (our users) more satisfied. Also, pirates (bugs) have a harder time hijacking the ship because it’s sturdier and defended by well-disciplined camels. Because the ship keeps getting more and more passengers who want to experience a delightful ride and take pictures with enigmatic camels, we require a constant influx of willing and able boat engineers (who aren’t allergic to camels) to extend and maintain the ship. (Yes, that means that we are hiring️.)

Thanks to Raman and Louis for fact checking this post.

One and a half years of ReasonML in production was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

How to write a library for BuckleScript and Native

ahrefs — 2019-10-22T10:09:09-00:00

Written with Javier Chávarri and Feihong Hsu.

The first Reason Conf US just ended. Many talks mentioned native compilation. Sharing code between BuckleScript and native artifacts is a use case which is more and more common. This blog post is an introduction on how to set up a library available for both worlds, sharing as much code as possible.

The goal

What we try to produce is a library with an identical interface for BuckleScript and native. But without duplicating code. It should also be possible to have some parts of the library that are a different implementation depending on the target, as we want to be able to leverage existing libraries that are working only in one of the worlds.

The build systems

For BuckleScript, there is only one build system: bsb. It is driven by a bsconfig.json file. And is installed as part of the bs-platform.

On the native side, there are a lot of different build systems that are available. But recently one of them became a de facto standard: dune. It works with a very minimal amount of configuration. And it supports the reason syntax by default.

These two tools are working in a way which is pretty similar. They share a lot of concepts. And it is easy to set them up so that both are working in the same codebase.

The main similarities that interest us are:

The ability to work on specific source directories
Namespacing in bsb and wrapping in dune are both putting all the
files of the library under a single module name

The source code file tree

The code of the library is split into 3 directories.

├── js/
├── native/
└── shared/

shared is meant to host most of the code and all the code in this directory will be compiled in both modes.
js contains the parts that are specific to BuckleScript.
native contains the parts that are specific to native OCaml.

Set up the build systems

Once we have our basic skeleton for the library, it is time to set up the build systems. We want to have two configurations as similar as possible to make them easier to understand. Once we are done, the tree will look like this:

├── bsconfig.json
├── dune
├── dune-project
├── js/
├── native/
└── shared/

BuckleScript

At the root of the library we need a bsconfig.json file to drive
bsb. The documentation is available at https://bucklescript.github.io/docs/en/build-configuration.

The main part for us is sources. We will use it to tell bsb to look at the js and shared folders. We also want to set namespace to true, which will wrap all your project’s files under a common module name.

  "namespace": true,
  "sources": [
    {
      "dir": "js",
      "subdirs": true
    }, {
      "dir": "shared",
      "subdirs": true
    }
  ],

The rest of the file is as usual.

{
  "name": "sharedlib",
  "namespace": true,
  "sources": [
    {
      "dir": "js",
      "subdirs": true
    }, {
      "dir": "shared",
      "subdirs": true
    }
  ],
  "package-specs": {
    "module": "es6",
    "in-source": true
  },
  "refmt": 3,
  "suffix": ".bs.js",
  "generate-merlin": true,
}

Dune

We must also add a dune file to the root of the library. For dune, we have different options — it is possible to ignore the js directory but read everything else. Or to check only shared and native. To make the configuration similar to BuckleScript, we will go with the second solution.

The dune directive to do that is dirs. By defaults it tells dune to explore every directory except the ones hidden (starting with a dot) or starting with an underscore. More details in dune’s documentation. To make it do what we want, the configuration should be:

(dirs shared native)

We also use another option of dune to tell it to include the content of those two directories as if it was at the root of the project. Without this stanza, dune would only use the source files at the root of the project and ignore everything in the sub directories.

(include_subdirs unqualified)

Then we need the usual library stanza to give a name to our library, state the dependencies, compilation flags, etc. In our simple case, the only information needed is the name. We can explicitly set wrapped to true, but this is already the default behavior. The documentation for the whole library stanza describes how to specify more details.

The final dune file looks like this:

(dirs shared native)
 (include_subdirs unqualified)
 (library
  (name sharedlib))

We also want a basic dune-project. If we don’t write it by hand, dune will generate it for us. I am using version 1.10 as an example. But it can be changed to whatever version suits your project.

(lang dune 1.10)

Compilation

With the setup described above, the compilation for BuckleScript and native is the same as in a setup with only one or the other.

bsb -make-world for BuckleScript
dune build @all for dune

The call to bsb is usally put in package.json in the scripts part, so that the usual yarn build can be used. For native, it depends if you rely on esy or opam.

How to consume the library

This is exactly the same setup that would be used in a pure BuckleScript or pure native library.

To use your library in BuckleScript:

Add the name and version to package.json
Add the name to bsbconfig.json of consuming library/app

To use your library in native OCaml, add the name of your library to the libraries part an executable or library stanza, e.g.

(executable
 (name main)
 (libraries sharedlib))

Module naming

If you want your module name to contain capital letters in the middle (e.g. TeenageMutantNinjaTurtles), then be aware that name munging works differently between bsbconfig.json and dune. For example, if you want to refer to your module as CoolSharedLib in your code, then the name in bsbconfig.json must be cool-shared-lib, and in dune it must be coolSharedLib.

Platform specific code

The whole library does not have to be exactly the same in the two platform. It is possible to add modules that are available only in one mode. Or to have modules with a different interface.

For example, by adding a file Foo.re in js but not in native, the library now has a module Foo available when compiled to javascript. But only when compiled to javascript.

Downsides

Both bsb and dune generate .merlin files when they compile our library. They override each other. It might be troublesome if the version of ocaml used for native code is not 4.02.3. Simply recompile the library for your platform to solve the problem.
Out of the box, this approach doesn’t really allow us to share interface files between both platforms: native and BuckleScript. One workaround for that, if we wanted to share some module Foo, is to:
1. add Foo.mli or Foo.rei file in shared
2. add include FooImplementation in Foo.ml
3. add FooImplementation in both native and js folder
It’s not possible to be platform specific for just a few lines of code (e.g. if IS_NATIVE foo else bar), the minimal per-platform unit is a file/module.

Example project

We have set up a simple library to showcase what a repository looks like once the whole configuration is in place. It is available on github.

For now the repository contains only a library. But with this setup, it is actually possible to build an executable too. It is also possible to enrich it, for example by adding atdgen to communicate between both sides of the library.

How to write a library for BuckleScript and Native was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Getting started with atdgen and bucklescript

ahrefs — 2018-09-12T02:53:58-00:00

atdgen is a project to create types and data structures that can be serialized to JSON. It is very convenient when communicating between multiple processes, creating a REST API or consuming JSON objects from other tools. It can be compared to JSON schema or Protocol Buffers, but with richer types and more features.

The idea is to write a list of types in a specification file, an .atd file. Then running atdgen, it is possible to generate OCaml or Java code to serialize/deserialize values of those types to/from corresponding json.

Until very recently, atdgen could generate code only for native OCaml. But the support of bucklescript has been merged! atdgen the cli tool is still a native OCaml binary. But it can output some OCaml code that can be compiled using bucklescript.

The work to implement this new feature of atdgen has been funded by Ahrefs. We highly appreciate open source tools. And as much as possible, we prefer to contribute to existing open source projects rather than to re-invent the wheel internally.

Installation

To install atdgen we first need to install opam (OCaml package manager), as atdgen doesn’t provide ready to use binaries and is only distributed as source package via opam. The procedure is simple and documented here: https://opam.ocaml.org/doc/2.0/Install.html

Then we need to initialize opam and create a switch. Any version of ocaml greater or equal to 4.03.0 should be fine.

opam init -a
opam switch create . 4.07.1 -y

Once it is done, we have to install the development version of atdgen. The support of bucklescript is not officially released.

opam pin add atd --dev-repo   
opam pin add atdgen --dev-repo

Make sure that atdgen is available.

$ which atdgen                 
(current $PWD)/_opam/bin/atdgen

Of course, we need bucklescript.

yarn init                 
yarn add bs-platform --dev

We also need the bucklescript runtime for atdgen, as it is not currently provided by atdgen itself. So we have written and open-sourced our version of the runtime : https://github.com/ahrefs/bs-atdgen-codec-runtime.

This runtime is responsible for the conversion between JSON values and OCaml values. The JSON values are based on the standard Js.Json.t type provided by bucklescript to be sure that it is easy to interoperate with the rest of the ecosystem.

It is published on npm for easy integration in bucklescript projects.

yarn add @ahrefs/bs-atdgen-codec-runtime

Project configuration

After the previous section, package.json should be almost ready. We can add a few scripts to make it more convenient to compile the project. Here is how it should look once completed.

{
  "name": "demo-bs-atdgen",
  "version": "0.0.1",
  "description": "demo of atdgen with bucklescript",
  "scripts": {
    "clean": "bsb -clean-world",
    "build": "bsb -make-world",
    "watch": "bsb -make-world -w",
    "atdgen": "atdgen -t meetup.atd && atdgen -bs meetup.atd"
  },
  "devDependencies": {
    "bs-platform": "^4.0.5"
  },
  "peerDependencies": {
    "bs-platform": "^4.0.5"
  },
  "dependencies": {
    "@ahrefs/bs-atdgen-codec-runtime": "^1.0.4"
  }
}

The bucklescript configuration is very simple. We use the basic configuration that can be found in any bucklescript project. Except that we need to add one dependency to bsconfig.json:

{
  "name": "demo-bs-atdgen",
  "version": "0.0.1",
  "sources": {
    "dir": "src",
    "subdirs": true
  },
  "package-specs": {
    "module": "commonjs",
    "in-source": true
  },
  "suffix": ".bs.js",
  "bs-dependencies": [
    "@ahrefs/bs-atdgen-codec-runtime"
  ],
  "warnings": {
    "error": "+101"
  },
  "generate-merlin": true,
  "namespace": true,
  "refmt": 3
}

First ATD definitions

It is time to create a first .atd file, containing our types. This part is also documented on https://atd.readthedocs.io/en/latest/tutorial.html#getting-started

For this example, I decided to go with a meetup event. Put the type definitions in src/meetup.atd.

(* This is a comment. Same syntax as in ocaml. *)

type access = [ Private | Public ]

(* the date will be a float in the json and a Js.Date.t in ocaml *)
type date = float wrap <ocaml module="Js.Date" wrap="Js.Date.fromFloat" unwrap="Js.Date.valueOf">

(* Some people don't want to provide a phone number, make it optional *)
type person = {
  name: string;
  email: string;
  ?phone: string nullable;
}

type event = {
  access: access;
  name: string;
  host: person;
  date: date;
  guests: person list;
}

type events = event list

We use the atdgen binary (compiled previously) to generate the ocaml types and the code to serialize/deserialize those types.

atdgen -t meetup.atd # generates an ocaml file containing the types
atdgen -bs meetup.atd # generates the code to (de)serialize

The generated files are:

meetup_t.ml(i) which contain the ocaml types corresponding to our ATD definitions.
meetup_bs.ml(i) which contain the ocaml code to transform from and to json values.

At this point we can compile our project.

yarn build

If everything worked properly, we now have two .bs.js files in the src directory.

$ tree src
src
├── meetup.atd
├── meetup_bs.bs.js
├── meetup_bs.ml
├── meetup_bs.mli
├── meetup_t.bs.js
├── meetup_t.ml
└── meetup_t.mli

0 directories, 7 files

At this point, we can create new OCaml/Reason files in the src directory and use all the code atdgen generated for us. Two examples to illustrate that.

Query a REST API

A common usage of atdgen is to decode the JSON returned by a REST API. Here is a short example, using the reason syntax and bs-fetch.

let get = (url, decode) =>
  Js.Promise.(
    Fetch.fetchWithInit(
      url,
      Fetch.RequestInit.make(~method_=Get, ()),
    )
    |> then_(Fetch.Response.json)
    |> then_(json => json |> decode |> resolve)
  );

let v: Meetup_t.events =
  get(
    "http://localhost:8000/events",
    Atdgen_codec_runtime.Decode.decode(Meetup_bs.read_events),
  );

Read and write a JSON file

Atdgen for bucklescript doesn’t take care of converting a string to a JSON object. Which allows us to use the performant json parser included in nodejs or the browser.

let read_events filename =
  (* Read and parse the json file from disk, this doesn't involve atdgen. *)
  let json =
    Node_fs.readFileAsUtf8Sync filename
    |> Js.Json.parseExn
  in
  (* Turn it into a proper record. The annotation is of course optional. *)
  let events: Meetup_t.events =
    Atdgen_codec_runtime.Decode.decode Meetup_bs.read_events json
  in
  events

The reverse operation, converting a record to a JSON object and writing it in a file is also straightforward.

let write_events filename events =
  Atdgen_codec_runtime.Encode.encode Meetup_bs.write_events events (* turn a list of records into json *)
  |. Js.Json.stringifyWithSpace 2   (* convert the json to a pretty string *)
  |> Node_fs.writeFileAsUtf8Sync filename  (* write the json in our file *)

Full example

Now that we have our functions to read and write events, we can build a small cli to pretty print the list of events and add new events.

The source code of the full example is available on github.

You can run it like this:

$ echo "[]" > events.json
$ nodejs src/cli.bs.js add louis louis@nospam.com
$ nodejs src/cli.bs.js add bob bob@nospam.com
$ nodejs src/cli.bs.js print
=== OCaml/Reason Meetup! summary ===
date: Tue, 11 Sep 2018 15:04:16 GMT
access: public
host: bob <bob@nospam.com>
guests: 1
=== OCaml/Reason Meetup! summary ===
date: Tue, 11 Sep 2018 15:04:13 GMT
access: public
host: louis <louis@nospam.com>
guests: 1
$ cat events.json
[
  {
    "guests": [
      {
        "email": "bob@nospam.com",
        "name": "bob"
      }
    ],
    "date": 1536678256177,
    "host": {
      "email": "bob@nospam.com",
      "name": "bob"
    },
    "name": "OCaml/Reason Meetup!",
    "access": "Public"
  },
  {
    "guests": [
      {
        "email": "louis@nospam.com",
        "name": "louis"
      }
    ],
    "date": 1536678253790,
    "host": {
      "email": "louis@nospam.com",
      "name": "louis"
    },
    "name": "OCaml/Reason Meetup!",
    "access": "Public"
  }
]

Getting started with atdgen and bucklescript was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Skylake bug: a detective story

ahrefs — 2017-06-28T18:34:51-00:00

It was a dark and stormy night; the skylake CPU buzzed with excitement, and then, suddenly, the hyperthreads started to lock up..

Or something like that.

This week a new erratum for the Intel Skylake and Kabylake processors families was brought to public attention on the Debian mailing list, and then on various social media and news outlets.

We have been investigating this issue since January with the core OCaml team, as we were struggling with a mysterious bug affecting our developers machines, and ultimately our production system, resulting in a corruption of important data in our databases.

At Ahrefs, we operate a fleet of thousands of servers, running a wide variety of services (huge web crawler among others). At this scale, dealing with unexpected application behaviors is common. While we try to reduce the probability of the software not functioning as expected, bugs are sadly a real part of our everyday life. Even though we can assume the underlying hardware running any infrastructure can be thought of as more reliable and less prone to bugs than software components, issues can still arise in unexpected ways. When the number of servers increases, it is not unusual to observe faults in the hardware preventing the system from functioning as specified.

It is certainly not frequent to encounter such problems in CPUs but reading through the list of errata published by any manufacturer, each CPU model contains a fair amount of bugs. This story is about the bug in the microcode of Skylake processor leading to incorrect code execution under certain conditions. This is certainly scary at first sight: how can we trust our system if we cannot trust its main component ? Yet, like software bugs, processor defects can be identified, contained, and we can take actions to prevent them from impacting the operation of the infrastructure.

We do not know the full implications of this particular bug, especially security implications in case of untrusted code execution. But we’d like to tell the story of this erratum from our point of view, to provide some context, and show that dealing with it was not much different than dealing with any usual software flaw. While this post aims to cover our own perspective on this adventure, we would like to thank Mark Shinwell, Xavier Leroy, Frédéric Bour, everyone involved in the Mantis issue and the OCaml IRC channel for their help and time spent investigating with us. Update: Xavier Leroy told his own side of the story in another blog post.

Setting the scene

Our story starts in late 2016 after some of our backend developers received new laptops to work on. After a few days Enguerrand Decorne noticed unusual crashes during compilation of our OCaml codebase.

This issue, considered mildly annoying at first, seemed to affect only Enguerrand’s machine. For a few days no other machine would exhibit the same behavior, so we figured this was a fault specific to his system configuration.

However, concerns were subsequently raised after witnessing the generation of invalid machine code and later on, after the deployment of a service on one of our new clusters composed of Skylake Xeon processors, leading to the insertion of corrupted data into our storage system. The priority raised from the annoying level, to potentially critical. Other developers started working together to obtain more information and assess the impact on our infrastructure. Soon after we were able to reproduce the issue on several machines.

The remainder of this post is a technical description of the steps taken to ensure that our systems were operating safely. It is intended to show that such low level CPU issues is not necessarily fatal — in less than two weeks, with the great help of core OCaml developers, we identified the conditions of the crash and set up a workaround.

Tracking down crashes in OCaml

Most of our backend code is written in OCaml, a high level and expressive language supporting functional programming style (among others), which allows us to develop robust systems with ease, thanks to its strong type system and mature legacy.

The compiler segfaults were definitely a surprise, since this shouldn’t happen for any program written in OCaml, as type system and other features (such as automatic bounds-checking) usually guard us from such errors. However, stack overflows can be possible sources of segfault (when a non-optimal recursion is running too deep), so our first intuition was to increase the stack size when running the compiler. This didn’t change anything, and the reported fault address wasn’t anywhere near the stack address bound.

Before witnessing the crash on other machines, we suspected a failure in the virtualization software used by our two developers that were able to reproduce the crash, who use VMware as a part of their development workflow. We tried early on to switch to Virtualbox, but the migration proved itself fruitless as the crashes kept appearing. After a short while we began encountering the same issue on physical machines, so we ruled out a possible virtualization software bug.

The usual debugging process for crashing OCaml code didn’t prove effective — we needed to narrow down our approach.

OCaml ships with two backend implementations: a bytecode interpreter and a native compiler. We were able to reproduce the issue using both a native compiler and a compiler running on the bytecode interpreter. Consequently, this ruled out a miscompilation coming from the code emitted by the compiler, the OCaml runtime itself was misbehaving.

The runtime code is written in C, and implements low level functionalities, including the garbage collector used by both backends. After rebuilding the runtime with debug symbols, we were able to retrieve a proper stack trace and core dump. The stack trace pointed to the garbage collector’s mark phase. OCaml’s GC is a classic generational mark and sweep collector. The mark phase walks the heap starting from pointers on the stack and other registered root values, and marks every reachable block of memory.

Further inspection with gdb of the frame and address of crash revealed that the marking code encountered a corrupted block header with invalid size information, causing what looked like a buffer overrun error. Each memory block allocated in OCaml heap begins with a header word, storing metadata used by the GC, including a tag describing the kind of value present in this memory block. The header contains the size of the block, and the crash happened when the mark code was attempting to scan an array which was supposed to be more than 1TB large.

This was obviously not the cause of the problem but rather the consequence: something corrupted the header word after this block had been properly allocated, postponing the crash until the next GC cycle. It was the right time to escalate the issue to the OCaml bugtracker, after isolating a proper test case to reproduce the issue.

A set of strange leads

Escalating the issue to Mantis made us to take a step back and gather our findings, and we quickly got great feedback from the OCaml core team.

At this point, what does the problem look like?

We only had sparse information, but dmesg gave us interesting data point. When a page fault occurs and the kernel detects an incorrect memory access, it logs a line in kernel log buffer containing the fault address, the instruction pointer and stack pointer.

[22985.879907] ocamlopt.opt[48221]: segfault at af8 ip 00005564455169bd sp 00007ffc9f36b130 error 4 in ocamlopt.opt[556445006000+613000]

Next to the 3 addresses, already available in the coredumps, an error code is reported. This number in decimal form is actually a bitset, and the flags are documented in the Linux kernel sources in arch/x86/mm/fault.c. Error 4 can thus be read as a read access page fault from user mode, trying to read memory which had not been previously mmap’ed.

Error codes reported following our crashes involved protection faults or access to unmapped addresses, which corroborated our earlier buffer overrun hypothesis. More interestingly we witnessed a crash with the PF_RSVD flag enabled. This left us puzzled, none of us had ever seen such fault before. Apparently it indicates that the the page table was somehow corrupted, with some entries having non-zero bits reserved by the x86 architecture specification.

It was scary that the corruption would escape the process address space, and to our limited knowledge, it could only have been caused by kernel issue or potentially hardware issues, like memory errors. Yet we were able to reproduce this on several machines with different kernel version, and different hardware. We blamed virtual machines earlier but this theory was debunked already. We still have no explanation at this time, and pursuit on this front would require intimate knowledge of virtual memory implementations that we didn’t have.

One developer wasn’t able to reproduce the problem at all on his machine after hours of testing, but something was fishy: it didn’t sound right that an OCaml runtime bug would be able to modify the page table. Maybe it was some corner case with reserved addresses, but this something was beyond our reach here. Out of ideas, it was time to get some assistance from tools intended to track memory corruptions, like asan and ubsan.

Running Asan didn’t yield any meaningful results. Valgrind was later tried, following advises from the OCaml team, but every tools were preventing the crash. Quickly reproducing the bug for testing required running code in a loop, keeping the CPU and memory fully busy.

This was harder to do on developers machines, due to limited resources and other processes running, and Address Sanitizer would only increase the resources usage. Dedicating a powerful server would make further investigations more comfortable, and increase the likeliness of reproducing with instrumented code.

But with great surprise, it was not possible to reproduce the problem on a server machine, with and without instrumented code. This is when we realised that all the machines exhibiting the crashes were running a processor of the Intel Skylake processors family, while the server and other developer machines had CPUs from the Broadwell family.

The hardware, an unusual suspect

In the meantime several core OCaml developers had been closely investigating the issue and started auditing recent changes in the runtime, and identified a few suspicious changes and known bugs.

Certainly they were more qualified for this task, but it acted as an incentive to examine the history of this bug from our angle. At first, we had assumed that the bug was specific to the new laptop with virtual machines. This could not explain why the crash never manifested on older workstations equipped with Skylake processors. Several other developers had been using them for a few months, and only noticed the crash after awareness of the issue had been raised by Enguerrand.

What had changed, besides Skylake? Only a few week before, an internal migration from OCaml version 4.02.3 to 4.03.0 was rolled out in our codebase. Intrigued, we went ahead and tested OCaml 4.02.3 again, which showed no memory corruptions after several tests. It was time to browse the OCaml changelog for runtime related entries. The search stopped quickly on a promising item in the list: the OCaml C runtime build optimisation level had been increased to -O2 from -O1.

Could the optimizations dig out an undefined behavior in C code, leading to bad assumptions in the GC code corrupting the heap ? Rebuilding the runtime with -O1did not corrupt memory, so the source of the corruption was in the runtime and was triggered by some gcc specific optimization pass. This sounded like undefined behavior, although the information we had led us to some hardware bug.

The next day, Xavier Leroy commented on the bugreport reporting that the crash had been observed in the past. Another industrial OCaml user was affected, and they had discovered HyperThreading was part of the necessary conditions. After running the test case for several hours on several machines with HT disabled in the UEFI setup, it was clear we were facing a similar situation. This led to the hypothesis of a hardware bug:

Is it crazy to imagine that gcc -O2 on the OCaml 4.03 runtime produces a specific instruction sequence that causes hardware issues in (some steppings of) Skylake processors with hyperthreading? Perhaps it is crazy.

This possibility had struck us too, motivated by the HyperThreading, the page table corruption and the Skylake specific set of conditions.

This issue had certainly a strange profile. But nobody was ready to fully embrace the cpu bug hypothesis yet. We convinced ourselves that disabling HT could affect cache pressure and unfold some undefined behaviours.

HT could also explain the non-determinism, since cache pressure would depend on timings and scheduling. None of us had sufficient experience in this area to assess the strength of such hypothesis, and we did not quite buy it on a single threaded OCaml program. Our debugging motto claims that “assumptions are not facts”.

It was time to browse Intel errata list and attempt to update the CPU microcode. Although, the errata descriptions are formulated in vague terms, none of the issues disclosed at this time were looking similar to the situation under investigation. Unfortunately, CPUs microcode had no fix waiting for us either. OCaml developers investigated the errata list from their side but the lack of detailed information turned this into a fruitless and complex task.

In the absence of better alternative, we focused our work on pinpointing the exact source of the crash as if it was a software bug, in the hope of either finding a code issue or ruling out this hypothesis while getting more detailed data. We needed a way to identify the problematic code and find a workaround. From our side, it was not only a matter of finding whether or not there was a bug in OCaml code, but more crucially we needed a guarantee on the quality of our generated code running critical services in production.

Identifying the offending code

The other OCaml user affected by this issue reported that they had solved the problem by switching to another C compiler. Building the runtime with clang instead of GCC would prevent the GC from crashing. They also suggested to obtain a diff of the generated assembly. Indeed, once built with clang, the runtime would not crash. But clang generates widely different assembly from GCC and we did not have the resources to analyse several hundred thousand lines of changes.

If we could isolate the problematic C code, comparing the generated code would be easier. The problem had the form of a well known nail:

Around 50 C files composing the OCaml runtime,
There is a good state (when built with gcc -O1)
And a bad state (when built with gcc -O2)

This nail comes with a precious hammer: bisection.

The bisection approach had a downside in this occasion. Any state can be labeled bad with certainty as soon as the test crashes, but we would need to wait several hours to be confident enough to trust a non crashing test as good data-point. The reproducibility was not always consistent and a non-crashing state could be a false negative still waiting to trigger the conditions leading to the crash. A reduction of search space was necessary.

All the coredumps we had showed that the fault was caused by a corrupted heap block header, and our testcase involved the compiler. The OCaml compiler is not 100% deterministic, and IO/s primitives and unix environment in the runtime can affect timings and allocation patterns. But it sounded sensible to assume that the code corrupting a heap header block was also the code reading and writing those blocks: the major GC.

This hypothesis made bisecting fast: the first file we tried, major_gc.c, turned out to be the one. To make sure it was not a subtle issue in linker, reordering symbols or code blocks, we tried a few others files and confirmed changing the optimization level of some other files alone made no difference.

But the generated code difference was still way too large. Bringing this topic up on the OCaml IRC discussion channel led to some useful inputs. We were taught that gcc supports an attribute to enable specific optimizations at the function level, using __attribute__((optimize("options,..."))). Following the same strategy, it was easy to trace the source of the malfunctioning code to the sweep_slice function, which implements the sweeping phase of the classic mark and sweep garbage collector for the old generation.

Ignoring the subtle details of incremental GC, the sweep_slice function is the last pass of a normal major collection cycle. It is responsible for scanning all blocks in the major heap, and reclaiming unreachable blocks to the list of unallocated space.

The bulk of this function is a switch taking action for each block depending on its status :

https://medium.com/media/cfbc19fddccc5f2c1a76fcc802fae049/href

This finding felt consistent with the information at hand. When the block is reachable, the color (describing the reachability status of the block) is reset. If the block became unreachable (while color) - it is reclaimed. In both cases, the block header is modified.

Getting back to the assembly diff.

Nobody in the team knows a great deal about assembly and we only have a really basic understanding of most of the instructions used in both versions. It quickly became obvious that the noise level in this diff, with thousands of lines of changed, was still too high for us to spot anything related to the problem. This problem was getting far beyond the common knowledge of everyone in the team.

But this was still sounding like your day to day bug tracking process. The less you know, the more careful you need to be, tackling the problem step by step. We stuck to what approach had served us well until now: bisecting.

We went through the list of optimisation passes enabled by GCC at -O2. This is a fair amount of optimisation passes and it would have been too time consuming to try them one by one, given the time needed to trigger the crash. Yet we had a hint: a memory corruption was happening semi randomly in the garbage collector. We were also keeping the undefined behaviour bug as a potential explanation. It was likely a pass which would change the structure of the code, reordering blocks and changing conditions.

After reading the description of all switches in the detailed gcc manual, the -ftree-* pass family looked promising. This set of transformations works on the SSA form internal representation, a widespread intermediate language representation which has the benefit of being easy to read. They seem to make a huge impact on the generated assembly code, moving code blocks around and making assumptions on code invariants in order to move around, simplify or eliminate conditional checks altogether.

By looking at output of those passes on the related source code, we narrowed down the list of transformations to a couple of interesting passes, one of them being -ftree-vrp, which stands for Value Range Propagation. This pass computes bounds for each name binding and propagates proofs that a value must lie in a given range.

It turned out most of the other passes depended on it for further optimisations. Even though the issue ended up not being a bad assumptions in the range values, checking this pass proved to be worthwhile: enabling -ftree-vrp on sweep_slice function while every thing else was built with -O1 was enough to trigger a crash.

GCC provides very good diagnostics output, and after reading the manual we found the -fdump-tree-* switch to dump the SSA form before and after specific pass. The output is designed to be read by a human and provides meaningful naming, with source code locations, alongside the ranges propagated by the VRP pass. We spent some time studying the output and matched the difference in SSA tree between the crashing and not crashing code.

Examining the bounds and invariants derived by gcc, it was clear that no wrong hypothesis was stated.

https://medium.com/media/a5a311d0b5a4f890f9541f0aed91e73e/href

The only meaningful observable change involves the suppression of rechecking the loop condition in the else branch of the sweep_slice function, after Value Range Propagation proved that the condition was invariant in this branch.

Often, reading the code carefully is the fastest way to find a bug. But after spending hours staring at the major GC code, it was clear enough that this check removal should not cause any semantic changes.

In this process, we identified a suspicious bit of code, where a signed long variable was promoted to unsigned according to C standard rules, which was changing the bounds derived by gcc, assuming it was always positive. But after some thinking we realised it made no difference at assembly level and although wrong, this assumption was not used anywhere.

We were now ready to rule out the possibility of a bug in OCaml runtime. It was still possible that GCC backend had a bug and was miscompiling this particular shape of code. And we were back at the assembly level again. After writing some awk formatting script to cleanup assembly and minimise noise in the diff (by renaming labels, detecting spurious code move, etc), and preventing inlining, we found a minimal assembly patch causing the crash.

There were only cosmetic differences. The test removal was propagated down to assembly and caused gcc to reorganise the layout of each switch case block.

https://medium.com/media/532ab4896b11177a96203fd0c81eaa58/href

Among those minor differences and changes of layout, we noticed a particular change which impacted exactly the reachable block header updated which could have caused header corruption. In the unoptimised version, the updating code looked like this:

https://medium.com/media/38ca30a1decf86233084721c3803b1c4/href

For some reason, the block pointer was spilled to the stack.

Perhaps naively, and because we had earlier emitted the hypothesis of HT impacting cache pressure, we spent a few hours staring at this code and check if we were missing something subtle which could affect the control flow of the whole function and the stack location from which it was reloaded could be corrupted.

Despite our lack of assembly knowledge, after spending several hours reading this tiny change, we got convinced that it made strictly no semantic difference. Reading the x86 manual carefully didn’t give any hint on any subtle behavior which would trigger. Executing any of those two sequence of instruction should give the exact same output.

Mitigating the issue

We were now quite certain it was a CPU bug.

The OCaml developers had reached the same conclusion, and were working on escalating the issue to Intel. After internal discussions we decided to keep this bug as low profile as possible since we were unsure about potential security implications, especially for JIT implementations.

Even if we had no confirmation at this point nor any explanations of the cause of this bug, which was beyond our reach, we could take actions.

The first step was to decide against getting any new Skylake based servers until further announcement. We were left with several Skylake machines but we refrained from deploying any OCaml code on them. OCaml comes with a great package manager, opam, which supports compiler switches. Switches allow to set up a clean and distinct environment with specific packages and compiler configuration.

We patched our internal opam repository to distribute unoptimised runtime to all developers and moved forward, waiting for further announcements.

This situation made us realise that microcode requires constant updates, just like any other software in the stack. We raised awareness on this topic in our devops team, and they took measure to ensure we could roll out updates to prod easily.

Happy end

In late May, devops team noticed a debian package update for intel-microcode containing the following change:

Likely fix nightmare-level Skylake erratum SKL150. Fortunately,
either this erratum is very-low-hitting, or gcc/clang/icc/msvc
won’t usually issue the affected opcode pattern and it ends up
being rare.
SKL150 — Short loops using both the AH/BH/CH/DH registers and
the corresponding wide register *may* result in unpredictable
system behavior. Requires both logical processors of the same
core (i.e. sibling hyperthreads) to be active to trigger, as
well as a “complex set of micro-architectural conditions”

The erratum description immediately rang a bell as it matched the diff in the assembly we had observed. We tested the microcode update and confirmed it fixed the corruption.

Finally, our Skylake CPUs were feeling safe and OCaml compiler was happy.

Ahrefs runs an internet-scale bot that crawls the whole Web 24/7. Our backend system is powered by a custom petabyte-scale distributed key-value storage implemented in OCaml (and some C++ and Rust). We are a small team and strongly believe in better technology leading to better solutions for real-world problems. We worship functional languages and static typing, extensively employ code generation and meta-programming, value code clarity and predictability, and are constantly seeking to automate repetitive tasks and eliminate boilerplate. And we are hiring!

Skylake bug: a detective story was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

EmelleTV: Talking with Louis Roché about OCaml and Ahrefs

ahrefs — 2023-06-29T14:32:40-00:00

https://medium.com/media/aef5aa9cb9b71aaafb0a07cb48cb312d/href

Transcript

Louis: [00:00:59] Good, and you?

David: [00:01:01] Good. Very good. You obviously work at Ahrefs. At what team do you work?

David: [00:01:48] Nice.

Louis: [00:01:49] We’ll see how it goes.

David: [00:03:35] That’s the whole idea. You have been been writing OCaml for a long, long time. That’s fair to say.

Louis: [00:03:40] I’ve been writing OCaml since I’m 16 or 17 was when I wrote my first line, like 14 years ago, something like this. There was no Merlin at the time.

David: [00:05:25] Right.

David: [00:06:04] Build your own library for http.

Louis: [00:06:05] Yes.

Louis: [00:06:35] When you have no choice, you do it. You don’t know that it’s harder, you just see “Oh, I can do it,” and you do it.

Louis: [00:06:57] No, go ahead.

Louis: [00:07:49] I don’t know if that’s true. Partially, it’s a strategy of Ahrefs, so I have to say it’s true.

David: [00:07:55] [laughs]

David: [00:09:05] Right. It’s very unixy. The start of OCaml is very unixy. That’s true.

David: [00:09:59] All right.

Louis: [00:11:56] I’m not a big Twitch person. I don’t know [crosstalk] modern influencers.

David: [00:12:02] That’s true.

David: [00:12:32] [laughs] Okay. Would you say that Go is crap?

Louis: [00:12:38] Go is a language. I haven’t used Go 2 extensively.

David: [00:12:42] This is recorded. This is not a beer in Singapore. This is recorded. You can obviously bash go, that’s part of the game.

Louis: [00:12:50] Let’s say Go is not the most modern language there is.

David: [00:12:54] Right. Thank you. This is just for the headline. We don’t want the headlines- because Ahrefs is going to be like — No, I’m joking. [crosstalk] Yes, go on. Sorry.

David: [00:13:38] Yes, that’s true.

Louis: [00:13:39] I definitely think it has an influence.

David: [00:13:43] That’s true. From the community, how have you seen the Reason creation and adoption from your point of view? You can bash Reason if you want.

[00:13:43] [crosstalk]

David: [00:14:53] Right.

David: [00:16:36] Yes, definitely and you mentioned BuckleScript?

Louis: [00:16:40] Yes. BuckleScript, they didn’t have —

David: [00:16:42] That was not so well received.

David: [00:17:50] Yes, for the record, Louis was running the OCaml Paris Meetup, I think. Yes, go on with the story.

David: [00:18:35] Yes, I think I wouldn’t say that BuckleScript died. It’s more like BuckleScript has been working for seven years, I think.

So they don’t need to upgrade the compiler, the GCC compiler, for example. He thought the same idea can apply to OCaml, we don’t need to follow the upstream compiler all the time.

David: [00:19:27] Right. Yes.

Louis: [00:19:29] Most companies actually they don’t want to change compiler version. They want something stable. They want no surprise, which has some value, or the stability has some value.

David: [00:22:57] Yes, that’s true.

Louis: [00:23:00] The people behind Merlin are super smart. In a way, it’s not a surprise.

David: [00:23:05] Right. You actually contributed to the LSP and dune, to both projects, I saw your contributions.

Louis: [00:23:13] Yes, I have commits on many small- it’s mostly small contributions, but I have commit on everything, I think at some point.

David: [00:23:19] Right.

David: [00:29:35] Somehow risky, yes. That’s true.

David: [00:30:00] Right. It’s big enough, but it’s not populated with everything.

David: [00:34:56] 63 right? That’s the —

Louis: [00:34:59] Yes, one bits for the right GC, and then we have 63 bits for the value.

David: [00:38:06] That’s exactly my experience.

Louis: [00:39:10] I guess the first question is what is a crawler?

David: [00:41:48] You would loop. If you don’t do it properly, you would loop forever.

David: [00:44:35] First of all, what technology is that running on?

Louis: [00:44:40] I think that’s a combination of different technologies. That will be a ClickHouse and then some internal database. Custom stuff.

David: [00:46:43] Yes. That’s insane.

David: [00:48:01] Did that ever happen?

David: [00:49:28] Right. That’s interesting.

David: [00:49:48] Yes, you treat it differently or?

Louis: [00:49:50] It’s fairly automatic and- yes.

David: [00:51:08] Imagine trying to create a web engine just to see a page from the server. That’s very good. What’s the favorite part of working at Ahrefs?

David: [00:52:32] That’s good. What’s your favorite part of OCaml?

Louis: [00:52:37] Of OCaml?

David: [00:52:38] Yes. You can say the people.

Louis: [00:52:41] When it compiles, it works. That’s the key point. Unlike Go, we have some types.

[00:52:41] [laughter]

Last year we outgrow our neighbors, our language neighbors, I think it was Scala, maybe, Haskell. I think we outgrow them. What do you think about the conference?

David: [00:56:23] Please do. As core contributor of the code of conduct, please do.

David: [00:58:05] Right, because this year is on Seattle. Every year it changes the location.

Louis: [00:58:13] Yes.

David: [00:59:45] Right. Otherwise, you would have crazy bugs. Data corruption

David: [01:03:53] Okay.

David: [01:06:33] Okay.

Louis: [01:06:35] This is not the only company with a structure that is not well defined.

David: [01:06:44] Right.

David: [01:07:16] Right.

Louis: [01:09:09] I guess it’s not politics looking for power because there is no power to gain.

David: [01:09:16] Exactly.

Louis: [01:09:17] What do you want to own? There is nothing to own. You can try, but there is nothing to win at the end.

Louis: [01:09:43] I know what Melange is.

David: [01:09:46] Definitely.

It’s completely moved to Melange. This is amazing. I’m able to compile all this code in one comment. I go in the repo, I do “make dev” and everything works.

David: [01:10:56] These and many more advances, but yes, that’s the part that it’s funny.

David: [01:12:21] On those tools, you would always prioritize the developer experience, rather than technical merits? How would you choose —

Louis: [01:12:33] As a user or as a developer of those tools?

David: [01:12:36] As a developer of those tools.

David: [01:13:19] I see.

David: [01:14:53] You would want different ways of combining to Javascript? Because the sane competition? That’s true-

David: [01:15:34] Yes, it would give us type safety from front end all the way down. Sorry, backend all the way down.

Louis: [01:15:43] Yes.

David: [01:15:43] Sorry, go on.

David: [01:16:34] Right.

Louis: [01:16:35] I don’t think that Melange aims to do that. Because then —

David: [01:16:41] I think that’s the magic. Yes, I agree.

Louis: [01:16:44] In Melange you will have one file per module or something like this, which means I will need to run through webpack or something like this later on.

David: [01:16:51] Yes, you could but you would face a few problems. Marshall for example, that it’s the encoding/decoding on bytes, that doesn’t work in Melange.

Louis: [01:17:01] Well, it doesn’t work in js_of_ocaml I think too.

David: [01:17:05] Yes, but I think you can stub it, right? I think you can —

David: [01:20:10] You have like a wrapper, right?

David: [01:20:57] Do you remember that I said that every time that I’ll talk with a backend person, they always mention the runtime. Exactly that moment. You always think about the runtime.

Louis: [01:21:06] Actually, I’m not sure.

David: [01:25:14] Could have worked. I agree.

Louis: [01:31:10] I don’t think it’s pure in any way, shape, or form.

David: [01:31:17] You can write pure code, right?

David: [01:31:37] Right.

David: [01:32:18] Right.

David: [01:32:51] Yes.

David: [01:34:06] Yes, solve exactly that issue.

David: [01:34:57] Right.

Louis: [01:37:41] You would be happy with more purity?

David: [01:37:45] Would we be happy with more purity? No, I think —

Louis: [01:37:49] Would you wish to have a language that is closer to Haskell that is like —

It goes against many things that happened for 20 years, more or less. You have to make both of those worlds still somehow work together.

Of course, with all the tradeoffs, if you go 10 times — 10 years back and you say to a person, “No, we call all the time same endpoint.” You will say, “You guys are stupid.”

Louis: [01:41:58] RESTful was a trend.

David: [01:49:56] Yes. That’s very noble.

David: [01:51:44] You mean, the actual UI, the CLI?

Louis: [01:53:35] This exactly why there is no Rust code inside Ahrefs, because all the developers got stolen by the blockchain companies.

Louis: [01:54:07] It’s a pleasure.

Louis: [01:55:07] Thank for having me. That was fun.

David: [01:55:10] See you guys.

[01:55:13] [END OF AUDIO]

EmelleTV: Talking with Louis Roché about OCaml and Ahrefs was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Ahrefs is now built with Melange

ahrefs — 2023-05-03T19:06:58-00:00

OCaml, all the way down

Photo by Lance Grandahl on Unsplash

Some history

Build speed would increase due to less work needed to parse dune files, and more efficient rules planning and execution.
Developer ergonomics would get better, as Melange would become a first-class citizen in Dune, with concepts like Dune libraries and other stanzas becoming available to Melange users.

Heads down

During the following months, we focused on two tasks, iterating over multiple cycles where the progress on one task would inform the next steps to take for the other:

Evolve Dune to add stanzas, fields, and documentation to support Melange projects.
Migrate Ahrefs’ frontend codebase to use the Melange compiler and Dune, adapt third-party libraries and bindings to Melange, and polish the editor integration, build scripts, and other aspects of the development experience.

Migration strategies

Timings

Keep in mind that Ahrefs frontend setup has specific characteristics, which affect the performance measurements:

Before migration: Dune generated ml files from atd files, then ReScript build tool bsb built all hand-written source files plus the ones generated from atd files.
After migration: everything is built with Dune and Melange.

All measurements were taken on a node with 2x AMD EPYC 7742 cpu @3.2 GHz (nproc=256), 1TB RAM, Debian 11 x86_64 GNU/Linux. The build target is always the entire Ahrefs frontend codebase.

Cold build:

Before: real 0m28.232s, user 9m23.883s, sys 13m33.939s
After: real 1m14.208s, user 10m33.708s, sys 5m45.644s

Warm build, noop (no file is built):

Before: real 0m14.687s, user 3m17.058s, sys 3m57.903s
After: real 0m21.895s, user 0m20.528s, sys 0m1.372s

Watch mode, modifying an “edge” file with almost no reverse dependencies:

Before: 1002ms
After: 1576ms

Watch mode, modifying an “inner” file belonging to a library, with many reverse dependencies:

Before: 7032ms
After: 15394ms

Conclusions

The results so far are quite encouraging. These are some of the things that are possible thanks to the deeper integration between Dune and Melange, and its application within the Ahrefs codebase:

The same OCaml compiler is used on both frontend and backend codebases.
Access to all the bug fixes, error improvements, and new features that the OCaml compiler team added between versions 4.06 and 4.14 of the compiler.
A shared developer environment across teams, including editor extensions, OCaml LSP server, etc. No more need to maintain a different set of tooling for backend and frontend.
Removal of hand-written CI checks that were ensuring different tools in the frontend codebase would not access components from other tools. This is now solved by Dune libraries, and the OCaml compiler will complain if logical units try to reach outside their bounds.
Frontend and backend shared dependencies, such as anuragsoni/routes, can now be defined in a single place: an opam file.
Faster rebuilds and better watch mode, as Dune now controls all the build artifacts. Previously, Dune and ReScript were sharing responsibilities, which was leading to unnecessary rebuilds of some artifacts. Or alternatively, rebuilds were not starting when required due to the build system not tracking changes in some subsets of the sources.
Easier PPX maintenance, as there is no longer a need to publish pre-built versions of these tools.
Melange allows to run all ppxs from a single executable file, which has some nice performance benefits.
All the other advantages of using Dune: virtual libraries, watch mode, leverage integrations with tools like odoc…

What’s next?

Now, we have to make it easier for other people who are willing to try and use Melange and Dune. So our focus is shifting to documenting how Melange works.

How to contribute?

Otherwise, if you are missing features, find bugs, or run into confusing errors, please open an issue in the Melange public repo.

Ahrefs is now built with Melange was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

How to get started with OCaml in 2022

ahrefs — 2022-10-31T16:13:10-00:00

Intro

Best resources for learning OCaml

How to install OCaml

Editor support

Tips

Conclusion

How to get started with OCaml in 2022 was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Monorobot: a Slack bot for monorepos

ahrefs — 2021-12-09T15:19:04-00:00

Monorobot: a notification bot for monorepos

Monorobot enables configurable directory tree notifications for your monorepo.

A few years ago, we decided to move most of our code into a monorepo. Many advocates have highlighted its upsides, which include better cross-project coordination and simpler dependency management.

And now we’re open-sourcing Monorobot, for anybody to use in their monorepo setup! The package is available via OPAM:

opam install monorobot

Read on for more details about the motivation, an overview of the main features, and what’s in the pipeline.

Existing Slack integrations lack monorepo support

Suppose we operate various camel-related services, and we’re planning to launch a new camel ride-sharing app called Camel Ride. The directory structure could look like this:

monorepo/
| frontend/
| | camelride_ui/
| | | mobile/
| | | web/
| | cameldance/
| backend/
| | camelride/
| | | routing/
| | | pricing/
| | camelfood/

As you can see, both the frontend/ and backend/ directories contain code for our fictitious ride-sharing service, along with code from other projects.

Enter Monorobot

Monorobot, hard at work.

For pushed commits, it checks the path prefixes of the files modified in the commits.
For activity related to PRs and issues, it checks their labels.
For status updates on pushed commits (e.g., CI builds), it uses the same path prefix logic as pushed commits.

Additionally, Monorobot supports unfurling GitHub links shared in Slack.

Path prefix routing for commit push notifications

Continuing with our example, suppose we want to route all commit activity related to the Camel Ride project to a Slack channel called #camelride. Our configuration file might look like this:

{
  ...,
  "prefix_rules": {
    "rules": [
      {
        "match": [
          "frontend/camelride_ui/",
          "backend/camelride/"
        ],
        "ignore": [
          "frontend/camelride_ui/images",
        ],
        "channel": "camelride"
      }
    ]
  }
}

{
  ...,
  "prefix_rules": {
    "rules": [
      {
        "match": [
          "frontend/camelride_ui/",
          "backend/camelride/"
        ],
        "ignore": [
          "frontend/camelride_ui/images",
        ],
        "channel": "camelride"
      },
      {
        "match": [
          "backend/camelride/pricing/"
        ],
        "channel": "camelride-pricing"
      }
    ]
  }
}

There are additional configuration options for prefix rules (and for label rules discussed in the next section) that aren’t mentioned here. Visit the repository for the full details.

Label-based routing for PRs and issue notifications

{
  ...,
  "label_rules": {
    "default_channel": "notifications",
    "rules": [
      {
        "match": [
          "Camel Ride"
        ],
        "channel": "camelride"
      },
      {
        "match": [
          "Price Optimization"
        ],
        "channel": "camelride-pricing"
      }
    ]
  }
}

The default_channel field provides an option to fall back on a channel if no rule is matched; this option is available for prefix rules as well.

Status notifications

Link unfurling

Finally, Monorobot can unfurl links to GitHub repositories shared on Slack (including private ones, if a personal access token is provided). This applies to commit, issue, and pull request URLs.

What’s next

Monorobot is actively used at Ahrefs today, but there are lots of promising future directions it could take. Here, we list a few.

Unifying GitHub and Slack identities

Consolidating notifications

Better status notifications

A CI build failure can have multiple potential causes:

A bad commit
A previous bad commit that has yet to be fixed
An issue with the pipeline itself (this is out of scope for Monorobot)

Wrapping up

Monorobot is written in OCaml. We welcome your feedback and contributions on GitHub!

P.S. If anyone does make an actual ride sharing service for camels, do let us know…

Thanks to Feihong, Igor, and Louis for feedback on this post.

Monorobot: a Slack bot for monorepos was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Building Ahrefs codebase with Melange

ahrefs — 2021-05-18T15:24:20-00:00

Photo by Jens Lelie on Unsplash

ReScript: becoming its own language

It has now its own parser, incompatible with OCaml native applications
Official repository guidelines for technical writing mentions explicitly that no reference to OCaml should appear in docs
Upgrades to the latest version of OCaml compiler, which used to be part of the roadmap, have been deprioritized recently.

Melange: a fork of ReScript, focused on OCaml compatibility

When António Monteiro announced Melange, a fork of ReScript but with a strong focus on keeping compatibility with OCaml, we decided to try it out and see how it could work for us.

Ultimately, the experiment was successful. We managed to build all our frontend applications with Melange, while keeping the existing bundling setup, which currently uses Webpack.

Throughout this process, we had to modify some parts of the code. We will now go through the most relevant parts of the process:

Upgrade to OCaml 4.12: the most relevant part was the deprecation of Pervasives module to use Stdlib.
Use ppxlib in our ppxs: we had to upgrade the two ppxs that we use in the frontend codebase to the latest compiler version, bs-emotion-ppx and an in-house ppx for internationalization.
Configure esy: we were already using esy to bring the editor tooling into scope of the developer environment, so we just had to make sure melange would also be included in the json configuration.
Upgrade to Reason 3.7.0: a quite simple change too, as the whole process is automated by using refmt. As a side note, we ran into a small bug with some type annotations, that we were able to work around.
“Lift” dune workspace to the root of our monorepo: this is probably the most intrusive change. Because we have shared code between backend and frontend, and Dune needs to have access to all sources under its workspace, we had to “lift” the Dune workspace from the backend directory to the root of monorepo.

The good

This experiment allowed us to experience what a project like Melange could offer for our use case. Here are some of the things we might be able to leverage in a codebase built with Melange:

Recent version of the OCaml compiler: at some point, we could pin compiler version between backend and frontend teams, making upgrades more straightforward as they would happen atomically.
Shared editor tooling: the official OCaml vscode extension works great with Melange, as well as any other OCaml editor integration. Having backend and frontend teams use similar editor setup removes a lot of maintenance work for us.
Consuming ppxs from source: Melange allows to consume ppxs from source, which also removes issues with pre-compiled ppxs (like this issue with the recent M1 Macs).
Melange allows to run all ppxs from a single executable file, which has some nice performance benefits.
Use Dune for atd files generators: ReScript “generators” are unfortunately not documented anymore, but we use them extensively for atd file generation. Being able to share Dune rules in backend and frontend would make our build setup easier.
Access to OCaml documentation tooling: Melange allows to leverage existing tooling for generating documentation, like odoc.
Async syntax: the latest Reason version supports “let op” syntax, which is handy for client-side code.

The bad

While there are many things that are exciting about Melange, there are some other parts that can be improved.

Build performance: We already knew that performance would be far worse than ReScript, as Melange uses Dune in a way that it was not designed for. In our tests, builds with Melange are roughly 1 order of magnitude slower than ReScript ones.
First-class Dune support: if there was a deeper integration between Dune and Melange, we could explore features like shared libraries or shared rules between backend and frontend. As of today, Dune has no knowledge about Melange environment, so it can perform basic rules execution, but there is no access to high level stanzas like library in Melange.
Two-headed goal: finally, we see a more strategic risk in Melange proposition. Right now it has two goals: keep compatibility with both ReScript and OCaml. But we don’t know how long these goals will be feasible. If at some point ReScript decides to move away from the OCaml compiler fully, then Melange users would not be able to consume any updates to the ReScript ecosystem anymore.

Alright, but are you migrating to Melange or ReScript?

In the meantime, we will keep exploring how far the limitations of Melange can be mitigated. To be continued! 🚀

Thanks to Igor and Feihong for reviewing and improving earlier versions of this post.

Building Ahrefs codebase with Melange was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

One and a half years of ReasonML in production

ahrefs — 2020-07-26T15:19:31-00:00

Photo by https://unsplash.com/@willianjusten

We gave them the following questions as starting points (but they were free to talk about anything they wanted):

How does Reason compare to other languages you’ve used in the past?
What’s your favorite thing about Reason?
What’s your least favorite thing about Reason?
How does ReasonReact compare to other frameworks you’ve used?
Was it easy to pick up Reason? Why or why not?

Javi

How does Reason compare to other languages you’ve used in the past?

What’s your favorite thing about Reason?

Can I pick two things? It’s hard to choose only one.

What’s your least favorite thing about Reason?

How does ReasonReact compare to other frameworks you’ve used?

Was it easy to pick up Reason? Why or why not?

Ze

I really like working with Reason, and have wanted to do so for a while. I was quite happy to see that working with it matched my expectations.

The language has such a strong type system that you feel much more comfortable with your coding.

The editors integrations with the type system are quite good and help a lot to write code better and faster.

Also, compilation times are super fast.

Liubomyr

Egor

I switched to Reason when I joined Ahrefs team about a year ago, before that I worked mostly with Ruby language.

Seif

Bryan

Feihong

Summary

Thanks to Raman and Louis for fact checking this post.

One and a half years of ReasonML in production was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

How to write a library for BuckleScript and Native

ahrefs — 2019-10-22T10:09:09-00:00

Written with Javier Chávarri and Feihong Hsu.

The goal

The build systems

For BuckleScript, there is only one build system: bsb. It is driven by a bsconfig.json file. And is installed as part of the bs-platform.

These two tools are working in a way which is pretty similar. They share a lot of concepts. And it is easy to set them up so that both are working in the same codebase.

The main similarities that interest us are:

The ability to work on specific source directories
Namespacing in bsb and wrapping in dune are both putting all the
files of the library under a single module name

The source code file tree

The code of the library is split into 3 directories.

├── js/
├── native/
└── shared/

shared is meant to host most of the code and all the code in this directory will be compiled in both modes.
js contains the parts that are specific to BuckleScript.
native contains the parts that are specific to native OCaml.

Set up the build systems

├── bsconfig.json
├── dune
├── dune-project
├── js/
├── native/
└── shared/

BuckleScript

At the root of the library we need a bsconfig.json file to drive
bsb. The documentation is available at https://bucklescript.github.io/docs/en/build-configuration.

  "namespace": true,
  "sources": [
    {
      "dir": "js",
      "subdirs": true
    }, {
      "dir": "shared",
      "subdirs": true
    }
  ],

The rest of the file is as usual.

{
  "name": "sharedlib",
  "namespace": true,
  "sources": [
    {
      "dir": "js",
      "subdirs": true
    }, {
      "dir": "shared",
      "subdirs": true
    }
  ],
  "package-specs": {
    "module": "es6",
    "in-source": true
  },
  "refmt": 3,
  "suffix": ".bs.js",
  "generate-merlin": true,
}

Dune

(dirs shared native)

(include_subdirs unqualified)

The final dune file looks like this:

(dirs shared native)
 (include_subdirs unqualified)
 (library
  (name sharedlib))

We also want a basic dune-project. If we don’t write it by hand, dune will generate it for us. I am using version 1.10 as an example. But it can be changed to whatever version suits your project.

(lang dune 1.10)

Compilation

With the setup described above, the compilation for BuckleScript and native is the same as in a setup with only one or the other.

bsb -make-world for BuckleScript
dune build @all for dune

The call to bsb is usally put in package.json in the scripts part, so that the usual yarn build can be used. For native, it depends if you rely on esy or opam.

How to consume the library

This is exactly the same setup that would be used in a pure BuckleScript or pure native library.

To use your library in BuckleScript:

Add the name and version to package.json
Add the name to bsbconfig.json of consuming library/app

To use your library in native OCaml, add the name of your library to the libraries part an executable or library stanza, e.g.

(executable
 (name main)
 (libraries sharedlib))

Module naming

Platform specific code

The whole library does not have to be exactly the same in the two platform. It is possible to add modules that are available only in one mode. Or to have modules with a different interface.

For example, by adding a file Foo.re in js but not in native, the library now has a module Foo available when compiled to javascript. But only when compiled to javascript.

Downsides

Both bsb and dune generate .merlin files when they compile our library. They override each other. It might be troublesome if the version of ocaml used for native code is not 4.02.3. Simply recompile the library for your platform to solve the problem.
Out of the box, this approach doesn’t really allow us to share interface files between both platforms: native and BuckleScript. One workaround for that, if we wanted to share some module Foo, is to:
1. add Foo.mli or Foo.rei file in shared
2. add include FooImplementation in Foo.ml
3. add FooImplementation in both native and js folder
It’s not possible to be platform specific for just a few lines of code (e.g. if IS_NATIVE foo else bar), the minimal per-platform unit is a file/module.

Example project

We have set up a simple library to showcase what a repository looks like once the whole configuration is in place. It is available on github.

How to write a library for BuckleScript and Native was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Getting started with atdgen and bucklescript

ahrefs — 2018-09-12T02:53:58-00:00

Installation

Then we need to initialize opam and create a switch. Any version of ocaml greater or equal to 4.03.0 should be fine.

opam init -a
opam switch create . 4.07.1 -y

Once it is done, we have to install the development version of atdgen. The support of bucklescript is not officially released.

opam pin add atd --dev-repo   
opam pin add atdgen --dev-repo

Make sure that atdgen is available.

$ which atdgen                 
(current $PWD)/_opam/bin/atdgen

Of course, we need bucklescript.

yarn init                 
yarn add bs-platform --dev

It is published on npm for easy integration in bucklescript projects.

yarn add @ahrefs/bs-atdgen-codec-runtime

Project configuration

After the previous section, package.json should be almost ready. We can add a few scripts to make it more convenient to compile the project. Here is how it should look once completed.

{
  "name": "demo-bs-atdgen",
  "version": "0.0.1",
  "description": "demo of atdgen with bucklescript",
  "scripts": {
    "clean": "bsb -clean-world",
    "build": "bsb -make-world",
    "watch": "bsb -make-world -w",
    "atdgen": "atdgen -t meetup.atd && atdgen -bs meetup.atd"
  },
  "devDependencies": {
    "bs-platform": "^4.0.5"
  },
  "peerDependencies": {
    "bs-platform": "^4.0.5"
  },
  "dependencies": {
    "@ahrefs/bs-atdgen-codec-runtime": "^1.0.4"
  }
}

The bucklescript configuration is very simple. We use the basic configuration that can be found in any bucklescript project. Except that we need to add one dependency to bsconfig.json:

{
  "name": "demo-bs-atdgen",
  "version": "0.0.1",
  "sources": {
    "dir": "src",
    "subdirs": true
  },
  "package-specs": {
    "module": "commonjs",
    "in-source": true
  },
  "suffix": ".bs.js",
  "bs-dependencies": [
    "@ahrefs/bs-atdgen-codec-runtime"
  ],
  "warnings": {
    "error": "+101"
  },
  "generate-merlin": true,
  "namespace": true,
  "refmt": 3
}

First ATD definitions

It is time to create a first .atd file, containing our types. This part is also documented on https://atd.readthedocs.io/en/latest/tutorial.html#getting-started

For this example, I decided to go with a meetup event. Put the type definitions in src/meetup.atd.

(* This is a comment. Same syntax as in ocaml. *)

type access = [ Private | Public ]

(* the date will be a float in the json and a Js.Date.t in ocaml *)
type date = float wrap <ocaml module="Js.Date" wrap="Js.Date.fromFloat" unwrap="Js.Date.valueOf">

(* Some people don't want to provide a phone number, make it optional *)
type person = {
  name: string;
  email: string;
  ?phone: string nullable;
}

type event = {
  access: access;
  name: string;
  host: person;
  date: date;
  guests: person list;
}

type events = event list

We use the atdgen binary (compiled previously) to generate the ocaml types and the code to serialize/deserialize those types.

atdgen -t meetup.atd # generates an ocaml file containing the types
atdgen -bs meetup.atd # generates the code to (de)serialize

The generated files are:

meetup_t.ml(i) which contain the ocaml types corresponding to our ATD definitions.
meetup_bs.ml(i) which contain the ocaml code to transform from and to json values.

At this point we can compile our project.

yarn build

If everything worked properly, we now have two .bs.js files in the src directory.

$ tree src
src
├── meetup.atd
├── meetup_bs.bs.js
├── meetup_bs.ml
├── meetup_bs.mli
├── meetup_t.bs.js
├── meetup_t.ml
└── meetup_t.mli

0 directories, 7 files

At this point, we can create new OCaml/Reason files in the src directory and use all the code atdgen generated for us. Two examples to illustrate that.

Query a REST API

A common usage of atdgen is to decode the JSON returned by a REST API. Here is a short example, using the reason syntax and bs-fetch.

let get = (url, decode) =>
  Js.Promise.(
    Fetch.fetchWithInit(
      url,
      Fetch.RequestInit.make(~method_=Get, ()),
    )
    |> then_(Fetch.Response.json)
    |> then_(json => json |> decode |> resolve)
  );

let v: Meetup_t.events =
  get(
    "http://localhost:8000/events",
    Atdgen_codec_runtime.Decode.decode(Meetup_bs.read_events),
  );

Read and write a JSON file

Atdgen for bucklescript doesn’t take care of converting a string to a JSON object. Which allows us to use the performant json parser included in nodejs or the browser.

let read_events filename =
  (* Read and parse the json file from disk, this doesn't involve atdgen. *)
  let json =
    Node_fs.readFileAsUtf8Sync filename
    |> Js.Json.parseExn
  in
  (* Turn it into a proper record. The annotation is of course optional. *)
  let events: Meetup_t.events =
    Atdgen_codec_runtime.Decode.decode Meetup_bs.read_events json
  in
  events

The reverse operation, converting a record to a JSON object and writing it in a file is also straightforward.

let write_events filename events =
  Atdgen_codec_runtime.Encode.encode Meetup_bs.write_events events (* turn a list of records into json *)
  |. Js.Json.stringifyWithSpace 2   (* convert the json to a pretty string *)
  |> Node_fs.writeFileAsUtf8Sync filename  (* write the json in our file *)

Full example

Now that we have our functions to read and write events, we can build a small cli to pretty print the list of events and add new events.

The source code of the full example is available on github.

You can run it like this:

$ echo "[]" > events.json
$ nodejs src/cli.bs.js add louis louis@nospam.com
$ nodejs src/cli.bs.js add bob bob@nospam.com
$ nodejs src/cli.bs.js print
=== OCaml/Reason Meetup! summary ===
date: Tue, 11 Sep 2018 15:04:16 GMT
access: public
host: bob <bob@nospam.com>
guests: 1
=== OCaml/Reason Meetup! summary ===
date: Tue, 11 Sep 2018 15:04:13 GMT
access: public
host: louis <louis@nospam.com>
guests: 1
$ cat events.json
[
  {
    "guests": [
      {
        "email": "bob@nospam.com",
        "name": "bob"
      }
    ],
    "date": 1536678256177,
    "host": {
      "email": "bob@nospam.com",
      "name": "bob"
    },
    "name": "OCaml/Reason Meetup!",
    "access": "Public"
  },
  {
    "guests": [
      {
        "email": "louis@nospam.com",
        "name": "louis"
      }
    ],
    "date": 1536678253790,
    "host": {
      "email": "louis@nospam.com",
      "name": "louis"
    },
    "name": "OCaml/Reason Meetup!",
    "access": "Public"
  }
]

Getting started with atdgen and bucklescript was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Skylake bug: a detective story

ahrefs — 2017-06-28T18:34:51-00:00

It was a dark and stormy night; the skylake CPU buzzed with excitement, and then, suddenly, the hyperthreads started to lock up..

Or something like that.

This week a new erratum for the Intel Skylake and Kabylake processors families was brought to public attention on the Debian mailing list, and then on various social media and news outlets.

Setting the scene

Tracking down crashes in OCaml

The usual debugging process for crashing OCaml code didn’t prove effective — we needed to narrow down our approach.

A set of strange leads

Escalating the issue to Mantis made us to take a step back and gather our findings, and we quickly got great feedback from the OCaml core team.

At this point, what does the problem look like?

[22985.879907] ocamlopt.opt[48221]: segfault at af8 ip 00005564455169bd sp 00007ffc9f36b130 error 4 in ocamlopt.opt[556445006000+613000]

The hardware, an unusual suspect

In the meantime several core OCaml developers had been closely investigating the issue and started auditing recent changes in the runtime, and identified a few suspicious changes and known bugs.

Is it crazy to imagine that gcc -O2 on the OCaml 4.03 runtime produces a specific instruction sequence that causes hardware issues in (some steppings of) Skylake processors with hyperthreading? Perhaps it is crazy.

This possibility had struck us too, motivated by the HyperThreading, the page table corruption and the Skylake specific set of conditions.

Identifying the offending code

If we could isolate the problematic C code, comparing the generated code would be easier. The problem had the form of a well known nail:

Around 50 C files composing the OCaml runtime,
There is a good state (when built with gcc -O1)
And a bad state (when built with gcc -O2)

This nail comes with a precious hammer: bisection.

The bulk of this function is a switch taking action for each block depending on its status :

https://medium.com/media/cfbc19fddccc5f2c1a76fcc802fae049/href

Getting back to the assembly diff.

Examining the bounds and invariants derived by gcc, it was clear that no wrong hypothesis was stated.

https://medium.com/media/a5a311d0b5a4f890f9541f0aed91e73e/href

There were only cosmetic differences. The test removal was propagated down to assembly and caused gcc to reorganise the layout of each switch case block.

https://medium.com/media/532ab4896b11177a96203fd0c81eaa58/href

https://medium.com/media/38ca30a1decf86233084721c3803b1c4/href

For some reason, the block pointer was spilled to the stack.

Mitigating the issue

We were now quite certain it was a CPU bug.

Even if we had no confirmation at this point nor any explanations of the cause of this bug, which was beyond our reach, we could take actions.

We patched our internal opam repository to distribute unoptimised runtime to all developers and moved forward, waiting for further announcements.

Happy end

In late May, devops team noticed a debian package update for intel-microcode containing the following change:

Likely fix nightmare-level Skylake erratum SKL150. Fortunately,
either this erratum is very-low-hitting, or gcc/clang/icc/msvc
won’t usually issue the affected opcode pattern and it ends up
being rare.
SKL150 — Short loops using both the AH/BH/CH/DH registers and
the corresponding wide register *may* result in unpredictable
system behavior. Requires both logical processors of the same
core (i.e. sibling hyperthreads) to be active to trigger, as
well as a “complex set of micro-architectural conditions”

The erratum description immediately rang a bell as it matched the diff in the assembly we had observed. We tested the microcode update and confirmed it fixed the corruption.

Finally, our Skylake CPUs were feeling safe and OCaml compiler was happy.

Skylake bug: a detective story was originally published in Ahrefs on Medium, where people are continuing the conversation by highlighting and responding to this story.

ambassadortothecomputers

ambassadortothecomputers — 2023-07-14T05:47:25-00:00

In this post I want to build on the backtracking logic monad we covered last time by adding unification, yielding an embedded DSL for Prolog-style logic programming.

+http://ambassadortothecomputers.blogspot.com/feeds/posts/default?alt=rssambassadortothecomputers2023-07-14T05:47:25-00:00http://ambassadortothecomputers.blogspot.com/feeds/posts/default?alt=rssambassadortothecomputers

In this post I want to build on the backtracking logic monad we covered last time by adding unification, yielding an embedded DSL for Prolog-style logic programming.

Prolog

Here is a small Prolog example, the rough equivalent of List.contains in Scala:

  member(X, [X|T]). 
diff --git a/data/planet/andrej.xml b/data/planet/andrej.xml
index fcbd2411ec..59a8b8c95d 100644
--- a/data/planet/andrej.xml
+++ b/data/planet/andrej.xml
@@ -1,5 +1,5 @@
 
-http://math.andrej.com/feed.xmlandrej2023-07-13T05:47:27-00:00http://math.andrej.com/feed.xmlandrejI am at the Machine assisted proofs workshop at the UCLA Institute for Pure and Applied Mathematics, where I am about to give a talk on “Formalizing invisible mathematics”.
+http://math.andrej.com/feed.xmlandrej2023-07-14T05:47:24-00:00http://math.andrej.com/feed.xmlandrejI am at the Machine assisted proofs workshop at the UCLA Institute for Pure and Applied Mathematics, where I am about to give a talk on “Formalizing invisible mathematics”.
 
 Here are the slides with speaker notes and the video recording of the talk.
 
diff --git a/data/planet/ashishagarwal.xml b/data/planet/ashishagarwal.xml
index d8301fbf76..adb8f11c8b 100644
--- a/data/planet/ashishagarwal.xml
+++ b/data/planet/ashishagarwal.xml
@@ -1,5 +1,5 @@
 
-http://ashishagarwal.org/tag/ocaml/feed/ashishagarwal2023-07-13T05:47:27-00:00http://ashishagarwal.org/tag/ocaml/feed/ashishagarwalAbstract

+http://ashishagarwal.org/tag/ocaml/feed/ashishagarwal2023-07-14T05:47:24-00:00http://ashishagarwal.org/tag/ocaml/feed/ashishagarwal
Abstract

 High-throughput genomic sequencing is characterized by large diverse datasets and numerous analysis methods. It is normal for an individual bioinformatician to work with thousands of data files and employ hundreds of distinct computations during the course of a single project. This problem is magnified in “core facilities”, which support multiple researchers working on diverse projects. Most investigators use ad hoc methods to manage this complexity with dire consequences: analyses frequently fail to meet the scientific mandate of reproducibility; improved analysis methods are often not considered because recomputing all downstream steps would be overwhelming; hard drives and CPUs are used sub-optimally; and, in some cases, raw data is lost.
 We describe HITSCORE, an OCaml software stack that operates all computational aspects of the Genomics Core Facility at New York University’s Center for Genomics and Systems Biology. HITSCORE has been in production use for one year, and was implemented quickly by less than two programmers following design advice from several biologists. A simple domain specific language (DSL) enables generating type safe database bindings and GUI components, and greatly eases updates and migration of our data model. We found a multi-lingual stack too burdensome in a small team setting, and credit OCaml for fulfilling the needs of our full application stack. It has good database bindings, excels at encoding complex domain logic, and now allows construction of rich websites due to the Ocsigen web programming framework. Higher level libraries for distributed computing would be a welcome improvement.
 The opportunity to build this system did not stem directly from any strength of functional programming or OCaml. It was necessary for a person with credibility amongst biologists to champion its development, and this credibility took several years to build. Rapid development appears to be the single most important factor in allaying doubts about using a lesser known language, and we will briefly describe our experiences in bringing OCaml to several high profile projects.
diff --git a/data/planet/bap.xml b/data/planet/bap.xml
index 35ba898957..fe8f95c9db 100644
--- a/data/planet/bap.xml
+++ b/data/planet/bap.xml
@@ -1,5 +1,5 @@
 
-https://binaryanalysisplatform.github.io/feed.xmlbap2023-07-13T05:47:28-00:00https://binaryanalysisplatform.github.io/feed.xmlbapDefining instructions semantics using Primus Lisp (Tutorial)
+https://binaryanalysisplatform.github.io/feed.xmlbap2023-07-14T05:47:25-00:00https://binaryanalysisplatform.github.io/feed.xmlbapDefining instructions semantics using Primus Lisp (Tutorial)
 
 Introduction
 
diff --git a/data/planet/baturin.xml b/data/planet/baturin.xml
index 99c1f19f38..b29ff508be 100644
--- a/data/planet/baturin.xml
+++ b/data/planet/baturin.xml
@@ -1,5 +1,5 @@
 
-https://baturin.org/blog/atom-ocaml.xmlbaturin2023-07-05T05:48:06-00:00https://baturin.org/blog/atom-ocaml.xmlbaturin
+https://baturin.org/blog/atom-ocaml.xmlbaturin2023-07-14T05:47:25-00:00https://baturin.org/blog/atom-ocaml.xmlbaturin
     One thing I find odd about many interpreter projects is that they are designed as standalone and can't be used as embedded
 scripting languages. Lua-ML is completely different in that regard: it's designed as an extension language first and offers
 some unique features for that use case, including a reconfigurable runtime library. You can remove modules from its standard library or replace
diff --git a/data/planet/cameleon.xml b/data/planet/cameleon.xml
index f39d113fac..cdbfc9b4e4 100644
--- a/data/planet/cameleon.xml
+++ b/data/planet/cameleon.xml
@@ -1,2 +1,2 @@
 
-http://www.blogger.com/feeds/7617521785419311079/posts/defaultcameleon2023-07-13T05:47:27-00:00http://www.blogger.com/feeds/7617521785419311079/posts/defaultcameleonNew commands are available in the OCaml mode of Chamo. They use the .annot files generated by ocamlc. Here is a list of these commands. Default key bindings are shown between parenthesis.

ocaml_display_type_annot (A-t) already existed to display the type of the expression under the cursor. Now ocaml_copy_type_annot (CA-t) does the same but also copies the type in the clipboard so that it can be easily pasted somewhere else, for example in a .mli file. This prevents from having to launch a ocamlc -i command to get the type.

ocaml_display_ident_annot (A-i) displays the "ident" information, that is whether the identifier is local to the file or its fully qualified name if it is defined elsewhere. ocaml_jump_to_local_def (A-j) does the same but if the ident is defined in the file, the cursor is positioned on the identifier definition.


ocaml_display_call_annot (A-c) displays the "call" information, that is whether a function call is "tail" or "stack". ocaml_show_stack_calls (CA-c) highlights (or not) all the calls using the stack. This can be useful to check whether a function is tail-recursive. The screenshot below shows the result on the List module.

At last, ocaml_expand_ext_idents (CA-x) prompts the user to replace each external identifier by its fully qualified identifier (except for idents from Pervasives). I personnally use the open directive only not to prefix record fields and constructors, but always use fully qualified identifiers. This command is useful when working on a code not written by me and full of open's at the beginning of a file.

All these commands are available from revision 745 of the repository and in the nightly snapshot.



R mode in Chamo
cameleon — 2008-05-06T14:37:00-00:00
A R mode is now available for Chamo, as a separate snippet. By now, this mode has only one key binding, to call the "r_eval" command. This command sends the contents of the active view to a background R process. The output of this process is displayed in the "outputs" window. Other commands can easily be defined the same way to interact with other programs (see the details).

If some text is selected in the view, then only this text is sent to the R process. Closing the R output tab in the "outputs" window closes the R process. When the "r_eval" command is executed again, a new process is created. If the command is executed and the R process is still running, then it is used and no other process is launched.




OCamlbuild support in Chamo
cameleon — 2008-04-30T14:33:00-00:00
The Chamo editor now includes some ocamlbuild support through the internal command "ocaml_build": An ocamlbuild command is proposed to the user who can edit it and run it.

















The output of the command is displayed in a new "outputs" window, and is analyzed. In case of error (or warning considered as error), the editor displays, in the active view, the file where the error was found, and highlights the corresponding characters.










The command used to compile is kept associated to the file, so that it is proposed the next time the internal command "ocaml_build" is launch on the file. These associations are even stored on disk to be kept between two sessions of Chamo launched in the same directory.


Playing with .annot files
cameleon — 2010-01-21T14:58:00-00:00
New commands are available in the OCaml mode of Chamo. They use the .annot files generated by ocamlc. Here is a list of these commands. Default key bindings are shown between parenthesis.

ocaml_display_type_annot (A-t) already existed to display the type of the expression under the cursor. Now ocaml_copy_type_annot (CA-t) does the same but also copies the type in the clipboard so that it can be easily pasted somewhere else, for example in a .mli file. This prevents from having to launch a ocamlc -i command to get the type.

ocaml_display_ident_annot (A-i) displays the "ident" information, that is whether the identifier is local to the file or its fully qualified name if it is defined elsewhere. ocaml_jump_to_local_def (A-j) does the same but if the ident is defined in the file, the cursor is positioned on the identifier definition.


ocaml_display_call_annot (A-c) displays the "call" information, that is whether a function call is "tail" or "stack". ocaml_show_stack_calls (CA-c) highlights (or not) all the calls using the stack. This can be useful to check whether a function is tail-recursive. The screenshot below shows the result on the List module.

At last, ocaml_expand_ext_idents (CA-x) prompts the user to replace each external identifier by its fully qualified identifier (except for idents from Pervasives). I personnally use the open directive only not to prefix record fields and constructors, but always use fully qualified identifiers. This command is useful when working on a code not written by me and full of open's at the beginning of a file.

All these commands are available from revision 745 of the repository and in the nightly snapshot.


R mode in Chamo
cameleon — 2008-05-06T14:37:00-00:00
A R mode is now available for Chamo, as a separate snippet. By now, this mode has only one key binding, to call the "r_eval" command. This command sends the contents of the active view to a background R process. The output of this process is displayed in the "outputs" window. Other commands can easily be defined the same way to interact with other programs (see the details).

If some text is selected in the view, then only this text is sent to the R process. Closing the R output tab in the "outputs" window closes the R process. When the "r_eval" command is executed again, a new process is created. If the command is executed and the R process is still running, then it is used and no other process is launched.




OCamlbuild support in Chamo
cameleon — 2008-04-30T14:33:00-00:00
The Chamo editor now includes some ocamlbuild support through the internal command "ocaml_build": An ocamlbuild command is proposed to the user who can edit it and run it.

















The output of the command is displayed in a new "outputs" window, and is analyzed. In case of error (or warning considered as error), the editor displays, in the active view, the file where the error was found, and highlights the corresponding characters.










The command used to compile is kept associated to the file, so that it is proposed the next time the internal command "ocaml_build" is launch on the file. These associations are even stored on disk to be kept between two sessions of Chamo launched in the same directory.


cburnout
cburnout — 2023-07-14T05:47:24-00:00

+http://www.blogger.com/feeds/9108979482982930820/posts/default/-/OCamlcburnout2023-07-14T05:47:24-00:00http://www.blogger.com/feeds/9108979482982930820/posts/default/-/OCamlcburnout
 

 A journey toward a state-machine without mutable state, but enough flexibility to express GUI behaviors. This is nothing new in it's parts, but maybe as a whole. In scouring GUI lore there is very little in a functional style.

 

diff --git a/data/planet/coherentpdf.xml b/data/planet/coherentpdf.xml
index fcbe736eef..ba4feaa93c 100644
--- a/data/planet/coherentpdf.xml
+++ b/data/planet/coherentpdf.xml
@@ -1,5 +1,5 @@
 
-https://coherentpdf.com/blog/?tag=ocaml&feed=rss2coherentpdf2023-07-13T05:47:27-00:00https://coherentpdf.com/blog/?tag=ocaml&feed=rss2coherentpdfThe cpdf tool has been available commercially since 2007, and is widely used in industry and government. The same technology is used in the Proview PDF Editor for the mac.
+https://coherentpdf.com/blog/?tag=ocaml&feed=rss2coherentpdf2023-07-14T05:47:24-00:00https://coherentpdf.com/blog/?tag=ocaml&feed=rss2coherentpdfThe cpdf tool has been available commercially since 2007, and is widely used in industry and government. The same technology is used in the Proview PDF Editor for the mac.
 Now we’re releasing it for free, under a special not-for-commercial-use license.
 Binaries for Windows, Mac and Linux, OCaml source code, and full documentation are all available here.
 The Coherent PDF Command Line Tools allow you to manipulate existing PDF files in a variety of ways. For example:
diff --git a/data/planet/coq.xml b/data/planet/coq.xml
index 705360fbfe..97e19eaa2a 100644
--- a/data/planet/coq.xml
+++ b/data/planet/coq.xml
@@ -1,5 +1,5 @@
 
-https://coq.inria.fr/rss.xmlcoq2023-07-13T05:47:27-00:00https://coq.inria.fr/rss.xmlcoq
+https://coq.inria.fr/rss.xmlcoq2023-07-14T05:47:24-00:00https://coq.inria.fr/rss.xmlcoq
 
 
 The Coq development team is proud to announce the immediate availability of
diff --git a/data/planet/cufp.xml b/data/planet/cufp.xml
index 9c5b4e7f49..39c8d7b12d 100644
--- a/data/planet/cufp.xml
+++ b/data/planet/cufp.xml
@@ -1,5 +1,5 @@
 
-http://cufp.org/blog/all.rss.xmlcufp2023-07-13T05:47:27-00:00http://cufp.org/blog/all.rss.xmlcufp
+http://cufp.org/blog/all.rss.xmlcufp2023-07-14T05:47:24-00:00http://cufp.org/blog/all.rss.xmlcufp
 
 
CUFP 2015 will be held in Vancouver, Canada. It is co-located with ICFP 2015.
 The call for presentations is available here. Submissions are due by June 14.
diff --git a/data/planet/cwn.xml b/data/planet/cwn.xml
index ab80e519c8..7d5fb8631d 100644
--- a/data/planet/cwn.xml
+++ b/data/planet/cwn.xml
@@ -1,2 +1,2 @@
 
-https://alan.petitepomme.net/cwn/cwn.rsscwn2023-07-13T05:47:26-00:00https://alan.petitepomme.net/cwn/cwn.rsscwnopam 2.2.0~alpha release!
Initial Emissions Monitoring of the OCaml.org Infrastructure
Autofonce, a modern runner for GNU Autotests suites
First beta release of OCaml 5.1.0
Dream-html - DSL to build HTML, integrated with Dream
ppx_subliner 0.2.0
happy-eyeballs, dns, http-{lwt,mirage}-client ++ writeup of happy-eyeballs issue and fix
Getting started with ReasonReact and Melange
Making OCaml 5 Succeed for Developers and Organisations
Llama, an OCaml library for declaratively building software-defined modular synthesizers
dune 3.9.0
Depending on non-OCaml languages from the opam repository
Status of DkML Windows


OCaml Weekly News, 04 Jul 2023
cwn — 2023-07-04T12:00:00-00:00
New release of CAISAR, a platform for characterizing AI safety and robustness
stdlib-random release 1.1.0
dune 3.8.0
plotly release 1.0.0, a wrapper for Plotly open source graphing library
Help us Make the New Learn Area on OCaml.org Awesome
Sedlex 3.2
dune 3.9.0
Other OCaml News


OCaml Weekly News, 27 Jun 2023
cwn — 2023-06-27T12:00:00-00:00
dune 3.8.0
moonpool 0.2
aeroon, wrapper to Aeron
Representing OCaml on Lemmy
awa-ssh release 0.3
Ppxlib dev meetings
User-friendly continuations over OCaml 5 effects?
Raising the minimum tested version of OCaml to 4.05 or 4.11 (from 4.02)
guroobi, wrapper to Gurobi
euler, an arithmetic library for native integers
CFP - JFLA 2024 - Journées Francophones des Langages Applicatifs
OCaml.org Newsletter: May 2023
Other OCaml News


OCaml Weekly News, 20 Jun 2023
cwn — 2023-06-20T12:00:00-00:00
Packaging an executable for homebrew install for non-OCaml users
v0.16 release of Jane Street packages
New release of Menhir (20230608)
Dream-html - DSL to build HTML, integrated with Dream
release of minisat 0.5
Which filters would you like to see on the OCaml blog at OCaml.org?
OCaml Platform Newsletter, May 2023
Congratulation to the OCaml team for the 2023 SIGPLAN programming languages software award! 🏆
Other OCaml News


OCaml Weekly News, 13 Jun 2023
cwn — 2023-06-13T12:00:00-00:00
release of minisat 0.5
containers 3.12
Recent tar releases (2.5.0, 2.4.0, ...)
mpp.0.3.6, which adds compatibility with OCaml 5, is now in opam
trace 0.1, tracy-client 0.1
A short review of fmlib_browser (web frontend library)
moonpool 0.1
Other OCaml News


OCaml Weekly News, 06 Jun 2023
cwn — 2023-06-06T12:00:00-00:00
Meetup group in New York City
Creating a tutorial on sequences
new release: Merlin 4.9
Taking Inventory of the OCaml Ecosystem on OCaml.org
New release: DocuLib 1.3.5
opam 2.1.5 release!
dune 3.8.0
Second alpha release of OCaml 5.1.0
ML'23: ACM SIGPLAN ML Family Workshop — Call for presentations
qcheck-lin and qcheck-stm 0.2
Melange 1.0 – compile OCaml / ReasonML to JavaScript
Debugging Native Code in "Second OCaml" YouTube Video
Sandmark nightly now supports latency profiling
Update on Eio (effects-based direct-style IO for OCaml 5)
Initial Emissions Monitoring of the OCaml.org Infrastructure
Other OCaml News


OCaml Weekly News, 30 May 2023
cwn — 2023-05-30T12:00:00-00:00
ML'23: ACM SIGPLAN ML Family Workshop — Call for presentations
OCaml Workshop 2023 -- presentation deadline on June 1st, online attendance option
A Roadmap for the OCaml Platform - Seeking Your Feedback
new release: tiny_httpd 0.13


OCaml Weekly News, 23 May 2023
cwn — 2023-05-23T12:00:00-00:00
Looking for example tsdl games
OCaml.org Newsletter: April 2023
A bestiary of GADT examples?
2023 StackOverflow Developer Survey
Release 0.5.4 of ~Fmlib_browser~
OCaml Platform Newsletter: April 2023
Eio Developer Meetings
New major release of Parany (v14.0.0)


OCaml Weekly News, 16 May 2023
cwn — 2023-05-16T12:00:00-00:00
Rendering React in OCaml
Ahrefs is Hiring
tmx: Import 2D game maps with ease
Brr 0.0.5, the WebGPU edition
A bestiary of GADT examples?
Open-source tool to make a static blog in OCaml?
BER MetaOCaml N114, for OCaml 4.14.1
Building iOS apps with OCaml?
A Minimal Prototype of In-Package Search is on staging.ocaml.org
New release of Fix (20230505)
QCheck 0.21


OCaml Weekly News, 09 May 2023
cwn — 2023-05-09T12:00:00-00:00
Overview of libraries for showing OCaml values
kcas and kcas_data 0.3.0: Software Transactional Memory
OCaml.org Newsletter: March 2023
Creating a tutorial on sequences
You started to learn OCaml less than 12 months ago? Please help us with our user survey on the OCaml.org Learning Area
Explorations on Package Management in Dune
Functional web applications running in the browser
Ahrefs is now built with Melange
Other OCaml News


OCaml Weekly News, 11 Jul 2023
cwn — 2023-07-11T12:00:00-00:00
opam 2.2.0~alpha release!
Initial Emissions Monitoring of the OCaml.org Infrastructure
Autofonce, a modern runner for GNU Autotests suites
First beta release of OCaml 5.1.0
Dream-html - DSL to build HTML, integrated with Dream
ppx_subliner 0.2.0
happy-eyeballs, dns, http-{lwt,mirage}-client ++ writeup of happy-eyeballs issue and fix
Getting started with ReasonReact and Melange
Making OCaml 5 Succeed for Developers and Organisations
Llama, an OCaml library for declaratively building software-defined modular synthesizers
dune 3.9.0
Depending on non-OCaml languages from the opam repository
Status of DkML Windows


OCaml Weekly News, 04 Jul 2023
cwn — 2023-07-04T12:00:00-00:00
New release of CAISAR, a platform for characterizing AI safety and robustness
stdlib-random release 1.1.0
dune 3.8.0
plotly release 1.0.0, a wrapper for Plotly open source graphing library
Help us Make the New Learn Area on OCaml.org Awesome
Sedlex 3.2
dune 3.9.0
Other OCaml News


OCaml Weekly News, 27 Jun 2023
cwn — 2023-06-27T12:00:00-00:00
dune 3.8.0
moonpool 0.2
aeroon, wrapper to Aeron
Representing OCaml on Lemmy
awa-ssh release 0.3
Ppxlib dev meetings
User-friendly continuations over OCaml 5 effects?
Raising the minimum tested version of OCaml to 4.05 or 4.11 (from 4.02)
guroobi, wrapper to Gurobi
euler, an arithmetic library for native integers
CFP - JFLA 2024 - Journées Francophones des Langages Applicatifs
OCaml.org Newsletter: May 2023
Other OCaml News


OCaml Weekly News, 20 Jun 2023
cwn — 2023-06-20T12:00:00-00:00
Packaging an executable for homebrew install for non-OCaml users
v0.16 release of Jane Street packages
New release of Menhir (20230608)
Dream-html - DSL to build HTML, integrated with Dream
release of minisat 0.5
Which filters would you like to see on the OCaml blog at OCaml.org?
OCaml Platform Newsletter, May 2023
Congratulation to the OCaml team for the 2023 SIGPLAN programming languages software award! 🏆
Other OCaml News


OCaml Weekly News, 13 Jun 2023
cwn — 2023-06-13T12:00:00-00:00
release of minisat 0.5
containers 3.12
Recent tar releases (2.5.0, 2.4.0, ...)
mpp.0.3.6, which adds compatibility with OCaml 5, is now in opam
trace 0.1, tracy-client 0.1
A short review of fmlib_browser (web frontend library)
moonpool 0.1
Other OCaml News


OCaml Weekly News, 06 Jun 2023
cwn — 2023-06-06T12:00:00-00:00
Meetup group in New York City
Creating a tutorial on sequences
new release: Merlin 4.9
Taking Inventory of the OCaml Ecosystem on OCaml.org
New release: DocuLib 1.3.5
opam 2.1.5 release!
dune 3.8.0
Second alpha release of OCaml 5.1.0
ML'23: ACM SIGPLAN ML Family Workshop — Call for presentations
qcheck-lin and qcheck-stm 0.2
Melange 1.0 – compile OCaml / ReasonML to JavaScript
Debugging Native Code in "Second OCaml" YouTube Video
Sandmark nightly now supports latency profiling
Update on Eio (effects-based direct-style IO for OCaml 5)
Initial Emissions Monitoring of the OCaml.org Infrastructure
Other OCaml News


OCaml Weekly News, 30 May 2023
cwn — 2023-05-30T12:00:00-00:00
ML'23: ACM SIGPLAN ML Family Workshop — Call for presentations
OCaml Workshop 2023 -- presentation deadline on June 1st, online attendance option
A Roadmap for the OCaml Platform - Seeking Your Feedback
new release: tiny_httpd 0.13


OCaml Weekly News, 23 May 2023
cwn — 2023-05-23T12:00:00-00:00
Looking for example tsdl games
OCaml.org Newsletter: April 2023
A bestiary of GADT examples?
2023 StackOverflow Developer Survey
Release 0.5.4 of ~Fmlib_browser~
OCaml Platform Newsletter: April 2023
Eio Developer Meetings
New major release of Parany (v14.0.0)


OCaml Weekly News, 16 May 2023
cwn — 2023-05-16T12:00:00-00:00
Rendering React in OCaml
Ahrefs is Hiring
tmx: Import 2D game maps with ease
Brr 0.0.5, the WebGPU edition
A bestiary of GADT examples?
Open-source tool to make a static blog in OCaml?
BER MetaOCaml N114, for OCaml 4.14.1
Building iOS apps with OCaml?
A Minimal Prototype of In-Package Search is on staging.ocaml.org
New release of Fix (20230505)
QCheck 0.21


OCaml Weekly News, 09 May 2023
cwn — 2023-05-09T12:00:00-00:00
Overview of libraries for showing OCaml values
kcas and kcas_data 0.3.0: Software Transactional Memory
OCaml.org Newsletter: March 2023
Creating a tutorial on sequences
You started to learn OCaml less than 12 months ago? Please help us with our user survey on the OCaml.org Learning Area
Explorations on Package Management in Dune
Functional web applications running in the browser
Ahrefs is now built with Melange
Other OCaml News


dbaelde
dbaelde — 2023-07-14T05:47:24-00:00
Maintenant que je travaille un peu dans le domaine de la sécurité, je pense notamment aux générateurs de nombres aléatoires. Après quelques lectures sur le sujet, je me suis demandé comment fonctionnait le module Random de Caml.

+http://www.blogger.com/feeds/17133288/posts/default/-/ocamldbaelde2023-07-14T05:47:24-00:00http://www.blogger.com/feeds/17133288/posts/default/-/ocamldbaeldeMaintenant que je travaille un peu dans le domaine de la sécurité, je pense notamment aux générateurs de nombres aléatoires. Après quelques lectures sur le sujet, je me suis demandé comment fonctionnait le module Random de Caml.

 

 Il y a sûrement beaucoup de choses à dire, mais mon attention a été retenue par un truc simple: self_init initialise le générateur de façon très simple, à partir de l'heure et de l'identifiant du processus, données pour le moins aléatoires. Il faut donc vraiment se méfier de cela si on souhaite utiliser le module Random à des fins cryptographiques. En pratique, le petit bout de code suivant devine assez rapidement la graine générée, en supposant connu l'identifiant du processus (ce n'est pas bien dur à deviner sinon) et en supposant approximativement connue l'heure de l'initialisation:

 

diff --git a/data/planet/drup.xml b/data/planet/drup.xml
index b81aa7fd69..1a5c28a583 100644
--- a/data/planet/drup.xml
+++ b/data/planet/drup.xml
@@ -1,5 +1,5 @@
 
-http://drup.github.io/feed-ocaml.xmldrup2023-07-13T05:47:28-00:00http://drup.github.io/feed-ocaml.xmldrup
+http://drup.github.io/feed-ocaml.xmldrup2023-07-14T05:47:25-00:00http://drup.github.io/feed-ocaml.xmldrup
         
         
         
diff --git a/data/planet/dutherenverseauborddelatable.xml b/data/planet/dutherenverseauborddelatable.xml
index 86f6cc98d9..f497f74b35 100644
--- a/data/planet/dutherenverseauborddelatable.xml
+++ b/data/planet/dutherenverseauborddelatable.xml
@@ -1,5 +1,5 @@
 
-https://dutherenverseauborddelatable.wordpress.com/category/ocaml/feed/dutherenverseauborddelatable2023-07-13T05:47:27-00:00https://dutherenverseauborddelatable.wordpress.com/category/ocaml/feed/dutherenverseauborddelatabletl;dr
+https://dutherenverseauborddelatable.wordpress.com/category/ocaml/feed/dutherenverseauborddelatable2023-07-14T05:47:24-00:00https://dutherenverseauborddelatable.wordpress.com/category/ocaml/feed/dutherenverseauborddelatabletl;dr
 JavaScript is a dynamic language. However, by borrowing a few pages from static languages – and a few existing tools – we can considerable improve reliability and maintainability.
 « Writing one million lines of code of JavaScript is simply impossible »
 (source: speaker in a recent open-source conference)
diff --git a/data/planet/emelle-tv.xml b/data/planet/emelle-tv.xml
index 818118acb5..83fc562c94 100644
--- a/data/planet/emelle-tv.xml
+++ b/data/planet/emelle-tv.xml
@@ -1,5 +1,5 @@
 
-https://watch.ocaml.org/feeds/videos.xml?videoChannelId=23emelle-tv2023-07-13T05:47:26-00:00https://watch.ocaml.org/feeds/videos.xml?videoChannelId=23emelle-tv


emelle-tv
emelle-tv — 2023-07-14T05:47:24-00:00
If you would like to sponsor  the show contact https://twitter.com/davesnx
+https://watch.ocaml.org/feeds/videos.xml?videoChannelId=23emelle-tv2023-07-14T05:47:24-00:00https://watch.ocaml.org/feeds/videos.xml?videoChannelId=23emelle-tv


Talking with António Monteiro about Melange,  Esy, Reason, OCaml and more
emelle-tv — 2023-03-17T21:10:48-00:00
If you would like to sponsor  the show contact https://twitter.com/davesnx
 Consider sponsoring António's work: https://github.com/sponsors/anmonteiro


emillon
emelle-tv — 2023-07-13T05:47:27-00:00
Craig Ferguson is a software developer at Tarides. Join us in this episode to talk more about OCaml, MirageOS, Irmin and much more!
 We are looking for sponsors! If you'd like to support more content for Ocaml, Reason and ReScript just send us a message at emelletvshow@gmail.com -- Watch live at https://www.twitch.tv/emelletv
diff --git a/data/planet/emillon.xml b/data/planet/emillon.xml
index 2296fb165d..728f1890d6 100644
--- a/data/planet/emillon.xml
+++ b/data/planet/emillon.xml
@@ -1,5 +1,5 @@
 
-http://blog.emillon.org/feeds/ocaml.xmlemillon2023-07-13T05:47:27-00:00http://blog.emillon.org/feeds/ocaml.xmlemillonHello! It’s 2016, it’s November, and apparently it rhymes with #NaBoMaMo 2016,
+http://blog.emillon.org/feeds/ocaml.xmlemillon2023-07-14T05:47:24-00:00http://blog.emillon.org/feeds/ocaml.xmlemillon
Hello! It’s 2016, it’s November, and apparently it rhymes with #NaBoMaMo 2016,
 the National Bot Making Month. I made a bot!.
 Full disclosure: it’s actually 2017, but I started writing this in 2016 so
 it’s OK. Also I’m not actually from the US, but I’ll relax the definition a bit
diff --git a/data/planet/erratique.xml b/data/planet/erratique.xml
index fac285ca9a..1361c0dae2 100644
--- a/data/planet/erratique.xml
+++ b/data/planet/erratique.xml
@@ -1,2 +1,2 @@
 
-https://erratique.ch/feeds/news.atomerratique2023-07-13T05:47:27-00:00https://erratique.ch/feeds/news.atomerratiqueNeologisms for the current computing era.


Qrc
erratique — 2020-10-22T23:50:55-00:00
First release of Qrc, a QR code encoder for OCaml.


Brr
erratique — 2020-10-14T08:00:55-00:00
First release of Brr, a toolkit for programming browsers in OCaml.


Down
erratique — 2019-07-23T21:15:55-00:00
First release of Down, an unintrusive OCaml toplevel experience upgrade.


Unicode 10.0.0 refresh
erratique — 2017-06-21T00:15:00-00:00
Uucd, Uucp, Uunf and Uuseg were refreshed for Unicode 10.0.0. Find the details via the software page.


10
erratique — 2017-02-27T19:15:55-00:00
Ten years ago I published a first bit of free software. Over the years, this has now expanded to 28 OCaml packages, without counting random informally published elucubrations. Thanks to OCaml Labs' support, there will be more.


Unicode 9.0.0 refresh
erratique — 2016-11-23T17:00:49-00:00
Uucd, Uucp, Uunf, Uuseg and Uutf were all refreshed for Unicode 9.0.0. They also support the new Uchar.t type introduced in OCaml 4.03 and compile with safe string. Find about them on the software page.


Asetmap, Bos, Hmap and Webbrowser
erratique — 2016-09-29T21:40:49-00:00
A few other new releases that went unannounced on this feed. Find about them on the software page.


Topkg
erratique — 2016-09-29T21:40:48-00:00
It seems the Topkg rewrite went unannounced on this feed. If you are in need of a no-nonsense release workflow to package and publish your packages on OPAM, simply topkg them.


Odig 0.0.1
erratique — 2016-09-29T20:53:49-00:00
First release of Odig. If you can't keep track of the documentation of the OCaml packages you use, just odig them locally.


Astring, Fmt, Logs, Ptime, Mtime, Rresult
erratique — 2015-12-24T22:55:00-00:00
A few new releases that went unannounced on this feed. Find about them on the software page.


Release of Uucd 3.0.0, Uucp 1.0.0, Uunf 1.0.0 and Uuseg 0.9.0
erratique — 2015-06-17T23:55:50-00:00
Release of Uucd, Uucp, Uunf and Uuseg to support Unicode 8.0.0.


On the book « More OCaml »
erratique — 2015-05-23T15:17:21-00:00
I eventually read through John Whitington's latest book « More OCaml. » It's again a very good one, I made a small review here.


Uuseg 0.8.0
erratique — 2014-12-23T21:07:47-00:00
First release of Uuseg, Unicode text segmentation for OCaml.


Cmdliner 0.9.6
erratique — 2014-11-18T14:23:09-00:00
Release of Cmdliner 0.9.6, consult the release notes for details.


Uucp 0.9.0
erratique — 2014-06-28T21:53:49-00:00
First release of Uucp, access to Unicode character properties for OCaml. Thanks to OCaml Labs for sponsoring this work.


Uucd 2.0.0 and Uunf 0.9.3
erratique — 2014-06-16T23:41:53-00:00
Release of Uucd and Uunf to support Unicode 7.0.0.


Tsdl 0.8.0
erratique — 2014-05-18T23:57:31-00:00
First release of Tsdl, thin bindings to SDL for OCaml.


Tgls 0.8.0
erratique — 2014-05-18T17:05:49-00:00
First release of Tgls, thin bindings to OpenGL {3,4} and OpenGL ES {2,3} for OCaml.


Log erratique
erratique — 2014-05-18T14:00:32-00:00
Opening a log for publishing erratic snapshots of ongoing work.


New Cambridgian summer
erratique — 2014-05-09T09:13:30-00:00
OCaml Labs kindly invited me to spend a second summer in Cambridge for working on OCaml free software graphical libraries. I'm looking forward to reconnect with the regular lab members and meet the other interesting guests.


React 1.0.0
erratique — 2014-04-02T16:05:26-00:00
Release of React 1.0.0, consult the release notes for details.


Cmdliner 0.9.4
erratique — 2014-02-09T21:38:16-00:00
Release of Cmdliner 0.9.4, consult the release notes for details.


Uucd 1.0.0 and Uunf 0.9.2
erratique — 2013-10-01T12:54:03-00:00
Release of Uucd and Uunf to support Unicode 6.3.0. Consult the individual release notes for details.


Gg 0.8.0 and Vg 0.8.0
erratique — 2013-09-24T22:29:33-00:00
First release of Gg and Vg. Thanks to Citrix Systems R&D and OCaml Labs for sponsoring this work.


Uutf 0.9.3 and Xmlm 1.2.0
erratique — 2013-09-06T16:55:54-00:00
Release of Uutf 0.9.3 and Xmlm 1.2.0. Consult the individual release notes for details.


On the book « OCaml from the very beginning »
erratique — 2013-07-04T22:43:32-00:00
Cambridge overflows with OCaml programmers. I got to meet John Whitington of camlpdf fame. He kindly offered me a copy of his book « OCaml from the very beginning. » I read it and made a small review here.


Four
erratique — 2013-02-14T21:29:49-00:00
Gaspard Buma's show Boats to nowhere is now playing at the Arsenic. I helped with the show's visuals and along the way prototyped Four a lightweight modern OpenGL rendering engine for Lua.


Winter distribution
erratique — 2013-01-05T17:18:05-00:00
Release of Cmdliner 0.9.3, Uutf 0.9.2, Uunf 0.9.1 and Uucd 0.9.2. Consult the individual release notes for details.


Uunf 0.9.0
erratique — 2012-09-07T16:47:59-00:00
First release of Uunf, an OCaml module for Unicode text normalization.


Uucd 0.9.0
erratique — 2012-09-07T16:47:58-00:00
First release of Uucd, an Unicode character database decoder for OCaml.


Jsonm 0.9.0
erratique — 2012-05-05T14:18:37-00:00
First release of Jsonm, a non-blocking streaming JSON codec for OCaml.


Uutf 0.9.0
erratique — 2012-05-05T14:16:08-00:00
First release of Uutf, a non-blocking streaming Unicode codec for OCaml.


Spring release
erratique — 2012-03-17T11:42:26-00:00
Release of Cmdliner 0.9.1, React 0.9.3, Rtime 0.9.2, Uuidm 0.9.4, and Xmlm 1.1.0. Consult the individual release notes for details. All modules now support OASIS.


Cmdliner 0.9.0
erratique — 2011-05-27T14:49:02-00:00
First release of Cmdliner, an OCaml module for the declarative definition of command line interfaces.


React 0.9.2
erratique — 2010-04-25T20:41:26-00:00
Fixes a bug in the weak heap implementation. Thanks to Jake Donham for reporting and discussing the fix.


React 0.9.1 and Rtime 0.9.1
erratique — 2010-04-15T17:05:26-00:00
Minor release of React and Rtime.


Rtime 0.9.0
erratique — 2009-01-20T03:00:20-00:00
First release of Rtime, an OCaml module implementing timelines for React.


React 0.9.0
erratique — 2009-01-20T02:57:19-00:00
First release of React, an OCaml module for functional reactive programming (FRP).


Uuidm 0.9.0
erratique — 2008-06-11T18:10:19-00:00
First release of Uuidm, an OCaml module for universally unique identifiers (UUIDs).


Xmlm 1.0.0
erratique — 2008-03-17T21:59:00-00:00
New release of Xmlm. Changes are detailed in the release notes of the distribution.


Anti-personnel computing
erratique — 2023-06-11T14:13:42-00:00
Neologisms for the current computing era.


Qrc
erratique — 2020-10-22T23:50:55-00:00
First release of Qrc, a QR code encoder for OCaml.


Brr
erratique — 2020-10-14T08:00:55-00:00
First release of Brr, a toolkit for programming browsers in OCaml.


Down
erratique — 2019-07-23T21:15:55-00:00
First release of Down, an unintrusive OCaml toplevel experience upgrade.


Unicode 10.0.0 refresh
erratique — 2017-06-21T00:15:00-00:00
Uucd, Uucp, Uunf and Uuseg were refreshed for Unicode 10.0.0. Find the details via the software page.


10
erratique — 2017-02-27T19:15:55-00:00
Ten years ago I published a first bit of free software. Over the years, this has now expanded to 28 OCaml packages, without counting random informally published elucubrations. Thanks to OCaml Labs' support, there will be more.


Unicode 9.0.0 refresh
erratique — 2016-11-23T17:00:49-00:00
Uucd, Uucp, Uunf, Uuseg and Uutf were all refreshed for Unicode 9.0.0. They also support the new Uchar.t type introduced in OCaml 4.03 and compile with safe string. Find about them on the software page.


Asetmap, Bos, Hmap and Webbrowser
erratique — 2016-09-29T21:40:49-00:00
A few other new releases that went unannounced on this feed. Find about them on the software page.


Topkg
erratique — 2016-09-29T21:40:48-00:00
It seems the Topkg rewrite went unannounced on this feed. If you are in need of a no-nonsense release workflow to package and publish your packages on OPAM, simply topkg them.


Odig 0.0.1
erratique — 2016-09-29T20:53:49-00:00
First release of Odig. If you can't keep track of the documentation of the OCaml packages you use, just odig them locally.


Astring, Fmt, Logs, Ptime, Mtime, Rresult
erratique — 2015-12-24T22:55:00-00:00
A few new releases that went unannounced on this feed. Find about them on the software page.


Release of Uucd 3.0.0, Uucp 1.0.0, Uunf 1.0.0 and Uuseg 0.9.0
erratique — 2015-06-17T23:55:50-00:00
Release of Uucd, Uucp, Uunf and Uuseg to support Unicode 8.0.0.


On the book « More OCaml »
erratique — 2015-05-23T15:17:21-00:00
I eventually read through John Whitington's latest book « More OCaml. » It's again a very good one, I made a small review here.


Uuseg 0.8.0
erratique — 2014-12-23T21:07:47-00:00
First release of Uuseg, Unicode text segmentation for OCaml.


Cmdliner 0.9.6
erratique — 2014-11-18T14:23:09-00:00
Release of Cmdliner 0.9.6, consult the release notes for details.


Uucp 0.9.0
erratique — 2014-06-28T21:53:49-00:00
First release of Uucp, access to Unicode character properties for OCaml. Thanks to OCaml Labs for sponsoring this work.


Uucd 2.0.0 and Uunf 0.9.3
erratique — 2014-06-16T23:41:53-00:00
Release of Uucd and Uunf to support Unicode 7.0.0.


Tsdl 0.8.0
erratique — 2014-05-18T23:57:31-00:00
First release of Tsdl, thin bindings to SDL for OCaml.


Tgls 0.8.0
erratique — 2014-05-18T17:05:49-00:00
First release of Tgls, thin bindings to OpenGL {3,4} and OpenGL ES {2,3} for OCaml.


Log erratique
erratique — 2014-05-18T14:00:32-00:00
Opening a log for publishing erratic snapshots of ongoing work.


New Cambridgian summer
erratique — 2014-05-09T09:13:30-00:00
OCaml Labs kindly invited me to spend a second summer in Cambridge for working on OCaml free software graphical libraries. I'm looking forward to reconnect with the regular lab members and meet the other interesting guests.


React 1.0.0
erratique — 2014-04-02T16:05:26-00:00
Release of React 1.0.0, consult the release notes for details.


Cmdliner 0.9.4
erratique — 2014-02-09T21:38:16-00:00
Release of Cmdliner 0.9.4, consult the release notes for details.


Uucd 1.0.0 and Uunf 0.9.2
erratique — 2013-10-01T12:54:03-00:00
Release of Uucd and Uunf to support Unicode 6.3.0. Consult the individual release notes for details.


Gg 0.8.0 and Vg 0.8.0
erratique — 2013-09-24T22:29:33-00:00
First release of Gg and Vg. Thanks to Citrix Systems R&D and OCaml Labs for sponsoring this work.


Uutf 0.9.3 and Xmlm 1.2.0
erratique — 2013-09-06T16:55:54-00:00
Release of Uutf 0.9.3 and Xmlm 1.2.0. Consult the individual release notes for details.


On the book « OCaml from the very beginning »
erratique — 2013-07-04T22:43:32-00:00
Cambridge overflows with OCaml programmers. I got to meet John Whitington of camlpdf fame. He kindly offered me a copy of his book « OCaml from the very beginning. » I read it and made a small review here.


Four
erratique — 2013-02-14T21:29:49-00:00
Gaspard Buma's show Boats to nowhere is now playing at the Arsenic. I helped with the show's visuals and along the way prototyped Four a lightweight modern OpenGL rendering engine for Lua.


Winter distribution
erratique — 2013-01-05T17:18:05-00:00
Release of Cmdliner 0.9.3, Uutf 0.9.2, Uunf 0.9.1 and Uucd 0.9.2. Consult the individual release notes for details.


Uunf 0.9.0
erratique — 2012-09-07T16:47:59-00:00
First release of Uunf, an OCaml module for Unicode text normalization.


Uucd 0.9.0
erratique — 2012-09-07T16:47:58-00:00
First release of Uucd, an Unicode character database decoder for OCaml.


Jsonm 0.9.0
erratique — 2012-05-05T14:18:37-00:00
First release of Jsonm, a non-blocking streaming JSON codec for OCaml.


Uutf 0.9.0
erratique — 2012-05-05T14:16:08-00:00
First release of Uutf, a non-blocking streaming Unicode codec for OCaml.


Spring release
erratique — 2012-03-17T11:42:26-00:00
Release of Cmdliner 0.9.1, React 0.9.3, Rtime 0.9.2, Uuidm 0.9.4, and Xmlm 1.1.0. Consult the individual release notes for details. All modules now support OASIS.


Cmdliner 0.9.0
erratique — 2011-05-27T14:49:02-00:00
First release of Cmdliner, an OCaml module for the declarative definition of command line interfaces.


React 0.9.2
erratique — 2010-04-25T20:41:26-00:00
Fixes a bug in the weak heap implementation. Thanks to Jake Donham for reporting and discussing the fix.


React 0.9.1 and Rtime 0.9.1
erratique — 2010-04-15T17:05:26-00:00
Minor release of React and Rtime.


Rtime 0.9.0
erratique — 2009-01-20T03:00:20-00:00
First release of Rtime, an OCaml module implementing timelines for React.


React 0.9.0
erratique — 2009-01-20T02:57:19-00:00
First release of React, an OCaml module for functional reactive programming (FRP).


Uuidm 0.9.0
erratique — 2008-06-11T18:10:19-00:00
First release of Uuidm, an OCaml module for universally unique identifiers (UUIDs).


Xmlm 1.0.0
erratique — 2008-03-17T21:59:00-00:00
New release of Xmlm. Changes are detailed in the release notes of the distribution.


Release of Frama-C 27.0 (Cobalt)
frama-c — 2023-06-15T00:00:00-00:00



Beta release of Frama-C 27.0~beta (Cobalt)
frama-c — 2023-05-16T00:00:00-00:00



Cyberhackathon Frama-C + Binsec
frama-c — 2023-04-28T00:00:00-00:00



3-year Engineer Position at CEA LIST - LSL
frama-c — 2023-02-28T00:00:00-00:00



3-year Computer Scientist Position at CEA LIST - LSL
frama-c — 2023-02-28T00:00:00-00:00



3-year Engineer Position at CEA LIST - LSL
frama-c — 2023-02-28T00:00:00-00:00



3-year Computer Scientist Position at CEA LIST - LSL
frama-c — 2023-02-28T00:00:00-00:00



Release of Frama-C 26.1 (Iron)
frama-c — 2023-02-15T00:00:00-00:00



MetAcsl for Frama-C 26.0 Iron
frama-c — 2022-12-09T00:00:00-00:00



Release of Frama-C 26.0 (Iron)
frama-c — 2022-11-23T00:00:00-00:00



Internship Position at CEA LIST - LSL
frama-c — 2022-11-04T00:00:00-00:00



Beta release of Frama-C 26.0~beta (Iron)
frama-c — 2022-10-28T00:00:00-00:00



Internship Position at CEA LIST - LSL
frama-c — 2022-10-21T00:00:00-00:00



Release of Frama-Clang 0.0.13
frama-c — 2022-10-06T00:00:00-00:00



Release of Frama-C 25.0 (Manganese)
frama-c — 2022-06-21T00:00:00-00:00



Beta release of Frama-C 25.0~beta (Manganese)
frama-c — 2022-05-11T00:00:00-00:00



Release of Frama-Clang 0.0.12
frama-c — 2022-04-12T00:00:00-00:00



Release of LTest 0.1
frama-c — 2022-04-08T00:00:00-00:00



PhD Position at CEA LIST - LSL
frama-c — 2022-03-28T00:00:00-00:00



Permanent Computer Scientist Position at CEA LIST - LSL
frama-c — 2022-02-01T00:00:00-00:00



Release of Frama-C 24.0 (Chromium)
frama-c — 2021-11-30T00:00:00-00:00



Beta release of Frama-C 24.0-beta (Chromium)
frama-c — 2021-11-04T00:00:00-00:00



PhD Position at CEA List - LSL
frama-c — 2021-08-05T00:00:00-00:00



Postdoc Position at CEA List - LSL
frama-c — 2021-08-05T00:00:00-00:00



Postdoc Position at CEA List - LSL
frama-c — 2021-08-05T00:00:00-00:00



Release of Frama-C 23.1 (Vanadium)
frama-c — 2021-07-21T00:00:00-00:00



Release of Frama-Clang 0.0.11
frama-c — 2021-07-13T00:00:00-00:00



Release of Frama-C 23.0 (Vanadium)
frama-c — 2021-07-06T00:00:00-00:00



Beta release of Frama-C 23.0~rc1 (Vanadium)
frama-c — 2021-05-20T00:00:00-00:00



Release of Frama-Clang 0.0.10
frama-c — 2021-03-08T00:00:00-00:00



First release of MetAcsl plugin
frama-c — 2020-12-09T00:00:00-00:00



Release of Frama-C 22.0 (Titanium)
frama-c — 2020-11-20T00:00:00-00:00



Beta release of Frama-C 22.0-beta (Titanium)
frama-c — 2020-10-28T00:00:00-00:00



Release of Frama-Clang 0.0.9
frama-c — 2020-07-15T00:00:00-00:00



Release of Frama-C 21.1 (Scandium)
frama-c — 2020-06-25T00:00:00-00:00



Release of Frama-C 21.0 (Scandium)
frama-c — 2020-06-12T00:00:00-00:00



Release of Frama-Clang 0.0.8
frama-c — 2020-03-10T00:00:00-00:00



Release of Frama-C 20.0 (Calcium)
frama-c — 2019-12-04T00:00:00-00:00



Release of Frama-C 19.1 (Potassium)
frama-c — 2019-09-17T00:00:00-00:00



Release of Frama-Clang 0.0.7
frama-c — 2019-09-13T00:00:00-00:00



Release of Frama-C 19.0 (Potassium)
frama-c — 2019-06-21T00:00:00-00:00



Frama-C and SPARK Day 2019
frama-c — 2019-05-18T00:00:00-00:00



Release of Frama-C 18.0 (Argon)
frama-c — 2018-10-29T00:00:00-00:00



Release of Frama-Clang 0.0.6
frama-c — 2018-07-23T00:00:00-00:00



Release of Frama-C 17.0 (Chlorine)
frama-c — 2018-05-31T00:00:00-00:00



Release of Frama-Clang 0.0.5
frama-c — 2018-02-19T00:00:00-00:00



Release of Frama-Clang 0.0.4
frama-c — 2017-12-21T00:00:00-00:00



Release of Frama-C 16.0 (Sulfur)
frama-c — 2017-11-29T00:00:00-00:00



Release of Frama-Clang 0.0.3
frama-c — 2017-08-02T00:00:00-00:00



Release of Frama-C 15.0 (Phosphorus)
frama-c — 2017-05-31T00:00:00-00:00



Release of Frama-Clang 0.0.2
frama-c — 2017-05-05T00:00:00-00:00



Release of E-ACSL 0.0.8
frama-c — 2017-01-17T00:00:00-00:00



Release of Frama-C 14.0 (Silicon)
frama-c — 2016-12-01T00:00:00-00:00



Release of Frama-Clang 0.0.1
frama-c — 2016-07-04T00:00:00-00:00



Release of Frama-C 13.0 (Aluminium)
frama-c — 2016-05-31T00:00:00-00:00



Release of Frama-C 12.0 (Magnesium)
frama-c — 2016-01-15T00:00:00-00:00



Release of Frama-C 11.0 (Sodium)
frama-c — 2015-03-06T00:00:00-00:00



Release of Frama-C 10.0 (Neon)
frama-c — 2014-03-07T00:00:00-00:00



Release of Frama-C 27.0 (Cobalt)
frama-c — 2023-06-15T00:00:00-00:00



Beta release of Frama-C 27.0~beta (Cobalt)
frama-c — 2023-05-16T00:00:00-00:00



Cyberhackathon Frama-C + Binsec
frama-c — 2023-04-28T00:00:00-00:00



3-year Engineer Position at CEA LIST - LSL
frama-c — 2023-02-28T00:00:00-00:00



3-year Computer Scientist Position at CEA LIST - LSL
frama-c — 2023-02-28T00:00:00-00:00



3-year Engineer Position at CEA LIST - LSL
frama-c — 2023-02-28T00:00:00-00:00



3-year Computer Scientist Position at CEA LIST - LSL
frama-c — 2023-02-28T00:00:00-00:00



Release of Frama-C 26.1 (Iron)
frama-c — 2023-02-15T00:00:00-00:00



MetAcsl for Frama-C 26.0 Iron
frama-c — 2022-12-09T00:00:00-00:00



Release of Frama-C 26.0 (Iron)
frama-c — 2022-11-23T00:00:00-00:00



Internship Position at CEA LIST - LSL
frama-c — 2022-11-04T00:00:00-00:00



Beta release of Frama-C 26.0~beta (Iron)
frama-c — 2022-10-28T00:00:00-00:00



Internship Position at CEA LIST - LSL
frama-c — 2022-10-21T00:00:00-00:00



Release of Frama-Clang 0.0.13
frama-c — 2022-10-06T00:00:00-00:00



Release of Frama-C 25.0 (Manganese)
frama-c — 2022-06-21T00:00:00-00:00



Beta release of Frama-C 25.0~beta (Manganese)
frama-c — 2022-05-11T00:00:00-00:00



Release of Frama-Clang 0.0.12
frama-c — 2022-04-12T00:00:00-00:00



Release of LTest 0.1
frama-c — 2022-04-08T00:00:00-00:00



PhD Position at CEA LIST - LSL
frama-c — 2022-03-28T00:00:00-00:00



Permanent Computer Scientist Position at CEA LIST - LSL
frama-c — 2022-02-01T00:00:00-00:00



Release of Frama-C 24.0 (Chromium)
frama-c — 2021-11-30T00:00:00-00:00



Beta release of Frama-C 24.0-beta (Chromium)
frama-c — 2021-11-04T00:00:00-00:00



PhD Position at CEA List - LSL
frama-c — 2021-08-05T00:00:00-00:00



Postdoc Position at CEA List - LSL
frama-c — 2021-08-05T00:00:00-00:00



Postdoc Position at CEA List - LSL
frama-c — 2021-08-05T00:00:00-00:00



Release of Frama-C 23.1 (Vanadium)
frama-c — 2021-07-21T00:00:00-00:00



Release of Frama-Clang 0.0.11
frama-c — 2021-07-13T00:00:00-00:00



Release of Frama-C 23.0 (Vanadium)
frama-c — 2021-07-06T00:00:00-00:00



Beta release of Frama-C 23.0~rc1 (Vanadium)
frama-c — 2021-05-20T00:00:00-00:00



Release of Frama-Clang 0.0.10
frama-c — 2021-03-08T00:00:00-00:00



First release of MetAcsl plugin
frama-c — 2020-12-09T00:00:00-00:00



Release of Frama-C 22.0 (Titanium)
frama-c — 2020-11-20T00:00:00-00:00



Beta release of Frama-C 22.0-beta (Titanium)
frama-c — 2020-10-28T00:00:00-00:00



Release of Frama-Clang 0.0.9
frama-c — 2020-07-15T00:00:00-00:00



Release of Frama-C 21.1 (Scandium)
frama-c — 2020-06-25T00:00:00-00:00



Release of Frama-C 21.0 (Scandium)
frama-c — 2020-06-12T00:00:00-00:00



Release of Frama-Clang 0.0.8
frama-c — 2020-03-10T00:00:00-00:00



Release of Frama-C 20.0 (Calcium)
frama-c — 2019-12-04T00:00:00-00:00



Release of Frama-C 19.1 (Potassium)
frama-c — 2019-09-17T00:00:00-00:00



Release of Frama-Clang 0.0.7
frama-c — 2019-09-13T00:00:00-00:00



Release of Frama-C 19.0 (Potassium)
frama-c — 2019-06-21T00:00:00-00:00



Frama-C and SPARK Day 2019
frama-c — 2019-05-18T00:00:00-00:00



Release of Frama-C 18.0 (Argon)
frama-c — 2018-10-29T00:00:00-00:00



Release of Frama-Clang 0.0.6
frama-c — 2018-07-23T00:00:00-00:00



Release of Frama-C 17.0 (Chlorine)
frama-c — 2018-05-31T00:00:00-00:00



Release of Frama-Clang 0.0.5
frama-c — 2018-02-19T00:00:00-00:00



Release of Frama-Clang 0.0.4
frama-c — 2017-12-21T00:00:00-00:00



Release of Frama-C 16.0 (Sulfur)
frama-c — 2017-11-29T00:00:00-00:00



Release of Frama-Clang 0.0.3
frama-c — 2017-08-02T00:00:00-00:00



Release of Frama-C 15.0 (Phosphorus)
frama-c — 2017-05-31T00:00:00-00:00



Release of Frama-Clang 0.0.2
frama-c — 2017-05-05T00:00:00-00:00



Release of E-ACSL 0.0.8
frama-c — 2017-01-17T00:00:00-00:00



Release of Frama-C 14.0 (Silicon)
frama-c — 2016-12-01T00:00:00-00:00



Release of Frama-Clang 0.0.1
frama-c — 2016-07-04T00:00:00-00:00



Release of Frama-C 13.0 (Aluminium)
frama-c — 2016-05-31T00:00:00-00:00



Release of Frama-C 12.0 (Magnesium)
frama-c — 2016-01-15T00:00:00-00:00



Release of Frama-C 11.0 (Sodium)
frama-c — 2015-03-06T00:00:00-00:00



Release of Frama-C 10.0 (Neon)
frama-c — 2014-03-07T00:00:00-00:00



gaius
gaius — 2023-07-14T05:47:24-00:00
Generally, every program I write, regardless of what useful thing it actually does, and regardless of what programming language it is written in, has to do certain things, which usually includes
+https://gaius.tech/category/ocaml/feed/gaius2023-07-14T05:47:24-00:00https://gaius.tech/category/ocaml/feed/gaiusGenerally, every program I write, regardless of what useful thing it actually does, and regardless of what programming language it is written in, has to do certain things, which usually includes
 
 Importing a library and calling functions contained within that library
 Handling datatypes such as converting between strings and integers, and knowing when this is implicit or explicit, how dates and times work, and so on
diff --git a/data/planet/gallium.xml b/data/planet/gallium.xml
index e4cccf5ccf..eefd5b9b0e 100644
--- a/data/planet/gallium.xml
+++ b/data/planet/gallium.xml
@@ -1,5 +1,5 @@
 
-http://gallium.inria.fr/blog/index.rssgallium2023-07-13T05:47:27-00:00http://gallium.inria.fr/blog/index.rssgallium
+http://gallium.inria.fr/blog/index.rssgallium2023-07-14T05:47:24-00:00http://gallium.inria.fr/blog/index.rssgallium
 
 
 This series of blog posts aims to give a short weekly glimpse into my
diff --git a/data/planet/hannes.xml b/data/planet/hannes.xml
index d76b2f2dc4..baf8a0d9a8 100644
--- a/data/planet/hannes.xml
+++ b/data/planet/hannes.xml
@@ -1,5 +1,5 @@
 
-https://hannes.robur.coop/atomhannes2023-07-13T05:47:26-00:00https://hannes.robur.coop/atomhannes
EDIT (2023-05-16): Updated with albatross release version 2.0.0.
+https://hannes.robur.coop/atomhannes2023-07-14T05:47:24-00:00https://hannes.robur.coop/atomhannesEDIT (2023-05-16): Updated with albatross release version 2.0.0.
 Deploying MirageOS unikernels
 More than five years ago, I posted how to deploy MirageOS unikernels. My motivation to work on this topic is that I'm convinced of reduced complexity, improved security, and more sustainable resource footprint of MirageOS unikernels, and want to ease deployment thereof. More than one year ago, I described how to deploy reproducible unikernels.
 Albatross
diff --git a/data/planet/hongboz.xml b/data/planet/hongboz.xml
index a2413db438..765e4b847b 100644
--- a/data/planet/hongboz.xml
+++ b/data/planet/hongboz.xml
@@ -1,5 +1,5 @@
 
-https://hongboz.wordpress.com/feed/hongboz2023-07-13T05:47:28-00:00https://hongboz.wordpress.com/feed/hongboz
+https://hongboz.wordpress.com/feed/hongboz2023-07-14T05:47:24-00:00https://hongboz.wordpress.com/feed/hongboz
 
 Fan is a metaprogramming tool for OCaml.  I believe it would be an

 invaluable tool for the community when it’s production ready.
diff --git a/data/planet/inria.xml b/data/planet/inria.xml
index de2430852b..8e02eb7d3e 100644
--- a/data/planet/inria.xml
+++ b/data/planet/inria.xml
@@ -1,5 +1,5 @@
 
-http://caml.inria.fr/news.en.rssinria2023-07-13T05:47:27-00:00http://caml.inria.fr/news.en.rssinria


OCaml 4.06.1 released
inria — 2018-02-16T00:00:00-00:00



OCaml 4.06.0 released
inria — 2017-11-03T00:00:00-00:00



OCaml 4.05.0 released
inria — 2017-07-13T00:00:00-00:00



OCaml 4.04.2 released
inria — 2017-06-23T00:00:00-00:00



OCaml 4.04.1 released
inria — 2017-04-14T00:00:00-00:00



Docker joins the Caml Consortium
inria — 2017-01-17T00:00:00-00:00



OCaml 4.04.0 released
inria — 2016-11-04T00:00:00-00:00



Kernelyze LLC joins the Caml Consortium
inria — 2016-08-09T00:00:00-00:00



Ahrefs joins the Caml Consortium
inria — 2016-05-12T00:00:00-00:00



OCaml 4.03.0 released
inria — 2016-04-25T00:00:00-00:00



Facebook joins the Caml Consortium
inria — 2016-04-15T00:00:00-00:00



OCaml 4.02.3 released
inria — 2015-07-27T00:00:00-00:00



OCaml 4.02.2 released
inria — 2015-06-17T00:00:00-00:00



Aesthetic Integration joins the Caml Consortium
inria — 2015-06-03T00:00:00-00:00



Bloomberg joins the Caml Consortium
inria — 2014-11-06T00:00:00-00:00



OCaml 4.02.1 released
inria — 2014-10-14T00:00:00-00:00



New book: Apprendre à programmer avec OCaml:
+http://caml.inria.fr/news.en.rssinria</h1>
<p>inria — 2023-07-14T05:47:24-00:00</p>

</article>
<article>
<h1>OCaml 4.06.1 released</h1>
<p>inria — 2018-02-16T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.06.0 released</h1>
<p>inria — 2017-11-03T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.05.0 released</h1>
<p>inria — 2017-07-13T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.04.2 released</h1>
<p>inria — 2017-06-23T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.04.1 released</h1>
<p>inria — 2017-04-14T00:00:00-00:00</p>

</article>
<article>
<h1>Docker joins the Caml Consortium</h1>
<p>inria — 2017-01-17T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.04.0 released</h1>
<p>inria — 2016-11-04T00:00:00-00:00</p>

</article>
<article>
<h1>Kernelyze LLC joins the Caml Consortium</h1>
<p>inria — 2016-08-09T00:00:00-00:00</p>

</article>
<article>
<h1>Ahrefs joins the Caml Consortium</h1>
<p>inria — 2016-05-12T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.03.0 released</h1>
<p>inria — 2016-04-25T00:00:00-00:00</p>

</article>
<article>
<h1>Facebook joins the Caml Consortium</h1>
<p>inria — 2016-04-15T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.02.3 released</h1>
<p>inria — 2015-07-27T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.02.2 released</h1>
<p>inria — 2015-06-17T00:00:00-00:00</p>

</article>
<article>
<h1>Aesthetic Integration joins the Caml Consortium</h1>
<p>inria — 2015-06-03T00:00:00-00:00</p>

</article>
<article>
<h1>Bloomberg joins the Caml Consortium</h1>
<p>inria — 2014-11-06T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.02.1 released</h1>
<p>inria — 2014-10-14T00:00:00-00:00</p>

</article>
<article>
<h1>New book: Apprendre à programmer avec OCaml:
 		Algorithmes et structures de données,
       by Sylvain Conchon and Jean-Christophe Filliâtre</h1>
<p>inria — 2014-09-11T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.02.0 released</h1>
<p>inria — 2014-08-29T00:00:00-00:00</p>

</article>
<article>
<h1>New book: Real World OCaml, by Yaron Minsky, Anil Madhavepeddy, and Jason Hickey</h1>
<p>inria — 2013-11-17T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.01.0 released</h1>
<p>inria — 2013-09-12T00:00:00-00:00</p>

</article>
<article>
<h1>New book: OCaml from the very beginning, by John Whitington</h1>
<p>inria — 2013-06-20T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.00.1 released</h1>
<p>inria — 2012-10-05T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 4.00.0 released</h1>
<p>inria — 2012-07-26T00:00:00-00:00</p>

</article>
<article>
<h1>"OCaml for the Masses", a paper by Yaron Minsky, published in ACM Queue</h1>
<p>inria — 2011-11-05T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml renamed to OCaml</h1>
<p>inria — 2011-07-29T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml 3.12.1 released</h1>
<p>inria — 2011-07-04T00:00:00-00:00</p>

</article>
<article>
<h1>OCamlPro becomes member of the Caml Consortium</h1>
<p>inria — 2011-05-10T00:00:00-00:00</p>

</article>
<article>
<h1>Esterel Technologies becomes member of the Caml Consortium</h1>
<p>inria — 2011-04-20T00:00:00-00:00</p>

</article>
<article>
<h1>Mylife.com becomes member of the Caml Consortium</h1>
<p>inria — 2010-09-21T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.12.0 released</h1>
<p>inria — 2010-08-02T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.12.0+beta1 released</h1>
<p>inria — 2010-06-16T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.11.2 released</h1>
<p>inria — 2010-01-20T00:00:00-00:00</p>

</article>
<article>
<h1>MLstate becomes member of the Caml Consortium</h1>
<p>inria — 2010-01-04T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.11.1 released</h1>
<p>inria — 2009-06-12T00:00:00-00:00</p>

</article>
<article>
<h1>SimCorp becomes member of the Caml Consortium</h1>
<p>inria — 2009-02-13T00:00:00-00:00</p>

</article>
<article>
<h1>OCaml users' meeting in Grenoble</h1>
<p>inria — 2009-02-04T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.11.0 released</h1>
<p>inria — 2008-12-04T00:00:00-00:00</p>

</article>
<article>
<h1>OCamlCore becomes member of the Caml Consortium</h1>
<p>inria — 2008-09-19T00:00:00-00:00</p>

</article>
<article>
<h1>CEA becomes member of the Caml Consortium</h1>
<p>inria — 2008-06-03T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.10.2 released</h1>
<p>inria — 2008-02-29T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.10.1 released</h1>
<p>inria — 2008-01-11T00:00:00-00:00</p>

</article>
<article>
<h1>Jane Street Capital becomes member of the Caml Consortium</h1>
<p>inria — 2007-11-20T00:00:00-00:00</p>

</article>
<article>
<h1>XenSource becomes member of the Caml Consortium</h1>
<p>inria — 2007-09-01T00:00:00-00:00</p>

</article>
<article>
<h1>Intel becomes member of the Caml Consortium</h1>
<p>inria — 2007-06-15T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.10 released</h1>
<p>inria — 2007-05-17T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.09.3 released (bug-fix)</h1>
<p>inria — 2006-09-15T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.09.2 released (bug-fix)</h1>
<p>inria — 2006-04-14T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.09.1 released (bug-fix)</h1>
<p>inria — 2006-01-04T00:00:00-00:00</p>

</article>
<article>
<h1>New bug tracker for OCaml</h1>
<p>inria — 2005-11-18T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.09 released</h1>
<p>inria — 2005-10-27T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.08.4 released (bug-fix)</h1>
<p>inria — 2005-08-16T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.08.3 released (bug-fix)</h1>
<p>inria — 2005-03-17T00:00:00-00:00</p>

</article>
<article>
<h1>New web site</h1>
<p>inria — 2005-03-14T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.08.2 released (bug-fix)</h1>
<p>inria — 2004-11-01T00:00:00-00:00</p>

</article>
<article>
<h1>An Objective Caml program wins the judge's prize at the
     ICFP 2004 programming contest.</h1>
<p>inria — 2004-09-20T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.08.1 released (bug-fix)</h1>
<p>inria — 2004-08-01T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.08 released</h1>
<p>inria — 2004-07-01T00:00:00-00:00</p>

</article>
<article>
<h1>Objective Caml 3.07 released</h1>
<p>inria — 2003-10-01T00:00:00-00:00</p>

</article>
<article>
<h1>An Objective Caml program wins the "lightning division" at the
diff --git a/data/planet/janestreet.xml b/data/planet/janestreet.xml
index 2040a8db44..3c2048d189 100644
--- a/data/planet/janestreet.xml
+++ b/data/planet/janestreet.xml
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<feed xmlns="http://www.w3.org/2005/Atom"><id>https://blog.janestreet.com/feed.xml</id><title type="text">janestreet</h1>
<p>inria — 2023-07-13T05:47:26-00:00</p>
<p>Jane Street is excited to announce our sponsorship of
+<feed xmlns="http://www.w3.org/2005/Atom"><id>https://blog.janestreet.com/feed.xml</id><title type="text">janestreet2023-07-14T05:47:24-00:00https://blog.janestreet.com/feed.xmljanestreetJane Street is excited to announce our sponsorship of
 SoME3, Grant Sanderson and James Schloss’s
 third Summer of Math Exposition. SoME is a contest that Grant and
 James created to encourage the development of fun and interesting
diff --git a/data/planet/kcsrk.xml b/data/planet/kcsrk.xml
index 06b8119254..fbad7f3bdf 100644
--- a/data/planet/kcsrk.xml
+++ b/data/planet/kcsrk.xml
@@ -1,5 +1,5 @@
 
-https://kcsrk.info/atom-ocaml.xmlkcsrk2023-07-13T05:47:28-00:00https://kcsrk.info/atom-ocaml.xmlkcsrk
It has been 3 months since I joined IIT Madras and it
+https://kcsrk.info/atom-ocaml.xmlkcsrk2023-07-14T05:47:25-00:00https://kcsrk.info/atom-ocaml.xmlkcsrk
It has been 3 months since I joined IIT Madras and it
 has been good fun so far. Along with the members of the RISE
 group, we’ve initiated a project to build secure
 applications on top of secure extensions of the open-source
diff --git a/data/planet/lpw25.xml b/data/planet/lpw25.xml
index bf95504dd0..c9f90ad668 100644
--- a/data/planet/lpw25.xml
+++ b/data/planet/lpw25.xml
@@ -1,5 +1,5 @@
 
-http://lpw25.net/rss.xmllpw252023-07-13T05:47:28-00:00http://lpw25.net/rss.xmllpw25

+http://lpw25.net/rss.xmllpw252023-07-14T05:47:25-00:00http://lpw25.net/rss.xmllpw25
   
 Contents
 
diff --git a/data/planet/mcclurmc.xml b/data/planet/mcclurmc.xml
index 0cf535beee..3a4c5d0432 100644
--- a/data/planet/mcclurmc.xml
+++ b/data/planet/mcclurmc.xml
@@ -1,5 +1,5 @@
 
-https://mcclurmc.wordpress.com/feed/mcclurmc2023-07-13T05:47:28-00:00https://mcclurmc.wordpress.com/feed/mcclurmcI’ve recently stumbled upon a useful OCaml design pattern for functorizing an existing module, without changing the way existing clients of that module use it. This is really useful for stubbing out dependencies when you’re refactoring code for unit tests. Here’s a simplified example from a bit of code that I refactored today.
+https://mcclurmc.wordpress.com/feed/mcclurmc2023-07-14T05:47:25-00:00https://mcclurmc.wordpress.com/feed/mcclurmcI’ve recently stumbled upon a useful OCaml design pattern for functorizing an existing module, without changing the way existing clients of that module use it. This is really useful for stubbing out dependencies when you’re refactoring code for unit tests. Here’s a simplified example from a bit of code that I refactored today.
  module Ovs = struct
 
diff --git a/data/planet/mega-nerd.xml b/data/planet/mega-nerd.xml
index d7d4033b16..ed16862fb4 100644
--- a/data/planet/mega-nerd.xml
+++ b/data/planet/mega-nerd.xml
@@ -1,5 +1,5 @@
 
-http://www.mega-nerd.com/erikd/Blog/index.rss20mega-nerd2023-07-13T05:47:27-00:00http://www.mega-nerd.com/erikd/Blog/index.rss20mega-nerd
+http://www.mega-nerd.com/erikd/Blog/index.rss20mega-nerd2023-07-14T05:47:24-00:00http://www.mega-nerd.com/erikd/Blog/index.rss20mega-nerd
 
 
 Disclaimer: I work at Ambiata (our
diff --git a/data/planet/mgiovannini.xml b/data/planet/mgiovannini.xml
index ffce826126..4426c100a9 100644
--- a/data/planet/mgiovannini.xml
+++ b/data/planet/mgiovannini.xml
@@ -1,5 +1,5 @@
 
-http://www.blogger.com/feeds/5888658295182480819/posts/defaultmgiovannini2023-07-13T05:47:28-00:00http://www.blogger.com/feeds/5888658295182480819/posts/defaultmgiovanniniThis post might seem to be in apparent contradiction: Ocamlnet is a large, very opinionated framework for network programming that solves most, if not all, those infrastructure issues that need to be taken care of when writing a networked, distributed, fault-tolerant server, from process management to string decoding and including protocol parsing and building. The fact is that Ocamlnet is not


Merge Right
mgiovannini — 2012-08-07T00:15:00-00:00
The so-called master-transaction update is one of the, if not the defining algorithms of the discipline formerly known as "data processing". Given two sorted files of records with increasing keys, the process applies each record in the transaction file to each record of the the master file and outputs the result, if any, to the updated master file in one pass over each input. The same algorithm


mgiovannini
mgiovannini — 2023-07-14T05:47:25-00:00
You don't have to be writing an interpreter or some other kind of abstract code to profit from some phantom types. Suppose you have two or more functions that work by "cooking" a simple value (a float, say) with a lengthy computation before proceeding:
+http://www.blogger.com/feeds/5888658295182480819/posts/defaultmgiovannini2023-07-14T05:47:25-00:00http://www.blogger.com/feeds/5888658295182480819/posts/defaultmgiovanniniThis post might seem to be in apparent contradiction: Ocamlnet is a large, very opinionated framework for network programming that solves most, if not all, those infrastructure issues that need to be taken care of when writing a networked, distributed, fault-tolerant server, from process management to string decoding and including protocol parsing and building. The fact is that Ocamlnet is not


Merge Right
mgiovannini — 2012-08-07T00:15:00-00:00
The so-called master-transaction update is one of the, if not the defining algorithms of the discipline formerly known as "data processing". Given two sorted files of records with increasing keys, the process applies each record in the transaction file to each record of the the master file and outputs the result, if any, to the updated master file in one pass over each input. The same algorithm


mlin
mgiovannini — 2023-07-13T05:47:28-00:00
You don't have to be writing an interpreter or some other kind of abstract code to profit from some phantom types. Suppose you have two or more functions that work by "cooking" a simple value (a float, say) with a lengthy computation before proceeding:
 
 
 let sun_geometric_longitude j =
diff --git a/data/planet/mlin.xml b/data/planet/mlin.xml
index 4aab3e0ba6..5615bb321f 100644
--- a/data/planet/mlin.xml
+++ b/data/planet/mlin.xml
@@ -1,5 +1,5 @@
 
-http://www.blogger.com/feeds/3693082774051755513/posts/default/-/OCamlmlin2023-07-13T05:47:28-00:00http://www.blogger.com/feeds/3693082774051755513/posts/default/-/OCamlmlinUpdate (Oct 2013): Anil  Madhavapeddy has fleshed this out further.


+http://www.blogger.com/feeds/3693082774051755513/posts/default/-/OCamlmlin2023-07-14T05:47:25-00:00http://www.blogger.com/feeds/3693082774051755513/posts/default/-/OCamlmlinUpdate (Oct 2013): Anil  Madhavapeddy has fleshed this out further.


 This evening I spent some time getting unit tests for my OCaml projects to run on Travis CI, a free service for continuous integration on public GitHub projects. Although Travis has no built-in OCaml environment, it's straightforward to hijack its C environment to install OCaml and OPAM, then build an OCaml project and run its tests.

 

 1. Perform the initial setup to get Travis CI watching your GitHub repo (up to and including step two of that guide).

diff --git a/data/planet/ocaml-book.xml b/data/planet/ocaml-book.xml
index e4411cdd16..8c5cde4fd7 100644
--- a/data/planet/ocaml-book.xml
+++ b/data/planet/ocaml-book.xml
@@ -1,2 +1,2 @@
 
-http://ocaml-book.com/blog?format=rssocaml-book2023-07-13T05:47:28-00:00http://ocaml-book.com/blog?format=rssocaml-bookThanks to a grant from the OCaml Software Foundation (ocaml-sf.org), I am able to release my 2013 book OCaml from the Very Beginning free-of-charge in the existing PDF format, and in a new HTML format:

https://johnwhitington.net/ocamlfromtheverybeginning/

The book continues to be available in paperback and Kindle formats from Amazon.

The book has been updated for OCaml 4.14 in all formats.


"More OCaml" updated to support OCaml 4.06 immutable strings
ocaml-book — 2017-10-14T16:17:10-00:00
OCaml 4.06 has, by default, immutable strings. This necessitates some changes to "More OCaml" ("OCaml from the Very Beginning" is unaffected).
The paper and kindle books (on Amazon), as well as the PDF e-book here, have been updated, together with the online resources. The new code works on any version of OCaml starting from 4.02.0.
If you are an existing customer, and have not received an update email, please contact the author using the form on this website.


More OCaml: Algorithms, Methods & Diversions
ocaml-book — 2014-08-27T12:00:32-00:00
More OCaml: Algorithms, Methods & Diversions is now available on Amazon at $39.99 or less, or as a DRM-free PDF eBook for $19.99 at ocaml-book.com, where you will also find sample chapters, and the exercises and examples to download.
Here's the blurb:
 
In More OCaml John Whitington takes a meandering tour of functional programming with OCaml, introducing various language features and describing some classic algorithms. The book ends with a large worked example dealing with the production of PDF files. There are questions for each chapter together with worked answers and hints.
More OCaml will appeal both to existing OCaml programmers who wish to brush up their skills, and to experienced programmers eager to explore functional languages such as OCaml. It is hoped that each reader will find something new, or see an old thing in a new light. For the more casual reader, or those who are used to a different functional language, a summary of basic OCaml is provided at the front of the book.
 
If you buy the book, please do consider leaving a review on Amazon, which you can do even if you buy just the PDF eBook.
 


Reprinting of "OCaml from the Very Beginning"
ocaml-book — 2013-10-15T14:31:28-00:00
Owing to the print-on-demand process we're using, it's simple to update the book to fix the few errata which have been pointed out by kind readers since publication. The new text is now live on Amazon, the Kindle Store, and our website. 
To update the Kindle copy, go to "Manage My Kindle" on Amazon (there is no automatic update process). 
If you have purchased a PDF e-book, just email us for an updated PDF (it is possible to notify customers of an updated version by email, but since none of the errata are serious, we've opted not to do so). 
If you have a physical book, you can fix the errata yourself from the errata page on the website. In addition, if you have purchased the physical book, and would like a copy of the new PDF, please forward your amazon receipt to contact@coherentgraphics.co.uk 
And finally, if you have found mistakes in the book (or have any suggestions for the next edition), please do tell us, using the contact form on the website. 
 


OCaml  tutorial videos now available
ocaml-book — 2013-09-27T14:04:48-00:00
I've made a few little tutorial videos to go with the book. Just seven for now, covering the first four chapters, but more to come:
Evaluating an expression
True and false
Making decisions
Names and functions
Patterns
Lists
Take and drop
Appending two lists
They're not really a substitute for the book but, when learning, it's always helpful to go over things twice, particularly using a different medium.
The videos are available here. 
 


Now available on Amazon
ocaml-book — 2013-06-07T14:21:23-00:00
OCaml from the Very Beginning (204pp, paperback) is now available on Amazon for $39.99 / €29.99 / £24.99, as well as an e-book from ocaml-book.com for $19.99. Here are preview chapters and a .zip of the examples and exercises. 
Here's the blurb: 
In OCaml from the Very Beginning John Whitington takes a no-prerequisites approach to teaching a modern general-purpose programming language. Each small, self-contained chapter introduces a new topic, building until the reader can write quite substantial programs. There are plenty of questions and, crucially, worked answers and hints.
OCaml from the Very Beginning will appeal both to new programmers, and experienced programmers eager to explore functional languages such as OCaml. It is suitable both for formal use within an undergraduate or graduate curriculum, and for the interested amateur.
Please do review the book on Amazon if you have the chance.
 


OCaml from the Very Beginning now free in HTML and PDF formats
ocaml-book — 2022-02-18T16:49:32-00:00
Thanks to a grant from the OCaml Software Foundation (ocaml-sf.org), I am able to release my 2013 book OCaml from the Very Beginning free-of-charge in the existing PDF format, and in a new HTML format:

https://johnwhitington.net/ocamlfromtheverybeginning/

The book continues to be available in paperback and Kindle formats from Amazon.

The book has been updated for OCaml 4.14 in all formats.


"More OCaml" updated to support OCaml 4.06 immutable strings
ocaml-book — 2017-10-14T16:17:10-00:00
OCaml 4.06 has, by default, immutable strings. This necessitates some changes to "More OCaml" ("OCaml from the Very Beginning" is unaffected).
The paper and kindle books (on Amazon), as well as the PDF e-book here, have been updated, together with the online resources. The new code works on any version of OCaml starting from 4.02.0.
If you are an existing customer, and have not received an update email, please contact the author using the form on this website.


More OCaml: Algorithms, Methods & Diversions
ocaml-book — 2014-08-27T12:00:32-00:00
More OCaml: Algorithms, Methods & Diversions is now available on Amazon at $39.99 or less, or as a DRM-free PDF eBook for $19.99 at ocaml-book.com, where you will also find sample chapters, and the exercises and examples to download.
Here's the blurb:
 
In More OCaml John Whitington takes a meandering tour of functional programming with OCaml, introducing various language features and describing some classic algorithms. The book ends with a large worked example dealing with the production of PDF files. There are questions for each chapter together with worked answers and hints.
More OCaml will appeal both to existing OCaml programmers who wish to brush up their skills, and to experienced programmers eager to explore functional languages such as OCaml. It is hoped that each reader will find something new, or see an old thing in a new light. For the more casual reader, or those who are used to a different functional language, a summary of basic OCaml is provided at the front of the book.
 
If you buy the book, please do consider leaving a review on Amazon, which you can do even if you buy just the PDF eBook.
 


Reprinting of "OCaml from the Very Beginning"
ocaml-book — 2013-10-15T14:31:28-00:00
Owing to the print-on-demand process we're using, it's simple to update the book to fix the few errata which have been pointed out by kind readers since publication. The new text is now live on Amazon, the Kindle Store, and our website. 
To update the Kindle copy, go to "Manage My Kindle" on Amazon (there is no automatic update process). 
If you have purchased a PDF e-book, just email us for an updated PDF (it is possible to notify customers of an updated version by email, but since none of the errata are serious, we've opted not to do so). 
If you have a physical book, you can fix the errata yourself from the errata page on the website. In addition, if you have purchased the physical book, and would like a copy of the new PDF, please forward your amazon receipt to contact@coherentgraphics.co.uk 
And finally, if you have found mistakes in the book (or have any suggestions for the next edition), please do tell us, using the contact form on the website. 
 


OCaml  tutorial videos now available
ocaml-book — 2013-09-27T14:04:48-00:00
I've made a few little tutorial videos to go with the book. Just seven for now, covering the first four chapters, but more to come:
Evaluating an expression
True and false
Making decisions
Names and functions
Patterns
Lists
Take and drop
Appending two lists
They're not really a substitute for the book but, when learning, it's always helpful to go over things twice, particularly using a different medium.
The videos are available here. 
 


Now available on Amazon
ocaml-book — 2013-06-07T14:21:23-00:00
OCaml from the Very Beginning (204pp, paperback) is now available on Amazon for $39.99 / €29.99 / £24.99, as well as an e-book from ocaml-book.com for $19.99. Here are preview chapters and a .zip of the examples and exercises. 
Here's the blurb: 
In OCaml from the Very Beginning John Whitington takes a no-prerequisites approach to teaching a modern general-purpose programming language. Each small, self-contained chapter introduces a new topic, building until the reader can write quite substantial programs. There are plenty of questions and, crucially, worked answers and hints.
OCaml from the Very Beginning will appeal both to new programmers, and experienced programmers eager to explore functional languages such as OCaml. It is suitable both for formal use within an undergraduate or graduate curriculum, and for the interested amateur.
Please do review the book on Amazon if you have the chance.
 


ocamlcore
ocamlcore — 2023-07-14T05:47:25-00:00
As I have announced 2 years ago, I decided to move to a new company and to take new challenges. The website, www.ocamlcore.com, will soon be converted to a static website. As announced, this change will not affect at all the community services offered through ocamlcore.org (especially forge.ocamlcore.org), except that we will migrate to .ocaml.org DNS (soon) but we will keep the old DNS as well with some redirect.
+http://www.ocamlcore.com/wp/feed/?amp%3Blanguage=en&language=enocamlcore2023-07-14T05:47:25-00:00http://www.ocamlcore.com/wp/feed/?amp%3Blanguage=en&language=enocamlcoreAs I have announced 2 years ago, I decided to move to a new company and to take new challenges. The website, www.ocamlcore.com, will soon be converted to a static website. As announced, this change will not affect at all the community services offered through ocamlcore.org (especially forge.ocamlcore.org), except that we will migrate to .ocaml.org DNS (soon) but we will keep the old DNS as well with some redirect.
 I am very happy to have been able to run this company for 4 years. This was great and helps me to tackle various challenge in OCaml.
 Thanks to all the companies that supported me during this journey.


ocamljava
ocamlcore — 2023-07-13T05:47:28-00:00
4 weeks ago, on Friday April 15th, the annual OCaml Meeting took place at Telecom Paris. OCamlCore (the company), organized this meeting for the fourth time. The event was quite a successful day. There were 65 attendees and 12 talks.
diff --git a/data/planet/ocamljava.xml b/data/planet/ocamljava.xml
index c6939f818c..e468576fdb 100644
--- a/data/planet/ocamljava.xml
+++ b/data/planet/ocamljava.xml
@@ -1,5 +1,5 @@
 
-http://www.ocamljava.org/feed.xmlocamljava2023-07-13T05:47:28-00:00http://www.ocamljava.org/feed.xmlocamljavaThe third alpha version is finally available, fixing several bugs, and enhancing the Java library.
+http://www.ocamljava.org/feed.xmlocamljava2023-07-14T05:47:25-00:00http://www.ocamljava.org/feed.xmlocamljavaThe third alpha version is finally available, fixing several bugs, and enhancing the Java library.


Alpha2
ocamljava — 2014-09-22T21:00:00-00:00
The new release, namely 2.0-alpha2, is mainly a bugfix relase, fixing the following issues:


New alpha version
ocamljava — 2014-09-22T21:00:00-00:00
A new alpha version has been released, fixing several blocking / annoying bugs.


ocamllabs
ocamljava — 2023-07-13T05:47:28-00:00
The project is finally re-open-sourced. Well, at least partially: the code for the compiler and associated tools, as well as libraries is available. The code for the runtime support library is not yet, but will follow once cleaned. Put otherwise, the part of the project written in OCaml is fully available, while the part written in Java remains closed. As it is nevertheless quite a big step forward, the version has changed from early-access to alpha.
diff --git a/data/planet/ocamllabs.xml b/data/planet/ocamllabs.xml
index e835e88643..d27fbedbae 100644
--- a/data/planet/ocamllabs.xml
+++ b/data/planet/ocamllabs.xml
@@ -1,5 +1,5 @@
 
-http://ocamllabs.io/compiler-hacking/rss.xmlocamllabs2023-07-13T05:47:28-00:00http://ocamllabs.io/compiler-hacking/rss.xmlocamllabsOur next OCaml Compiler Hacking event will be on Thursday 7th December in The Thomas Gray Room at Pembroke College, Cambridge.
+http://ocamllabs.io/compiler-hacking/rss.xmlocamllabs2023-07-14T05:47:25-00:00http://ocamllabs.io/compiler-hacking/rss.xmlocamllabsOur next OCaml Compiler Hacking event will be on Thursday 7th December in The Thomas Gray Room at Pembroke College, Cambridge.
 
 If you're planning to come along, it'd be helpful if you could indicate interest via Doodle and sign up to the mailing list to receive updates.
 
diff --git a/data/planet/ocsigen.xml b/data/planet/ocsigen.xml
index 674108cf1f..9dffd34a9b 100644
--- a/data/planet/ocsigen.xml
+++ b/data/planet/ocsigen.xml
@@ -1,5 +1,5 @@
 
-https://ocsigen.org/feed.xmlocsigen2023-07-13T05:47:28-00:00https://ocsigen.org/feed.xmlocsigenNew release: Ocsigen Start 2.18
+https://ocsigen.org/feed.xmlocsigen2023-07-14T05:47:25-00:00https://ocsigen.org/feed.xmlocsigenNew release: Ocsigen Start 2.18
 
 Ocsigen Start is a template for client-server Web and/or mobile app in OCaml or ReasonML.
 It contains many standard features like user management, notifications, and many code examples.
diff --git a/data/planet/opa.xml b/data/planet/opa.xml
index f57d9c5d8d..313b879a4d 100644
--- a/data/planet/opa.xml
+++ b/data/planet/opa.xml
@@ -1,5 +1,5 @@
 
-http://www.blogger.com/feeds/2073503406800427577/posts/defaultopa2023-07-13T05:47:28-00:00http://www.blogger.com/feeds/2073503406800427577/posts/defaultopaAs many of you know, the main source to learn Opa is the O'Reilly book: Opa, Up and Running*.

+http://www.blogger.com/feeds/2073503406800427577/posts/defaultopa2023-07-14T05:47:25-00:00http://www.blogger.com/feeds/2073503406800427577/posts/defaultopaAs many of you know, the main source to learn Opa is the O'Reilly book: Opa, Up and Running*.

 

 One of the highlight of the book is a walk-through the development of Birdy, a micro-blogging platform, aka. mini-Twitter.

 

diff --git a/data/planet/orbitz.xml b/data/planet/orbitz.xml
index 6b762c7a85..a4a36cfc6e 100644
--- a/data/planet/orbitz.xml
+++ b/data/planet/orbitz.xml
@@ -1,5 +1,5 @@
 
-http://functional-orbitz.blogspot.com/feeds/posts/default/-/planetocaml?alt=rssorbitz2023-07-13T05:47:28-00:00http://functional-orbitz.blogspot.com/feeds/posts/default/-/planetocaml?alt=rssorbitz

+http://functional-orbitz.blogspot.com/feeds/posts/default/-/planetocaml?alt=rssorbitz2023-07-14T05:47:25-00:00http://functional-orbitz.blogspot.com/feeds/posts/default/-/planetocaml?alt=rssorbitz

 Note, this post is written against the 2.0.1 version of gen_server
 
 
diff --git a/data/planet/pdonadeo.xml b/data/planet/pdonadeo.xml
index 034bd3ba37..d420e31db0 100644
--- a/data/planet/pdonadeo.xml
+++ b/data/planet/pdonadeo.xml
@@ -1,5 +1,5 @@
 
-http://www.donadeo.net/facets/programming-languages/objective-caml/feed/pdonadeo2023-07-13T05:47:28-00:00http://www.donadeo.net/facets/programming-languages/objective-caml/feed/pdonadeo
+http://www.donadeo.net/facets/programming-languages/objective-caml/feed/pdonadeo2023-07-14T05:47:25-00:00http://www.donadeo.net/facets/programming-languages/objective-caml/feed/pdonadeo
 I'm in this situation: I need to compile an OCaml program on a CentOS 6.5 server. This is actually quite problematic, because CentOS 6.5 provide out of the box a very old OCaml 3.11.2, released by INRIA in January 2010, more than four years ago.
 
 No problem, OPAM come to the rescue! No. The OPAM team doesn't provide a binary executable compatible with the (actually very old) system libraries present in CentOS.
diff --git a/data/planet/psellos.xml b/data/planet/psellos.xml
index d095a09f78..a9beb27853 100644
--- a/data/planet/psellos.xml
+++ b/data/planet/psellos.xml
@@ -1,5 +1,5 @@
 
-http://psellos.com/atom.xmlpsellos2023-07-13T05:47:28-00:00http://psellos.com/atom.xmlpsellos
August 26, 2020
+http://psellos.com/atom.xmlpsellos2023-07-14T05:47:25-00:00http://psellos.com/atom.xmlpsellosAugust 26, 2020
 
 Since last year I’ve been using BuckleScript to build web apps in OCaml
 and have been enjoying it very much. However, when I went to the
diff --git a/data/planet/reynard.xml b/data/planet/reynard.xml
index 2df0c816a3..bdd159cd21 100644
--- a/data/planet/reynard.xml
+++ b/data/planet/reynard.xml
@@ -1,5 +1,5 @@
 
-http://reynard.io/feed.xmlreynard2023-07-13T05:47:28-00:00http://reynard.io/feed.xmlreynard
Thanks to everyone who joined us for OCaml Compiler Hacking in the (incredibly warm!) Old Library at Pembroke College this week.
+http://reynard.io/feed.xmlreynard2023-07-14T05:47:24-00:00http://reynard.io/feed.xmlreynardThanks to everyone who joined us for OCaml Compiler Hacking in the (incredibly warm!) Old Library at Pembroke College this week.
 
 Prior to the event, we attempted a temporary update of the “Things to Work On” wiki page, and added some projects needing attention (thank you to Mark Shinwell, David Allsopp and Anil Madhavapeddy). We will go through the list in more detail this month and update the remaining items. If you have any suggestions for compiler projects, please join the discussion on the mailing list, or in the OCaml Discourse Forum.
 
diff --git a/data/planet/rgrinberg.xml b/data/planet/rgrinberg.xml
index 42b55f5c86..dbd608fcf8 100644
--- a/data/planet/rgrinberg.xml
+++ b/data/planet/rgrinberg.xml
@@ -1,5 +1,5 @@
 
-http://rgrinberg.com/blog/atom.xmlrgrinberg2023-07-13T05:47:28-00:00http://rgrinberg.com/blog/atom.xmlrgrinbergThere’s been some recent grumbling about the usability of ppx in OCaml, and
+http://rgrinberg.com/blog/atom.xmlrgrinberg2023-07-14T05:47:25-00:00http://rgrinberg.com/blog/atom.xmlrgrinberg
There’s been some recent grumbling about the usability of ppx in OCaml, and
 instead of just letting it slide again, I’ve decided to do something
 constructive about this and write a little tutorial on how to write a
 deriving plugin.
diff --git a/data/planet/rjones.xml b/data/planet/rjones.xml
index cfa8f9d720..acc2219dac 100644
--- a/data/planet/rjones.xml
+++ b/data/planet/rjones.xml
@@ -1,5 +1,5 @@
 
-https://rwmj.wordpress.com/tag/ocaml/feed/rjones2023-07-13T05:47:28-00:00https://rwmj.wordpress.com/tag/ocaml/feed/rjoneshttps://github.com/ocaml/ocaml/commit/8f3833c4d0ef656c826359f4137c1eb3d46ea0ef
+https://rwmj.wordpress.com/tag/ocaml/feed/rjones2023-07-14T05:47:25-00:00https://rwmj.wordpress.com/tag/ocaml/feed/rjoneshttps://github.com/ocaml/ocaml/commit/8f3833c4d0ef656c826359f4137c1eb3d46ea0ef
 We’ve been using this patch in Fedora since Nov 2016.


sfletcher
rjones — 2023-07-13T05:47:28-00:00
libguestfs is a C library for creating and editing disk images.  In the most common (but not the only) configuration, it uses KVM to sandbox access to disk images.  The C library talks to a separate daemon running inside a KVM appliance, as in this Unicode-art diagram taken from the fine manual:
 diff --git a/data/planet/sfletcher.xml b/data/planet/sfletcher.xml
index f4504209f4..31695acd65 100644
--- a/data/planet/sfletcher.xml
+++ b/data/planet/sfletcher.xml
@@ -1,5 +1,5 @@
 
-http://blog.shaynefletcher.org/feeds/posts/default/-/OCamlsfletcher2023-07-13T05:47:28-00:00http://blog.shaynefletcher.org/feeds/posts/default/-/OCamlsfletcher
+http://blog.shaynefletcher.org/feeds/posts/default/-/OCamlsfletcher2023-07-14T05:47:25-00:00http://blog.shaynefletcher.org/feeds/posts/default/-/OCamlsfletcher
 
 
 Bucket Sort
diff --git a/data/planet/signalsandthreads.xml b/data/planet/signalsandthreads.xml
index 576ebd4ff6..31771ba8b3 100644
--- a/data/planet/signalsandthreads.xml
+++ b/data/planet/signalsandthreads.xml
@@ -1,5 +1,5 @@
 
-https://feeds.simplecast.com/L9810DOasignalsandthreads2023-07-13T05:47:28-00:00https://feeds.simplecast.com/L9810DOasignalsandthreadsRichard Eisenberg is one of the core maintainers of Haskell. He recently joined Jane Street’s Tools and Compilers team, where he hacks on the OCaml compiler. He and Ron discuss the powerful language feature that got him into PL design in the first place—dependent types—and its role in a world where AIs can (somewhat) competently write your code for you. They also discuss the differences between Haskell and OCaml; the perils of trying to make a language that works for everybody; and how best a company like Jane Street can collaborate with the open source community.
You can find the transcript for this episode  on our website.
Some links to topics that came up in the discussion:
Dependent types
GHC
Unboxed types in OCaml
Language extensions in Haskell
+https://feeds.simplecast.com/L9810DOasignalsandthreads2023-07-14T05:47:25-00:00https://feeds.simplecast.com/L9810DOasignalsandthreadsRichard Eisenberg is one of the core maintainers of Haskell. He recently joined Jane Street’s Tools and Compilers team, where he hacks on the OCaml compiler. He and Ron discuss the powerful language feature that got him into PL design in the first place—dependent types—and its role in a world where AIs can (somewhat) competently write your code for you. They also discuss the differences between Haskell and OCaml; the perils of trying to make a language that works for everybody; and how best a company like Jane Street can collaborate with the open source community.
You can find the transcript for this episode  on our website.
Some links to topics that came up in the discussion:
Dependent types
GHC
Unboxed types in OCaml
Language extensions in Haskell


Swapping the Engine Out of a Moving Race Car with Ella Ehrlich
signalsandthreads — 2022-09-12T20:46:54-00:00
Ella Ehrlich has been a developer at Jane Street for close to a decade. During much of that time, she’s worked on Gord, one of Jane Street’s oldest and most critical systems, which is responsible for normalizing and distributing the firm’s trading data. Ella and Ron talk about how to grow and modernize a legacy system without compromising uptime, why game developers are the “musicians of software,” and some of the work Jane Street has done to try to hire a more diverse set of software engineers.
You can find the transcript for this episode  on our website.
Some links to topics that came up in the discussion:
EG, The League of Legends team that Ella is a huge fan of.
Apache Kafka, the message bus that Gord migrated to.
Some of the various sources of symbology you have to deal with when normalizing trading data. (Really, there are too many sources to list here!)
A list of Jane Street’s recruiting Programs and Events, including INSIGHT, which focuses on women, and IN FOCUS, which focuses on historically underrepresented ethnic or racial minorities.


State Machine Replication, and Why You Should Care with Doug Patti
signalsandthreads — 2022-04-20T16:00:00-00:00
Doug Patti is a developer in Jane Street’s Client-Facing Tech team, where he works on a system called Concord that undergirds Jane Street’s client offerings. In this episode, Doug and Ron discuss how Concord, which has state-machine replication as its core abstraction, helps Jane Street achieve the reliability, scalability, and speed that the client business demands. They’ll also discuss Doug’s involvement in building a successor system called Aria, which is designed to deliver those same benefits to a much wider audience.
You can find the transcript for this episode  on our website.
Some links to topics that came up in the discussion:
Jane Street’s client-facing trading platforms
A Signals and Threads episode on market data and multicast which discusses some of the history of state-machine replication in the markets.
The FIX protocol
UDP multicast
Reliable multicast
Kafka


skjegstad
signalsandthreads — 2023-07-13T05:47:28-00:00
Stephen Dolan works on Jane Street’s Tools and Compilers team where he focuses on the OCaml compiler. In this episode, Stephen and Ron take a trip down memory lane, discussing how to manage computer memory efficiently and safely. They consider trade-offs between reference counting and garbage collection, the surprising gains achieved by prefetching, and how new language features like local allocation and unboxed types could give OCaml users more control over their memory.
You can find the transcript for this episode  on our website.
Some links to topics that came up in the discussion:
Stephen’s command-line JSON processor, jq
Stephen’s Cambridge dissertation, “Algebraic Subtyping”, and a protoype implementation of mlsub, a language based on those ideas.
A post from Stephen on how to benchmark different memory allocators.
A Jane Street tech talk on “Unboxed Types for OCaml”, and an RFC in the OCaml RFC repo.
A paper from Stephen and KC Sivaramakrishnan called “Bounding Data Races in Space and Time”, which is all about a new and better memory model for Multicore OCaml.
Another paper describing the design of OCaml’s multicore GC.
The Rust RFC for Higher-ranked trait bounds.
diff --git a/data/planet/skjegstad.xml b/data/planet/skjegstad.xml
index 67917c4e61..ae29e74f00 100644
--- a/data/planet/skjegstad.xml
+++ b/data/planet/skjegstad.xml
@@ -1,5 +1,5 @@
 
-http://www.skjegstad.com/feeds/ocaml.tag.atom.xmlskjegstad2023-07-13T05:47:28-00:00http://www.skjegstad.com/feeds/ocaml.tag.atom.xmlskjegstadJitsu - or Just-in-Time Summoning of Unikernels - is a prototype DNS server that can boot virtual machines on demand. When Jitsu receives a DNS query, a virtual machine is booted automatically before the query response is sent back to the client. If the virtual machine is a unikernel, it can boot in milliseconds and be available as soon as the client receives the response. To the client it will look like it was on the whole time. 
+http://www.skjegstad.com/feeds/ocaml.tag.atom.xmlskjegstad2023-07-14T05:47:25-00:00http://www.skjegstad.com/feeds/ocaml.tag.atom.xmlskjegstadJitsu - or Just-in-Time Summoning of Unikernels - is a prototype DNS server that can boot virtual machines on demand. When Jitsu receives a DNS query, a virtual machine is booted automatically before the query response is sent back to the client. If the virtual machine is a unikernel, it can boot in milliseconds and be available as soon as the client receives the response. To the client it will look like it was on the whole time. 
 Jitsu can be used to run microservices that only exist after they have been resolved in DNS - and perhaps in the future can facilitate demand-driven clouds or extreme scaling with a unikernel per URL. Jitsu has also been used to boot unikernels in milliseconds on ARM devices.
 A new version of Jitsu was just released and I'll summarize some of the old and new features here. This is the first version that supports both MirageOS and Rumprun unikernels and uses the distributed Irmin database to store state. A full list of changes is available here.
 
diff --git a/data/planet/smondet.xml b/data/planet/smondet.xml
index d055e03919..27ccdab19e 100644
--- a/data/planet/smondet.xml
+++ b/data/planet/smondet.xml
@@ -1,5 +1,5 @@
 
-https://seb.mondet.org/b/OCaml.rsssmondet2023-07-13T05:47:28-00:00https://seb.mondet.org/b/OCaml.rsssmondet
+https://seb.mondet.org/b/OCaml.rsssmondet2023-07-14T05:47:25-00:00https://seb.mondet.org/b/OCaml.rsssmondet
     TZComet and CleanNFT Metadata: a generic token-viewer ☄
     Also new domain tzcomet.io 🌠