-
Notifications
You must be signed in to change notification settings - Fork 0
Home
Weald is an experiment in building a Trac-like project management tool that incorporates CI/CD and leverages tools and practices commonly reserved for production environments.
When Agile development practices were introduced, almost 25 years ago, developers began to take responsibility for many activities that used to belong solely to others, from project managers, to QA departments, to network operations. Many modern tools look like they are aware of these changes, but the reality of using them is that coordination of data is still mostly a human activity.
These tools get more sophisticated over time, but in many ways they do not exceed the subtle power of simpler tools, like Trac. When I was forced to use Trac on a project, my first impression was that it was too simple to be useful. But for the most part, it did the job of connecting the data.
In the meantime, many features that CI/CD tools have implemented over the years in a 'good enough' (arguable) fashion now exist as mature, standalone libraries. If someone were writing a new project management tool from scratch, then the second thing they might want to think about is what they aren't going to implement, instead of what they are. What can we avoid, so that we can put more energy into the things of value?
The core competency of Trac in its early days was automatically wiring tickets, commits, and wiki pages together. It had only a very rudimentary concept of project management, but it was enough to do a respectable job of managing a Kanban project. Sometimes, it's not about what a tool does, but what it doesn't prevent.
Unfortunately it had very little notion of a release lifecycle, and no sense at all about CI/CD, so as useful as the connections were among everything else, having no connection to build, release, or deployment life cycles becomes problematic as a project grows. So a tool that thinks like Trac but works like modern development practices could have some significant potential.
The first Wiki I used in depth had a feature called backlinks. At the bottom of every page, you had a list of every page that linked to this one. It seemed like an odd feature, and I don't fault other Wikis for not implementing it, but I do miss it quite a bit. There are several reasons I prefer Wikis with bidirectional links.
Without careful stewardship, wikis tend to rot, and we don't always make the connections that a new or returning team member needs in order to understand how something works. With Backlinks, you get a second chance at finding related pages, because if at least one of the two pages acknowledges that it relates to another, the connection is there.
But the other power I discovered with this is that people make assumptions about things or other people, and they document it in the spot that is interesting to them, but nobody will find until it's too late. One of the other failure modes of projects is when you think someone is going do something for you before it becomes a blocking issue, and then when you go ask them about it you find out that they haven't started because they don't recall agreeing to any such thing, so not only is the solution not ready, but it might evaporate entirely in the ensuing conversation. That's a lot of bad news all at once.
What I found poking at Backlinks was people commenting that some person or tool would be fixing a problem that was never going to happen. Having these sorts of conversations early saved a lot of nasty surprises. And when someone left or got promoted, it was a fairly straightforward process to walk through the Wiki figuring out what duties of theirs needed to be distributed to other people or renegotiated. Similar for dysfunctional or expensive tools that were candidates for replacement. Which pages talk about it?
Most tools, and frankly most developers are so focused on the Happy Path with build automation that they ignore the ways in which they don't scale, often at their peril.
Often the very first task a developer needs to perform on a new job is to figure out how to build the code, and when most of that information is buried in the build tool, that becomes challenging for self-directed learning. I believe that tools like CircleCI and Travis exist in part to answer this concern. Contrast this to Bamboo, which not only tries to implement your whole build pipeline for you, out of band from your code (which often changes or is changed by how the code builds), but doubles down by introducing layers of information hiding so that not only can't you see most builds in the system, but you don't know what functionality Bamboo has because all of those buttons are hidden away. Information Hiding is not what developer tools should be doing. That's an institutionalized form of gatekeeping, and it makes Bamboo by far the worst CI/CD tool on the marked.
We want code that we deploy to run with as few differences between a local or pre-prod system and the production one. Why then do so many of us not demand the same from our build systems? Why should it build differently if it is meant to run the same?
If I'm not a very senior developer, the moment I'm facing a red build, all of my plans for the day are potentially out the window. This is very stressful. Not having all of the tribal knowledge for the project in my head, I'm now contending with a black box. This is a sure recipe for Learned Helplessness, which is one of the biggest enemies of scaling human systems.
When I can run things locally, I at least can step through the code and try to figure out what is going on, and if it's still too confusing, I can at least look for code smells and what is different between my build and the last green one. It is better if everything right up until the moment of publishing artifacts can be run locally, by as many people as who want to.
Running things locally means that nearly every step of the build process needs to be able to run outside of the build tool. For someone writing a CI/CD tool, this is fortunate because it means that your project roadmap is not dominated by implementing integrations to a million tools.
Logs are a game of telephone. You started with structured data, you recorded it to a human readable format, and now someone needs to turn that back into information. For some organizations that's a specialized human task (again, not scalable) or the domain of expensive software for log analysis. If you don't make it your business to integrate with every other tool in existence, then the amount of data under your control is not that onerous. Keeping the data instead of dumping it into the logs is not that difficult.
And for anything where trends matter, stats are a better data format, and these days all of the software for collecting and reporting this data is already mature. Using off the shelf components allows you to go a lot deeper on analyzing build telemetry data for trends. Everything from build times to test results have value when expressed as a graph over time. In some CI tools, the telemetry about the builds outlives the build logs by orders of magnitude. These vignettes into old data still can have a lot of value, even if you didn't notice it at the time.
As one concrete example, some performance regressions are not immediately evident when deployed to production, and are only detected at a later date, when it becomes difficult to reason about what change(s) may have been the culprit. Quite frequently, these changes show up as changes to the behavior of the build, and while you might not be able to find the exact build where the problem started, you might be able to narrow it to an afternoon, eliminating >90% of the suspects. And while not a day-to-day quality of life improvement, these sorts of experiences can show the real value of better tools.