diff --git a/.custom_wordlist.txt b/.custom_wordlist.txt index 519c850..db46323 100644 --- a/.custom_wordlist.txt +++ b/.custom_wordlist.txt @@ -8,6 +8,7 @@ appserver appservers artifact artifacts +aspirational attrgetter auditable auth @@ -17,17 +18,25 @@ authorizes autocert backend backends +Becca Blazingly bool boolean breakpoint browserconfig +bugtracker +BugWatch +BugWatches +bugzilla buildbot buildd bzr bzr's +centralized +Centralized cgroup cgroups +changeset charmcraft CMDLINE codebase @@ -44,6 +53,8 @@ cryptographic cryptographically customize CVE +Dalia +Dalia's DatabaseSetup ddl defense @@ -52,6 +63,8 @@ deps dev directDelivery DirectDelivery +distroseries +DistroSeries distutils DNS docstring @@ -62,8 +75,12 @@ doctest doctests docutils dogfood +downstreams dtrt el +else's +emphasizes +emphazising epydoc executables facetted @@ -90,6 +107,7 @@ integrations ip IPv IPython +javascript jenkaas jenkins jinja @@ -113,10 +131,14 @@ mbox mboxMailer memcache minimize +mockups mojo mozilla +natively OAuth +OEM ols +OOPSes OpenID opstats optimization @@ -124,6 +146,7 @@ optimize optimizing optstats organize +organizing orm OWASP OWASP's @@ -135,9 +158,11 @@ pgsql pgSQL pipx plaintext +po pooler PostgreSQL PostGreSQL +POSTs PPA PQM pre @@ -155,10 +180,13 @@ quickref quickstart rctp realfavicongenerator +realizes repo +RESTful rocketfuel rollout rollouts +rosetta RPC rst rsync @@ -176,6 +204,7 @@ SMTPMailer snapshotting sourcecode sourceforge +sourcepackage sourcetree soyuz specialized @@ -185,6 +214,7 @@ SREs SSO standardized stdin +steve stg stubmailer stubMailer @@ -194,6 +224,8 @@ sudo summarized svg symlinked +synchronize +synchronizing syntax syntaxes systemd @@ -210,16 +242,22 @@ TLS tradeoff tradeoffs Transactional +traversers tsearch ubuntu Ubuntu UCT UltimateVimPythonSetup +unauthorized +Uncomment +untriaged untrusted +upstreams vbuilder virtualenv virtualenvs VPN +webdav webhook webservice webservice's diff --git a/.wokeignore b/.wokeignore index c64a603..c8282f2 100644 --- a/.wokeignore +++ b/.wokeignore @@ -2,3 +2,10 @@ # cannot avoid for now, ie # https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html doc-cheat-sheet* +# Currently, it is unclear how to properly ignore bad words in +# ReStructuredText, see https://github.com/get-woke/woke/discussions/275 +# Completely ignoring the following files for now. +# This needs to be revised once we know a good way to go forward. +charms.rst +contributing-changes.rst +running.rst diff --git a/custom_conf.py b/custom_conf.py index ec6f6f0..4280bcf 100644 --- a/custom_conf.py +++ b/custom_conf.py @@ -117,6 +117,7 @@ 'https://bazaar.staging.launchpad.net', # broken, unclear why XXX 2023-10-14: jugmac00- check with team 'https://bugs.launchpad.net/charm-launchpad-buildd-image-modifier', # private 'https://bugs.launchpad.net/launchpad-vbuilder-manage', # private + 'http://example.com/.*', # ignore, example links 'https://launchpad.net/canonical-mojo-specs', # private r'https://launchpad\.test.*', # ignore, local test setup 'https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadRollout#Config-only_Rollouts', # private diff --git a/explanation/index.rst b/explanation/index.rst index 5b7bd5c..5a78a8d 100644 --- a/explanation/index.rst +++ b/explanation/index.rst @@ -15,3 +15,8 @@ Explanation codeimport performance security + malone + scope + strategy + values + webapp-process diff --git a/explanation/malone.rst b/explanation/malone.rst new file mode 100644 index 0000000..2348ce0 --- /dev/null +++ b/explanation/malone.rst @@ -0,0 +1,133 @@ +============ +About Malone +============ + +.. include:: ../includes/attention_out_of_date.rst + +The world has many excellent bug tracking tools already. It would not make +sense to create another bugtracker unless the vision behind that software +was substantially different to anything that had gone before it. This +document outlines that vision, explaining what it is that I hope Malone will +do for the open source community. + +The Vision behind Malone +======================== + +Malone is a unified bug tracker for the entire open source world. It is +designed to allow the whole open source community to collaborate on software +defect management, especially when a single piece of code is being used +across many projects. Malone presents a single page which gathers together +the combined wisdom and knowledge of the open source world regarding a +specific software defect. + +Upstream and Distributions +========================== + +A unique feature of Malone is that it understands the structure of the open +source community:: + + Software is developed by individuals or groups with a common interest in a + specific problem. We call this group "upstream". That software is + distributed in its pristine state ("tarballs", usually) and is usually + designed to be compiled and run on a variety of platforms. + + However, most people who use that software will not get it directly from + upstream, build it and install it locally. They will install a package + that has already been prepared for the specific platform they are running + on. For example, on Gentoo, they will type "emerge foo". On Ubuntu, they + would type "apt install foo". And on RedHat they would install a + custom RPM. So the same software code is being repackaged many times, for + Gentoo, Ubuntu, RedHat, and many other platforms. + +A natural consequence of this repackaging is that a bug in that software +might be detected and/or fixed by a variety of different people, without +upstream being aware of either the bug or the fix. In many cases, the people +doing the repackaging have entirely separate bug tracking tools to upstream, +and it is difficult for them to pass their information and patches to +upstream directly. + +Malone explicitly tracks the status of a bug both upstream and in any +distributions registered in Launchpad. This makes it possible, for +example, to see immediately if a fix has been found for a given bug by any +of the participating distributions, or upstream. The bug page shows this +information very prominently. + +Watches +======= + +It's unlikely that the whole world will shift to Malone. Many larger +projects have their own bug tracking tools (Bugzilla, Sourceforge and +Roundup are commonly used) and some have even created custom tools for this +purpose. For that reason, Malone supports BugWatches. A BugWatch is a +reference to a different bugtracker that is tracking the same bug. Of course +it will have a different bug number in that system, and the integration +between Malone and that remote bug system is possibly limited, compared to +the richness of the Malone data model, but this still allows us to keep +track of a bug in a different bug tracker. For example, a bug in the Firefox +package on Ubuntu would be tracked in Malone. If the same bug has been +identified upstream, it would be recorded in bugzilla.mozilla.org, and we +would create a BugWatch in the Malone bug pointing at that upstream bug. + +Email Integration +================= + +It's important that Malone be usable entirely in email. Many open source +developers use their email to track work that needs to be done. So all of +Malone's features should be accessible via email, including changing the +status of a bug, adding and updating watches, and possibly also requesting +reports of bugs on a product or distribution. + +Distribution Bugs +================= + +Malone is designed to track bugs upstream, and in distributions. The +requirements for a distribution bugtracker are somewhat specialised. A +distribution consists of many source packages and binary packages, and it +must be possible to track bugs at a fine level of granularity such as at the +source/binary package level. + +Malone allows us to create bugs that belong only to a distribution, or to a +sourcepackage in a distribution if we have that information. Bugs that are +not associated with a sourcepackage can be thought of as "untriaged" bugs. +In some cases, we should be able to know not only which source package, but +also the precise binary package that manifests the bug. + +Milestones and DistroSeries +=========================== + +In addition, it's important to be able to know which bugs need to be fix for +a given release of the distribution, or a given milestone upstream. Malone +allows us to specify a milestone or a distroseries by which a bug needs to +be fixed, which allows QA teams to keep track of the progress they are +making towards a release. + +Version Tracking +================ + +One very difficult problem faced by support teams in the open source world +is that users may not all be running the latest version of a piece of code. +In fact, that's pretty much guaranteed. So Malone needs to be able to say +whether a bug is found in a particular version of a package or not. + +Future +====== + +Bazaar Integration +------------------ + +Malone is part of Launchpad, a web based portal for open source +developers. Another component of that portal is the Bazaar, a repository of +data and metadata about code stored in the Bazaar revision control system. We +hope that Bazaar will be embraced by the open source world, as it solves a +number of problems with traditional centralised revision control systems and +is again designed to support distributed disconnected operation. + +Once more people start keeping their code in Bazaar, it should become possible +to streamline the cooperation process even further. For example, if the fix +for a particular Malone bug can be found in a Bazaar changeset, then it should +be possible for upstream and other distributions to merge in that fix to their +codebase automatically and easily. The integration could even be +bidirectional - once a fix had been merged in, Bazaar could possibly detect +that and mark the bug fixed in that codebase automatically. + + diff --git a/explanation/scope.rst b/explanation/scope.rst new file mode 100644 index 0000000..cdc105c --- /dev/null +++ b/explanation/scope.rst @@ -0,0 +1,341 @@ +================== +What is Launchpad? +================== + +Launchpad is a complete system for gathering changes from different types of +sources and collaboratively organizing them into packaged software for the end +user, delivered as part of an operating system that can be downloaded or that +comes already installed on purchased hardware. + +If you start by thinking of Launchpad as a traditional software “forge” - a +web service that provides bug tracking, code hosting and other related +services - then you are not too far off understanding what Launchpad is. +However, Launchpad has many distinctive traits that combine to put it into an +entirely different category of software. + +But at its core, it is best to think of Launchpad as a service that meshes +together two important networks: + +1. Networks of people making software +2. The network of dependencies between software + +But first, a story. + + +The Story of a Bug +================== + +*Arnold* writes software for a living, and he runs Ubuntu on his desktop. He +wishes he could contribute to open source, but he doesn't have much spare +time, and when he gets home from his job the last thing he wants to do is +program. However, the spirit of willingness is there. + +One day, Arnold notices that Tomboy loses his formatting if he alt-tabs at the +wrong time. Arnold knows that a well-filed bug report is a thing of beauty to +most programmers, so he decides to spend a few moments to file a bug against +Tomboy. + +Rather than search the net to find where Tomboy tracks its bugs, Arnold uses +Ubuntu's built-in bug filing mechanism. It asks him a bunch of questions, +invites Arnold to write his bug report and then files the bug on Launchpad +against the Tomboy package in Ubuntu. + +*Becca* has been dabbling in Ubuntu contribution for a while, mostly by +helping new users solve their problems or turn them into good bug reports. She +notices Arnold's bug, sees that it's well written and thinks that it would be +easy enough to test against trunk. She opens up the Tomboy source package in +Ubuntu and sees that there is a known-good daily build of Tomboy's trunk +hosted in a trusted user archive. Becca installs Tomboy from this archive and +tests to see if the bug is still there in the latest version of the code. It +is. Becca sees this, opens up the original bug report and clicks a button to +forward the bug to the upstream bug tracker. + +*Carlos* is one of the Tomboy developers. He sees the bug in the tracker, sees +that it has been tested against trunk, realizes that it's an annoying bug +that's easy to fix and decides to fix it. He does the fix, applies it to the +Tomboy trunk and marks the bug as fixed. + +At this point, both Arnold and Becca are notified that the bug is fixed in +Tomboy trunk, and that they can try a version of Tomboy that has the fix by +using the known-good daily build archive for Tomboy. They are warned that this +is dangerous and may cause data loss, but they are also told how they can try +the bug fix for free using a cloud-based Ubuntu desktop. They both try the +bug, see that it's fixed, and are happy, albeit a little impatient for the fix +to be actually released and part of stock Ubuntu. + +Meanwhile, *Dalia* is an Ubuntu developer who takes a special interest in +desktop productivity applications like Tomboy. She checks on the Ubuntu source +package for Tomboy from time to time. The last time she checked, she saw that +quite a few bugs have been fixed in trunk but not yet released. Since she +knows the Tomboy release manager from long hours of IRC chat, she contacts him +and gently suggests that he do a release. + +*Edmund*, the Tomboy release manager, takes Dalia's hint well and realizes +that a release is way overdue. He makes a release of Tomboy following his +normal procedure. + +Launchpad detects that Tomboy has a new, official release and alerts +interested distribution maintainers that the release has been made and now +would be a good time to package a new version. Dalia packages up a new +version, requests that an Ubuntu core developer sponsor the change and then +waits for the new version to be uploaded. Dalia also uploads the fixed version +to one of her personal archives so that others can easily get it without +waiting for the next release of Ubuntu. + +*Fiona* the Ubuntu core developer sees Dalia's patch in the sponsorship queue +on Launchpad, notes that it's all good and then uploads the patch to the +official Ubuntu archive. (Fiona might also choose to upload the patch to +Debian). + +Launchpad sees that this upload fixes a number of bugs, including the one +originally filed by Arnold, and automatically includes those bugs in the list +of bugs that will be fixed by the next release of Ubuntu. + +Two months later, the next release of Ubuntu is actually released. Arnold +upgrades on release day, and tries out Tomboy to see if his bug was really, +actually fixed. It is, and all is right with the world. + + + + +Distinctive traits +================== + +Launchpad is different from other "forges" in a few important ways: + + +Cross-project collaboration +--------------------------- + +No project lives in isolation. Each project is part of an ecosystem of +software. Projects must be able to interact with each other, share bugs, +teams, goals and code with each other. + +.. image:: ../images/cross-project-collab.svg + +Launchpad takes every chance it gets to show the connections between projects +and to bring the opportunities created by those connections to light. + +By encompassing the entire process, all the way to operating system delivery, +Launchpad can provide a unique service: enable each contributor to focus on +the work they care about, while giving them an ambient awareness of how their +work fits into a larger picture, and providing a path by which they can +participate in other parts of that picture when they feel the need. + + +Front-end to open source +------------------------ + +Launchpad aims to be a front-end to open source. Whether or not a project +chooses to host on Launchpad, opportunistic developers can use Launchpad to +navigate bugs, get code and send patches. Likewise, we aim to present a +uniform interface to the projects we have. + + +Centralized service +------------------- + +Because Launchpad emphasizes cross-project collaboration, and because +Launchpad aims to be a front-end to all of open source, it necessarily has to +be a centralized service rather than a product that users deploy on their own +servers. + + +Networks of collaborators +------------------------- + +Launchpad understands that much of the human interaction around open source is +not primarily social, but rather collaborative: many people working together +in different ways toward the same goals. + +As such, Launchpad highlights actions and opportunities rather than +conversations and status. It answers questions like, “what can I do for you?”, +“who could help me do this?”, “who is waiting on me in order to get their +thing done?”, “can I rely on the advice offered by this person?” and so forth. + + +Distributions are projects too +------------------------------ + +Launchpad hosts Linux distributions in much the same way as it hosts projects, +allowing for developers to feel at home when interacting with distributions. + + +Gated development +----------------- + +Sometimes, secrets are necessary. Launchpad understands that sometimes +development needs to be done privately, and the results only later shared with +the world. Security fixes, OEM development for new hardware, proprietary +services with open source clients are all examples of these. + + +Hardware matters +---------------- + +Many software developers like to pretend that hardware does not really +exist. When people distribute software as part of an operating system, they +don't have the luxury of forgetting. Launchpad understands that developers +often need to acknowledge and work around differences thrown up by hardware. + + +We don't care if you use Launchpad, sort of +------------------------------------------- + +Many other forges define their success by how many users they have. Although +we love our users and welcome every new user, Launchpad does not judge its +success by the number of users. If one project wishes to host its development +on another platform, Launchpad acts as a front-end to that platform. + + +One project, many communities +----------------------------- + +Any given project can have many distinct communities interested in it. These +communities have different interests and different motivations, but all work +in the same project space so that they can easily benefit from each others' +efforts. + + +Scope +===== + +Launchpad has many major components. These can be broken up into four major +areas of functionality: + +1. where work is done; developers interact with other developers +2. where plans are made and reviewed; expert users interact with expert users + and developers +3. where projects engage with their communities; developers interact with end + users and other developers, and vice-versa +4. major supporting features + +.. image:: ../images/scope.svg + +Work +---- + +At the core of every software project is the actual code that makes up that +project. Here "code" is a broad term that also includes the project's +documentation, the translatable and translated strings that make up its user +interface, the packaging and integration scripts required to get the software +installed on end user's systems and so forth. + +Launchpad is built to be able to take contributions from anybody, regardless +of how involved they are in a project. For packages, translations and code +proper we provide mechanisms to allow people to review changes from others and +then merge them into the official parts of the project. + +Launchpad pulls in changes that happen in the upstreams and downstreams of a +project, whether those changes are patches to code, new translations or +packaging updates. It makes contributors to a project aware of the work that's +going on upstream and downstream and helps them take advantage of that work. + +And, of course, all work is for nothing if it does not get to the people who +might want to actually use its results. As such, project maintainers can +publish released versions of their code, any contributor can publish Ubuntu +packages to unofficial archives or even set up Launchpad to automatically +build and publish packages of latest snapshots of code. + + +Plans +----- + +People who are interested in doing something great will need to coordinate +their work, keep track of the defects in the things they have already done and +describe the things that they aren't doing yet but wish they could. + +Every software product in the world has bugs. For some projects, the rate of +incoming bugs is fairly low, and each bug can expect to receive some attention +from a core developer. For other projects, the rate of new bugs filed is so +high that the core development team can never hope to keep up with it. +Launchpad supports both kinds of projects. + +If every software product has bugs, every software user has great ideas about +how a product can be improved. Project maintainers need to get at these ideas, +evaluate them, and develop them into workable concepts. + +Often, a problem is so tricky that those concerned need to have a detailed, +managed discussion about what exactly the problem is. At other times, the +problem is easy enough to define, but there are so many solutions with +difficult trade-offs or difficult implementations that it is better to talk +about them and plan them out before proceeding with any of them. Launchpad +acknowledges that this can happen on any project, and that becoming clear on a +problem or becoming clear on the “best” solution can be helped a great deal +using tools. + +Crucially, all of these different types of "plans" - bugs, specifications, +blueprints, ideas - can span more than one code base and more than one +conceptual project. These plans need to be drafted, discussed, clarified and +reviewed before work starts, monitored, evaluated and changed as work +progresses, and then the results of that work need to be checked against the +plan when the work is finished. + + +Community +--------- + +Not everything that's done on a project is work toward a particular outcome, +or plans for how to get there. Every project needs to have some things that +are more general and stable. + +Projects need to be able to present themselves to the world, confident in +their identity and communicating exactly what they are about. Project +maintainers need to be able to announce important news, such as releases, +license changes or new practices. Contributors need to get a sense of who is +working on which parts of the project. Users need to be able to ask questions, +get support and give feedback. + +Contributors also need to share documentation about the project and how the +project runs. They need to be able to discuss general topics about the +project. + +Launchpad supports all of these things, and also makes it clear how any +project fits into the broader ecosystem of projects. It shows which projects +are upstreams or downstreams, which projects are affiliated with other +projects, which projects share contributors with other projects and so forth. + + +Supporting features +------------------- + +Launchpad has many major areas of functionality that are best considered as +“supporting features”: APIs, migration services, privacy, the mail UI, +synchronizing with external systems. + + +New World +========= + +When Launchpad is really doing all of these things and doing them well, the +world of open source software will be significantly changed. + +Patches will no longer lie decaying in someone else's bug tracker, waiting to +be noticed. Instead, they will all be synced into a central code review system +and queued for review and approval. + +Instead of a distribution tracking one set of bugs and upstream projects +tracking their own set of sometimes duplicated bugs, both upstream and +downstream developers can seamlessly access both sets of bugs. + + +Glossary +======== + +Upstream + A software project itself, as opposed to the packaged version of a software + project that is included in a distribution. Note, can also be used as a + relative term, e.g. “Debian is upstream of Ubuntu”. + +Downstream + The opposite of an upstream. Can be used to refer either to a packaged + version of a specific software project, or the entire distribution where + that package occurs. + + +References +========== + +* :doc:`strategy` +* :doc:`values` +* `Feature checklist `_ diff --git a/explanation/strategy.rst b/explanation/strategy.rst new file mode 100644 index 0000000..c431104 --- /dev/null +++ b/explanation/strategy.rst @@ -0,0 +1,196 @@ +================== +Launchpad strategy +================== + +*We want to make Ubuntu the world's best operating system. To do this, we need +to give Canonical an edge in productivity over and above other Linux vendors +and, just as importantly, help make the development of open source software +faster, more efficient and more innovative than its proprietary rivals.* + +Launchpad does this by helping software developers share their work and +plans, not just within a project but also **between** projects. + + +Introduction +============ + +This document tries to answer two big questions: + +1. *why* are we making Launchpad? +2. *who* is Launchpad for? + +This is not our strategy for the year or the scope of Launchpad development +for the next six months. Rather, it is our answer to these fundamental +questions. + +When you are finished reading this document, you should know what problems we +want to solve, what we hope to gain from solving these problems and how we +know if Launchpad is doing well. + + +Audience +-------- + +This document is for everyone who cares about improving Launchpad. Primarily, +we've written it for Launchpad's stakeholders within Canonical and for the +developers of Launchpad, whether they are Canonical employees or not. + + +Why are we making Launchpad? +============================ + +The world we live in +-------------------- + +Open source software is bigger than you think. It is much more than simply +writing the code. Code has to be packaged, integrated and delivered to users +who can then give feedback and file bugs. Distributions made up of tens of +thousands of different software packages need to be released to meet a +deadline. Translations must be made into hundreds of different languages and +accumulated from a variety of sources. Everywhere bugs need to be tracked, +fixed and checked. Plans must be made and kept. Distributions have to be +made to work on a wide variety of hardware platforms with varying degrees of +openness. + +Those who make open source software and wish to profit commercially also face +unique challenges. Contributors are scattered across the world, making +coordination, communication and alignment just that little bit more difficult. +Many contributors are volunteers, and so decisions must often be made by +consensus, deadlines enforced without the leverage of an employment contract +and quality maintained without formal training. Users of open source software +use a widely heterogeneous stack of software and hardware, thus increasing the +burden of compatibility work. All of these things make open source software +development more difficult, thus increasing the need for tools to aid +collaboration. + +The Ubuntu community, together with Canonical, are dedicated to making the +very best open source operating system possible, one that far excels any +proprietary operating system. To do this, we need to ensure that the process +of making Ubuntu is as effective as possible. Moreover, we need to make the +process of making open source software as effective as possible, and then make +it easy, quick and desirable to get that software into Ubuntu. + +Secondarily, Canonical's main business is the provision of premium services +built around Ubuntu. Many of these services are based on proprietary +software, which Canonical must be able to make more quickly and at less cost +than any rival. + +The word "effective" covers a multitude of concepts. Here we mean doing the +*right* work with the highest possible *quality* as *quickly* and with as +little *waste* as possible. + + +Business goals +-------------- + +Launchpad exists to give Canonical a competitive advantage over other +operating system vendors and service providers, both proprietary and open +source. + +To gain an advantage over *open source* operating system vendors, Canonical is +relying on Launchpad to: + +* increase Canonical's effectiveness in making software +* grow and accelerate contributions to Ubuntu + +To gain an advantage over proprietary operating system vendors, Canonical +needs Launchpad to do both of the above and: + +* improve and accelerate open source software development in general beyond + that of proprietary software so that the software in Ubuntu is better than + the software in any rival proprietary operating system + +The value flow of Launchpad can be summed up in this diagram: + +.. image:: ../images/value-flow.svg + + +Who is Launchpad for? +===================== + +Launchpad is aimed at many different groups of users. They can be roughly +described as follows: + +Software developers + These are people who make or contribute to free and open source + software. They are made up of both paid professionals and volunteers working + in their spare time. They vary widely in expertise and patience. Any given + software developer might be working on both open source software and + proprietary software. + +Expert users of software + The sort of people who file bugs, try new releases, run the bleeding edge + snapshot, are interested in following development plans, who help other + people on mailing lists. Note that software developers are frequently but + not always expert users of software. + +End users of software + People who download and install software and then use it. These people have + little understanding about what software actually is or how it is made. + They use it, sometimes without noticing, sometimes enjoying it, sometimes + hating it. + +Translators + A special class of software developer who is normally a native speaker of a + language other than English. They contribute to open source software + projects not by submitting code, but by translating strings to new + languages. + +Managers + These are managers in the broad sense of people who are responsible for the + completion of a task and so need to know what many other people are doing + towards that goal. This includes release managers, project leads and + traditional corporate project managers. It does not necessarily mean people + who are employed as managers. + + +User needs +---------- + +The people who use Launchpad, in whatever role, share one broad goal: “make +great software and get it to its users”. To do this, they need: + +* tools to facilitate collaboration on their proprietary and open source + software projects +* a place to host and publish their open source software projects +* as little overhead as possible in maintaining these projects +* more contributors to their projects +* to be able to easily contribute to existing software projects + +Some of our users have particular needs: + +* managers need to be able to quickly get an overview of activity and + progress for their teams and their projects +* expert users of software need to be able to give high quality feedback to + the software developers + +Further, we believe that providing tools for cross-project collaboration, we +can benefit our users by: + +* giving them feedback from groups of their own users that they couldn't reach + before +* reducing the time and effort required to publish software to actual end + users +* pointing them to knowledge and fixes from other projects in their network +* helping OS-driven improvements reach them code faster, and their + improvements reach the OS faster + + +Conflicts between business goals & user needs +--------------------------------------------- + +Canonical is primarily interested in open source software that runs on Linux +or lives within the Linux ecosystem. Thus, even though Launchpad could be an +excellent, general platform for Windows, OS X, iOS and Android based software, +our main area of focus is software that is aimed to run natively on Linux. + +Canonical is much more interested in quality assurance and release management +than many open source and even proprietary projects. + + +References +========== + +* :doc:`scope` +* :doc:`values` +* `Feature checklist `_ diff --git a/explanation/values.rst b/explanation/values.rst new file mode 100644 index 0000000..1c36d76 --- /dev/null +++ b/explanation/values.rst @@ -0,0 +1,172 @@ +================ +Launchpad values +================ + +Whenever we are thinking about what Launchpad should be, or how we should +implement a change, or whether something is a good idea or not, we have +recourse to three distinct sets of guidelines. + +The first is :doc:`strategy`, which reminds us why we are making Launchpad and +helps us answer questions such as "How does this help Launchpad meet its +goals?". The second is :doc:`scope`, which helps us answer questions like "Is +this in scope?". Together, they sort out the 'matter' of Launchpad. + +The third is this document, the Launchpad Values. It tries to address the +'manner' of Launchpad. It, perhaps, will not answer specific questions for +you. Rather, it will rule out certain options and decisions before they are +even considered. + +Like the :doc:`strategy`, this document is living: it should be changed and +improved as we learn more about how to make Launchpad. It is also +aspirational: not all of Launchpad lives up to these values. + + +Better tools help +================= + +Launchpad is fundamentally based on the principle that improving the tools +that we use to make things will help us make better things than we could +before and indeed make better things more cheaply than we could before. + +Launchpad should be that "better tool" and be always aiming to smooth and +accelerate the process of making software. + + +Invisible. If not, fun. +======================== + +Launchpad is a tool to help busy people get important stuff done. It should +stay out of the way where possible. Bugs, OOPSes, downtime and slowness all +draw attention to Launchpad and away from the interesting problems that our +users are trying to solve. + +Where it is not possible to stay out of the way, Launchpad should be fun to +use. We make next actions obvious and draw attention to users' achievements. + + +Example +------- + +When a branch is merged into the trunk of a project, that's generally the end +of its story. Launchpad quietly and silently detects that it has been merged +and marks the branch and any merge proposals as such. The branch then no +longer appears on default listings. + + +Reveal opportunities +==================== + +One of the grand things about open source software is that it is open to +contributions from total strangers. + +Launchpad makes those contributions possible by removing as many barriers as +possible to contribution, and highlights areas where contributions would be +especially welcome. + +Example +------- + +When you click the Translations tab of a project that's translatable, you see +a list of all of the languages you speak, together with a progress bar telling +you how much translation work is available. You can click a language and +start translating right away. + + +Not our data +============ + +The data in Launchpad does not really belong to us, we are merely its +stewards. We make sure that users can get their data easily and that they can +change it as they see fit. + +Example +------- + +You can access almost all of the data in Launchpad through the RESTful APIs. + + +Not just their data +=================== + +The data people store in Launchpad doesn't just belong to them, though. It +also belongs to the wider open source community. The data needs to be used to +link between other projects, and to allow Launchpad to act as a front-end of +open source. + +Example +------- + +Someone who is not actually a maintainer of a project might register that +project on Launchpad so they can import code or synchronize bugs or so forth. +If the project's actual maintainer comes along and wishes to take it down, we +will *not* do so. + + +Cross-project consistency +========================= + +If you know how to contribute to one project on Launchpad, you ought to be +able to quickly and painlessly contribute to any other project on Launchpad. +Or, if you can't, it won't be Launchpad's fault. + +Example +------- + +You can get the trunk branch for any project with 'bzr branch lp:project', or +the source for any Ubuntu package using 'bzr branch lp:ubuntu/project'. + + +All of open source +================== + +Any open source project ought to be able to host itself on Launchpad. We do +not enforce workflows, rather we allow people to fit their project's existing +workflow into Launchpad. + +However, when we can, we encourage people toward best practices. After all, +we want to make open source software better. + +Example +------- + +Launchpad separates whether or not a reviewer approves of code from whether or +not a branch is approved to land. + + +All users are potential contributors +==================================== + +One of the glories of open source is that any user is a potential +contributor. Launchpad guides new users toward the ways where they can begin +to contribute. + + +Close the loop +============== + +Something magical happens when a feature or a workflow reaches all the way +back to where it began. The feature begins to reinforce itself, and make +things possible that weren't possible before. + +Another way of thinking about this is that the value is in the output, and +Launchpad is always concerned with the output of all of its features. + +Example +------- + +These are examples of where we aren't there yet. + +Being able to attach patches to bugs is great, but it's not good enough until +developers can easily *find* those patches. Finding the patches is only good +enough when you can merge, comment and reject those patches. + +Likewise, importing translations from an upstream is great, but it becomes +much, much better when those translations can be improved in Launchpad and +then sent back to the upstream. + + +References +========== + +* :doc:`strategy` +* :doc:`scope` diff --git a/explanation/webapp-process.rst b/explanation/webapp-process.rst new file mode 100644 index 0000000..ed25276 --- /dev/null +++ b/explanation/webapp-process.rst @@ -0,0 +1,353 @@ +========================================= +How to go about writing a web application +========================================= + +-- Steve Alexander + +.. include:: ../includes/attention_out_of_date.rst + + +Introduction +------------ + +This document presents an approach to constructing web applications, +emphazising well-designed user interaction. + +In summary, we write a web application by + +1. Design the database that lies behind the web application + - Entity relationship models + - SQL statements +2. Design the web application's user interface + - Table of URLs (see later) + - Page template mock-ups +3. Write the Interfaces that the application will use to access the database + - interfaces.py files +4. Write the code and templates of the application + - page templates + - supporting classes + - application components + +Of course, this isn't a completely linear process. Steps may be carried out +in parallel, and ongoing work in each step will feed into the other steps. + +So that we can make rapid progress, and learn about the application as early +as possible, we should do steps 2, 3 and 4 focusing on just a few URLs at +a time (say, five or so). Then, when those are finished, we can make any +necessary changes to the database model, and then repeat steps 2, 3 and 4 +with some new URLs and new application functionality. + + + +Web application architecture +---------------------------- + +For the purposes of this document, a web application is put together as +follows:: + + +--------------------------------------------------------------------------+ + | { Amorphous cloud of URLs } | + | URLS { /rosetta/foo/bar/baz.html } | + | { } | + | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | + | { Pages SubURLs | + | PRESENTATION { Views Templates | + | { Traversal Support classes | + | | + | WEB APP ------------------------------------------------- Interfaces | + | | + | APPLICATION { Utilities Components | + | COMPONENTS { Adapters | + | | + | | + | LIBRARIES { Python standard library | + | { Other libraries | + | | + | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | + | | + | DATABASE relations, fields, constraints | + +--------------------------------------------------------------------------+ + +The boundaries of our web application are URLs at the top and the database +at the bottom. So, we must carefully define these things that lie at the +boundaries, so that we can work out what must go in the middle. + +URLs +---- + +URLs are the boundary between the web application and web browsers. When you +point a web browser at a URL, you'll get one of the following: + +- page +- form +- image or other data +- redirect +- "not found" error +- "unauthorized" error + +We'll be mostly concerned with pages, forms and redirects. We'll be concerned +with image and other data only when they are "managed content" and not just +part of the skin of the site. The photograph of a member of the site is an +example of managed content: it is stored in the database and may be +manipulated through using the application. CSS files, javascript files, +icons and logos are typically elements that make up the "skin" of the +application, and are not managed through the application. + +Another example of image or other data that is managed content is if we want +to allow users to download a .po or .pot file of translations. + +Forms are rendered in HTML and will typically be "self-posting forms". That +is, the address posted to when the [submit] button is pressed will be the +same as the address the page was got from. This allows browsers' bookmarks +to work properly, allows us to straightforwardly validate data entered into +the form, and keeps the URL space neat. + +The "not found" error is the familiar "404 Not Found" given when someone +tries to access a URL that is not known by our system. + +An "unauthorized" error means that accessing the given URL requires some +authentication. The browser will prompt the user to provide some additional +authentication. Alternatively, if we use other login schemes than HTTP Basic +or HTTP Digest, then the web application may present a login form to the +user. + +A "redirect" causes the browser to immediately fetch a different URL. This +process is usually not noticed by the user. + + +The URL table +------------- + +We need to construct a table describing the URLs used in our application. We +won't bother about the protocol part or any leading path segments of the URL. +In the table below, we'll assume all URLs start http://example.com/rosetta/... + +The table has the following columns + +- URL: The rest of the URL after /rosetta. For clarity, we'll start it with + a "./". So, the rosetta application's main page might be "./index.html". + If this table is written in HTML, this may be a link to a mock-up HTML page + showing what will be found at that URL. + +- Default: For application components, the default page to use. More about + this later. + +- Type: This is one of "app component", "page", "form", "redirect", "data" + +- Description: A textual description of what is found at this URL. + This may be a link to further information about the functioning of that + page, form validation constraints, and so on. + +When you get an App Component at a particular URL, what does that mean? At +certain points in our URL space, we want to expose a set of related +functionality under a particular URL. For example, the URL +"./projects/mozilla/show-teams" might show the teams working on the Mozilla +project, while "./projects/mozilla/translations" might show the translations +of the mozilla project. Users of the system will come to understand that +things related to Mozilla are to be found at URLs starting with +"./projects/mozilla". We want to present some page at "./projects/mozilla". +Rather than make a special "no name" page for this, we choose one of the +other pages that we want to return. Mozilla is a Project. So, if the +default page for a Project is "translations", then going to +"./projects/mozilla" will return the same page as +"./projects/mozilla/translations". The usual default page is "index.html". + + +Here's an example of a table for the Rosetta project:: + + URL Default Type Description + + ./ index.html app component The rosetta application + + ./index.html page Initial navigation page + + ./intro.html page Introductory page + + ./signup.html form Allows a new user to + register with the system. + + ./projects index.html app component Collection of rosetta + projects + + ./projects/index.html form Shows ways to search + through projects + + ./projects/$PROJECT translations app component A particular project + $PROJECT is the name of + the project. See key below. + + ./projects/$PROJECT/translations page Shows all translations + for this project. + + + Key to $VARs + ============ + $PROJECT The name of the project. This is the name attribute of + the IProjectGroup interface, or the name field in the Project + relation. Case is not significant, and is normalized to + lower-case in the UI. Examples: 'mozilla', 'gtk+'. + +We can use the URL table for simple automated functional testing of the web +application, given some suitable $VAR substitutions. + + +Structure of a web application URL +---------------------------------- + +We need to know what types of things are at a particular URL. Here's an +example of a typical URL in a web application. This time, I've included +the "rosetta" path segment at the root:: + + /rosetta/projects/$PACKAGENAME/teams/$TEAM/add-member.html + | | | | | | + | | | | | page to add a new member + | | | | Name of a particular team "22" + | | | The teams working on this project + | | A particular project name, such as "mozilla" + | Collection of projects that can be queried + The rosetta application + + +Guidelines for URLs +------------------- + +* Make your URLs from lower-case letters, numbers, '-' and '+'. + +* Avoid '_', capital letters, other symbols. + Using these things makes the URL harder to read out over the phone or + write down unambiguously. + +* When you have a collection of things, such as people or projects, use + a plural noun for that part of the URL. For example, "./projects". + +* Consider using "+" as the last URL segment for the URL that adds things + to a collection. For example, "./projects/+" to add a new project. + +* Where possible, use self-posting forms. So, you would go to the URL + "./projects/+" to get a form asking you for the information needed to + add a new project. When you submit the form, it POSTs to the same + URL. If the provided information is invalid, you'll get the form back + with an error message. Otherwise, you'll get a "success" message, or be + redirected to the next page in the workflow. + + + +Development iterations +---------------------- + +When you're developing a new system, don't try to write the whole table of +URLs at once. Instead, we can work in iterative cycles, designing pages +and URLs, and making these work in software. That way, we can learn earlier +on if the URLs and pages we want will actually work in practice. + +Here's the overall process of developing the application. + +Overall Process +~~~~~~~~~~~~~~~ + +1. Lay out the total functionality of the system, and divide it into a number + of iterations. +2. Pick the next iteration. Go through the Iteration Process described below. +3. Review / refactor the specification for previous iterations based on what + we learned during this iteration. +4. Refactor the whole application implemented so far to match the refactored + specification. + +Each iteration (that is, step 2 above) looks like this. + +Iteration Process +~~~~~~~~~~~~~~~~~ + +1. Write the URLs required for this iteration into the URLs table. + Ideally, there should be 3 to 7 URLs in each iteration. +2. Document the functionality required for each page. +3. Produce page template mockups. +4. Implement the functionality, using stub application components rather + than real application components. +5. Connect the functionality to the real database, by replacing the stubs + with real application components. + + +I will note that these processes are just guidelines on how to go about +writing the software. You might choose to prototype the application in +order to learn about what URLs are required for some tricky interaction. Or, +you might decide to write two iterations' worth of URLs into the URLs table +all at once, but then implement them in two iterations. The important thing +is to understand where you are in this process, and why you are doing what +you are doing at any particular stage. + +Keep the iterations short! + + +Glossary +-------- + +Skin: + The way the user interface looks in a web browser. The elements of this + user interface, including CSS, images and an overall site template. + + It is possible to provide multiple skins to the same web application, + for example a simple one and a very complex one. + +Published: + Something that is made available at a particular URL is said to be + published. + +Presentation component: + Some software that interacts with the browser's request and returns + information to the browser. This is typically a page template or a + page template plus a supporting class. + + Other presentation components are traversers, which know what to do + when further path segments are given in a URL; and resources, which + are CSS files, javascript files, logos, icons, etc. + +Application component: + An object that represents application functionality, but not presentation + functionality. It should have a well-defined interface so that different + implementations of a given application component can be presented by + the same presentation components. + +Component: + An object that has clearly defined interfaces. + + These interfaces may represent what it offers, or what it requires in + order to function. + +Utility: + A component that is looked up by the interface that it provides. + +Adapter: + A component that knows how to use a particular interface in order to + provide a different interface. + +Interface: + A software-readable definition of an API provided by some object. + +View: + A kind of presentation component that provides a representation of + some other component. + +Browser presentation: + Presentation intended for a web browser, as distinct from a presentation + intended for XML-RPC or webdav. Or even email. + +Non-published {view,resource}: + A {view,resource} that is used by other presentation components, but + that is not itself addressable by a URL. + +Page: + An HTML document returned to a browser in response to a GET or POST + request to some URL. + +Form: + A page that contains HTML form elements and at least one "submit" + button. + +Self-posting form: + A form that's "action URL" is the same address that it was loaded from. + So, a form that was loaded from "./projects/+" would start:: + +
+ + diff --git a/how-to/chameleon.rst b/how-to/chameleon.rst new file mode 100644 index 0000000..11d4be4 --- /dev/null +++ b/how-to/chameleon.rst @@ -0,0 +1,29 @@ +Running Launchpad with Chameleon Template Engine +================================================ + +.. include:: ../includes/attention_out_of_date.rst + +- Uncomment ```` in ``zcml/zopeapp.zcml`` + to enable ``z3c.pt``. + + +Useful environment options for ``z3c.pt``:: + + # in debug-mode, templates on disk are reloaded if they're modified + CHAMELEON_DEBUG (default: false) + + # disable disk-cache to prevent the compiler from caching on disk + CHAMELEON_CACHE (default: true) + + # if eager parsing is enabled, templates are parsed upon + # instantiation, rather than when first called upon; this mode is + # useful for verifying validity of templates across a project + CHAMELEON_EAGER (default: false) + + # in strict mode, filled macro slots must exist in the macro that's + # being used. + CHAMELEON_STRICT (default: false) + + # when validation is enabled, dynamically inserted content is + # validated against the XHTML standard + CHAMELEON_VALIDATE (default: false) diff --git a/how-to/contributing-changes.rst b/how-to/contributing-changes.rst index 968a016..d27e73a 100644 --- a/how-to/contributing-changes.rst +++ b/how-to/contributing-changes.rst @@ -25,7 +25,7 @@ Once you have decided on the change to be made, clone the repository. git clone https://git.launchpad.net/launchpad Make your changes -------------------- +----------------- Create a branch from a reasonable point, such as ``master``. diff --git a/how-to/index.rst b/how-to/index.rst index adc4062..3e7d770 100644 --- a/how-to/index.rst +++ b/how-to/index.rst @@ -24,6 +24,7 @@ Common development tasks update-configuration-for-testing testing-scripts create-job-publish-artifact + chameleon Operating development instances ------------------------------- diff --git a/includes/attention_out_of_date.rst b/includes/attention_out_of_date.rst new file mode 100644 index 0000000..b58ec15 --- /dev/null +++ b/includes/attention_out_of_date.rst @@ -0,0 +1,3 @@ +.. attention:: + + This is a historical document and might be out of date.