xblock and course addressing

Problem statement

EdX code treats xblock and course ids as transparent, introspectable, and frequently manipulated addresses with a known set of fields throughout the stack. The applications serialize and deserialize these ids as parsable urls with fixed syntax (org, partial course id, category, block id or org, partial course id, course unique id). This treatment makes it almost impossible to add semantics such as versioning, subordinate organizations, lifecycle snapshots (e.g., draft, review, alpha, beta, published), or course encapsulation to the applications.

The db rearchitecture task known as "Split Mongo" uses all of the additional semantics listed above as well as separating the identity of courses and reusable definitions from the identity of xblocks within courses (usages). Xblock also distinguishes usages from definitions; so, this change is not unilateral. LMS and others always informally separated course identity from xblock usage identity. Split makes these distinctions formal.

We had a plan to have the two id representations exist simultaneously with continuous translation between them as necessary (Rollout Options); however, this plan raised significant performance questions as well as risks around fragility (possibly not finding some hardcoded ids and translating them to the proper representation for the caller or receiver.)

Discussion with Cale, Rob, Ned, DB, and others drove home the point that only the persistence layer should care about xblock usage and definition ids and only the persistence, auth, and registration layers should care about course ids. The application layers (LMS, CMS, analytics, ORA, Forums) should treat these ids as opaque tokens used to perform data CRUD and reference. The application layers should not parse the ids and should not request semantic information from the ids (e.g., category, org, course_id).

However, all of the apps currently serialize and deserialize the ids as subfields from urls; thus, to make the apps able to handle any persistence layer's id representations, we will need to change all of the urls and the url parsing. However, we must provide deprecated backward compatibility which can still interpret hardcoded and bookmarked urls using the existing syntax.

Proposed solution

The crux of the proposed solution is that all ids (course, xblock, definition) will be opaque keys to the applications. The persistence, auth, and enrollment modules may introspect these keys as necessary for additional information. If an application needs additional information such as org, course_id, category and just has an id, the application must as the owning service. No application should assume such requests are free: that is, they may require db operations or other non-trivial lookups. Sometimes context will make it clear what behaviors the service will likely support for the id (e.g., course_id for an xblock usage but not for an xblock definition, org for a course_id but not necessarily for a definition, category for an xblock usage or definition but not for a course id) but some services may support behaviors which other services don't support (e.g., version and version history for an xblock usage, list of available lifecycle snapshots such as draft v preview v live). Xblocks will themselves support some of these behaviors and, so, in many cases, it may make more sense for the app to retrieve the xblock given the address and query it.