From 1a6d08957555d0fc3974d23cf5023a3fb3343c2d Mon Sep 17 00:00:00 2001 From: "Henry H. Andrews" Date: Mon, 5 Feb 2024 12:01:22 -0800 Subject: [PATCH 1/3] Shorter, more focused referencing guide This reworks the less-controversial parts of a prevoius attempt at a referencing guide that was over-ambitious, among other things. --- referencing/faq.md | 69 ++++++++++++++++++++++++++++++++++++++++++++ referencing/index.md | 19 ++++++++++++ referencing/what.md | 27 +++++++++++++++++ 3 files changed, 115 insertions(+) create mode 100644 referencing/faq.md create mode 100644 referencing/index.md create mode 100644 referencing/what.md diff --git a/referencing/faq.md b/referencing/faq.md new file mode 100644 index 0000000..b9061fa --- /dev/null +++ b/referencing/faq.md @@ -0,0 +1,69 @@ +--- +layout: default +title: Referencing FAQ +nav_order: 1 +parent: Using References +--- + +# Referencing FAQ + +A few quick answers to common questions when using references with multi-document OpenAPI Descriptions (OADs). + +## Is it possible to convert all OADs to a single-document form without any references? + +The short answer is "no", but there are many cases where it is possible to do such a thing. The difficult cases are: + +* Cyclic references in JSON Schema ***cannot*** be removed, although if the references cross documents, it is possible to reduce them to same-document references in a single document +* While dynamic references in OAS 3.1 Schema Objects technically can be converted to static references, it can [grow the schema size exponentially](https://dl.acm.org/doi/10.1145/3632891) ([pdf](https://dl.acm.org/doi/pdf/10.1145/3632891)) +* OAS 3.1 Schema Objects, OAS 3.0 Reference Objects, and both OAS 3.0 and 3.1 Path Item Objects all allow keywords alongsidea`"$ref"`, each with slightly different semantics; generic reference removal tools, particularly ones that predate OAS 3.1, may fail to preserve the behavior while removing references +* In OAS 3.1 Schema Objects that use `"$id"`, reference removal tools that are not aware of `"$id"` will resolve references impacted by the `"$id"` incorrectly +* The Link Object's `"operationRef"` cannot be replaced with an inline Operation Object (although if the target Operation Object has a unique-within-the-OAD `"operationId"`, then the `"operationRef"` can be replaced by the `"operationId"`) +* The Discriminator Object's `"mapping"` field forbids inline Schema Objects; URI-reference values can be replaced with names from the `"schemas"` section of the Components Object, but when the Discriminator Object is in a referenced document, there are ambiguities involvied in how to resolve those names +* When using components published by 3rd parties such as standards organizations, tools might rely on reference URIs to identify standard components; while this usage is currently uncommon, it may become more common as OpenAPI is more broadly adopted by standards bodies + +If your OAD does not run afoul of any of the above difficulties, you _can_ produce a reference-less single-document OAD, and there are [tools](https://tools.openapis.org) that will do so (but see the caveats in the next section). + +## Can I convert my multi-document OAD into a single document, even if I cannot remove all references? + +Yes. However, depending on exactly how your multi-document OAD is structured, this may be more complex than just removing references. Sadly, the reasons have to do with [painfully obscure and technical ambiguities](https://github.com/OAI/oascomply/blob/main/reports/processing-model-summary.md) regarding how multi-document OADs are intended to be parsed. The OpenAPI Initiative hopes to clarify this in future versions of the specification. + +* It is possible for a Link Object's `"operationId"` to point to an Operation Object in a Path Item Object that is never referenced; it is unclear how this case should handle it, which is one reason the OAS itself [advises against](https://spec.openapis.org/oas/v3.1.0#fixed-fields-16) using `"operationId"` in multi-document OADs +* Security Scheme Objects are never referenced, only named in Security Requirement Objects, and must be taken into account +* There is no standard place to put referenced Path Item Objects in OAS 3.0 +* The resolution of server configuration and security settings can be unclear in multi-document OADs + +Again, in many cases none of this is a problem, and there are [tools](tools.openapis.org) that will do what you want. + +## What is "dynamic referencing" and would I use it? + +Dynamic referencing is a feature of JSON Schema draft 2020-12, and therefore included in OAS 3.1. It allows setting a placeholder reference target that can be overridden during validation depending on how the dynamic reference was reached. If that explanation didn't help you at all, don't worry, it boils down to two use cases: + +* [Implementing generic types](https://json-schema.org/blog/posts/dynamicref-and-generics#using-dynamic-references-to-support-generic-types) (a.k.a. templatized types), such as container data structures or payload envelopes +* Extending recursive schemas, as the OAS schema needed to do to [add OAS-specific keywords to the standard JSON Schema dialect](https://json-schema.org/blog/posts/validating-openapi-and-json-schema) + +While there may be other creative ways to use dynamic references, these are the situations most likely to show up in API descriptions. + +## How is reference resolution different from reference removal? + +A reference is said to be _resolved_ within a tool if: + +* Its target has been identified +* Any modifications to the target required by the OAS (e.g. because of fields adjacent to `"$ref"`) have been performed on a copy of the target +* The resulting target value has been associated with the reference source in some way that the tool can easily use when needed + +A reference is said to be _removed_ if it has been replaced by its (possibly modified) target. + +Reference resolution usually preserves the referencing information such as the URI used to identify the target, while reference removal usually discards it. In many cases this is not significant, except that not knowing how the parsed OAD relates to the refernces in your JSON or YAML document may make debugging more difficult. + +While plain JSON documents form a [tree](https://en.wikipedia.org/wiki/Tree_%28data_structure%29) structure, an OpenAPI Description with resolved references is not _necessarily_ a tree, but a [graph](https://en.wikipedia.org/wiki/Graph_%28abstract_data_type%29). + +Tools that resolve references in-memory and work with the graph structure can process all OADs. Tools that rely on reference removal, either as part of the tool or by a separate pre-processing tool, can only support OADs that form trees. + +## Is anything being done to reduce this complexity in future OpenAPI Specifications? + +Yes! + +The [Moonwalk project](https://github.com/OAI/sig-moonwalk/discussions) is considering a different approach that imports complete documents, associates them with namespaces, and only supports referencing by component name (not `"$ref"`). A small example can be seen in the [Moonwalk deployments proposal](https://github.com/OAI/sig-moonwalk/blob/main/Deployments.md), and there are discussions around an [initial draft proposal for imports](https://github.com/OAI/sig-moonwalk/discussions/72) and a few ideas on [how tomanage interactions with JSON Schema referencing](https://github.com/OAI/sig-moonwalk/discussions/73). + +The [proposed Workflows Specification](https://github.com/OAI/sig-workflows/blob/main/versions/1.0.0.md) is already using a `"sourceDescription"` field that is not unlike the Moonwalk proposal. + diff --git a/referencing/index.md b/referencing/index.md new file mode 100644 index 0000000..fc9a8e8 --- /dev/null +++ b/referencing/index.md @@ -0,0 +1,19 @@ +--- +layout: default +title: Using References +nav_order: 5 +has_children: true +has_toc: false +--- + +# Using References in OpenAPI Descriptions + +OpenAPI Referencing is a powerful tool. It allows managing document size and complexity, and allows re-using shared components and avoiding copy-paste or change management errors. + +However, the history of referencing and the `"$ref"` keyword is complex, leading to slightly different behavior depending on the version of the OpenAPI Specification (OAS) that you are using, and on where in your OpenAPI Description (OAD) the reference occurs. There are also other `"$ref"`-like keywords (`"operationRef"`, `"mapping"`) and behaviors (referencing by component name or operation ID) that are related but somwhat different. Referencing can also be challenging to use due to incomplete and inconsistent support across different tools, some of which require references to be pre-processed before they can read the OAD. + +This guide explains how to use referencing, and what to look for when assessing the referencing support in your OpenAPI tools. + +- [Referencing FAQ](faq) +- [What are References?](what) +- Resolving References (forthcoming) diff --git a/referencing/what.md b/referencing/what.md new file mode 100644 index 0000000..401a173 --- /dev/null +++ b/referencing/what.md @@ -0,0 +1,27 @@ +--- +layout: default +title: What Are References? +nav_order: 2 +parent: Using References +--- + +# What are references? + +A *reference* is a keyword and value that identifies a *reference target* with a URI. In some cases, this URI can be treated as a URL and de-referenced directly. In other cases, as we will see in the (forthcoming) guide to resolving references, it is helpful to separate the target's *identity* from its *location*. + +External references are how multiple documents are linked into a single OpenAPI Description (OAD). This means that referencing impacts how other linkages, such as those that use Components Object names, or values such as `operationId` in the Path Item Object, work. These other linkages can only work if the document (or with many tools, the specific JSON object) containing the name or other identifier has been referenced. + +## A taxonomy of references + +References exist in several variations in the OpenAPI Specification (OAS) versions 3.0 and 3.1, as shown in the following table. Note that an *adjacent keyword* is a keyword in the same JSON Object (whether it is written in JSON or YAML) as the reference keyword. + +| OAS Version | Object | Reference Keyword | Adjacent Keywords | Behavior | +| ----------- | ------ | ----------------- | ----------------- | -------- | +| 3.0 | Reference Object | `"$ref"` | ignored | logically replace the Reference Object with the reference target | +| 3.1 | Reference Object | `"$ref"` | `"summary"` and `"description"` allowed; others ignored | logically replace the Reference Object with a copy of the target, overwriting the target's `"summary"` and/or `"description"` fields with those of the Reference Object if they are present | +| 3.x | Path Item Object | `"$ref"` | allowed under some circumstances | logically replace the Path Item Object containing the `"$ref"` with a Path Item Object that combines the fields of the target Path Item Object with the non-`"$ref"` fields of the Path Item Object containing the `"$ref"`, as long as none of those fields conflict | +| 3.1 | Schema Object | `"$ref"` | allowed | Apply the target Schema Object to the same instance location as the Schema Object containing the `"$ref"`, and combine the results with the results of other keywords in the Schema Object containing the `"$ref"` just as you would any other keyword results; this is more-or-less equivalent to using a one-element `"allOf"` | +| 3.x | Link Object | `"operationRef"` | allowed, except `"operationId"` | treat the reference target as the target of the link described by the Link Object | +| 3.x | Discriminator Object | `"mapping"` | n/a | for each name under `"mapping"`, if the value is not the name of a Schema Object under the Components Object, treat it as a reference to the schema to use when the discriminator field matches the mapping name | + +Note that not all references are handled by replacing the source with the target. Some are defined in terms of behavior, and **cannot** be replaced with an inline value. From f4be60e25a6691eed17bb568f0d9b8485da7ed81 Mon Sep 17 00:00:00 2001 From: "Henry H. Andrews" Date: Sun, 11 Feb 2024 15:06:03 -0800 Subject: [PATCH 2/3] Fix typos --- referencing/faq.md | 8 ++++---- referencing/index.md | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/referencing/faq.md b/referencing/faq.md index b9061fa..fcf5e97 100644 --- a/referencing/faq.md +++ b/referencing/faq.md @@ -15,10 +15,10 @@ The short answer is "no", but there are many cases where it is possible to do su * Cyclic references in JSON Schema ***cannot*** be removed, although if the references cross documents, it is possible to reduce them to same-document references in a single document * While dynamic references in OAS 3.1 Schema Objects technically can be converted to static references, it can [grow the schema size exponentially](https://dl.acm.org/doi/10.1145/3632891) ([pdf](https://dl.acm.org/doi/pdf/10.1145/3632891)) -* OAS 3.1 Schema Objects, OAS 3.0 Reference Objects, and both OAS 3.0 and 3.1 Path Item Objects all allow keywords alongsidea`"$ref"`, each with slightly different semantics; generic reference removal tools, particularly ones that predate OAS 3.1, may fail to preserve the behavior while removing references +* OAS 3.1 Schema Objects, OAS 3.0 Reference Objects, and both OAS 3.0 and 3.1 Path Item Objects all allow keywords alongside a`"$ref"`, each with slightly different semantics; generic reference removal tools, particularly ones that predate OAS 3.1, may fail to preserve the behavior while removing references * In OAS 3.1 Schema Objects that use `"$id"`, reference removal tools that are not aware of `"$id"` will resolve references impacted by the `"$id"` incorrectly * The Link Object's `"operationRef"` cannot be replaced with an inline Operation Object (although if the target Operation Object has a unique-within-the-OAD `"operationId"`, then the `"operationRef"` can be replaced by the `"operationId"`) -* The Discriminator Object's `"mapping"` field forbids inline Schema Objects; URI-reference values can be replaced with names from the `"schemas"` section of the Components Object, but when the Discriminator Object is in a referenced document, there are ambiguities involvied in how to resolve those names +* The Discriminator Object's `"mapping"` field forbids inline Schema Objects; URI-reference values can be replaced with names from the `"schemas"` section of the Components Object, but when the Discriminator Object is in a referenced document, there are ambiguities involved in how to resolve those names * When using components published by 3rd parties such as standards organizations, tools might rely on reference URIs to identify standard components; while this usage is currently uncommon, it may become more common as OpenAPI is more broadly adopted by standards bodies If your OAD does not run afoul of any of the above difficulties, you _can_ produce a reference-less single-document OAD, and there are [tools](https://tools.openapis.org) that will do so (but see the caveats in the next section). @@ -53,7 +53,7 @@ A reference is said to be _resolved_ within a tool if: A reference is said to be _removed_ if it has been replaced by its (possibly modified) target. -Reference resolution usually preserves the referencing information such as the URI used to identify the target, while reference removal usually discards it. In many cases this is not significant, except that not knowing how the parsed OAD relates to the refernces in your JSON or YAML document may make debugging more difficult. +Reference resolution usually preserves the referencing information such as the URI used to identify the target, while reference removal usually discards it. In many cases this is not significant, except that not knowing how the parsed OAD relates to the references in your JSON or YAML document may make debugging more difficult. While plain JSON documents form a [tree](https://en.wikipedia.org/wiki/Tree_%28data_structure%29) structure, an OpenAPI Description with resolved references is not _necessarily_ a tree, but a [graph](https://en.wikipedia.org/wiki/Graph_%28abstract_data_type%29). @@ -63,7 +63,7 @@ Tools that resolve references in-memory and work with the graph structure can pr Yes! -The [Moonwalk project](https://github.com/OAI/sig-moonwalk/discussions) is considering a different approach that imports complete documents, associates them with namespaces, and only supports referencing by component name (not `"$ref"`). A small example can be seen in the [Moonwalk deployments proposal](https://github.com/OAI/sig-moonwalk/blob/main/Deployments.md), and there are discussions around an [initial draft proposal for imports](https://github.com/OAI/sig-moonwalk/discussions/72) and a few ideas on [how tomanage interactions with JSON Schema referencing](https://github.com/OAI/sig-moonwalk/discussions/73). +The [Moonwalk project](https://github.com/OAI/sig-moonwalk/discussions) is considering a different approach that imports complete documents, associates them with namespaces, and only supports referencing by component name (not `"$ref"`). A small example can be seen in the [Moonwalk deployments proposal](https://github.com/OAI/sig-moonwalk/blob/main/Deployments.md), and there are discussions around an [initial draft proposal for imports](https://github.com/OAI/sig-moonwalk/discussions/72) and a few ideas on [how to manage interactions with JSON Schema referencing](https://github.com/OAI/sig-moonwalk/discussions/73). The [proposed Workflows Specification](https://github.com/OAI/sig-workflows/blob/main/versions/1.0.0.md) is already using a `"sourceDescription"` field that is not unlike the Moonwalk proposal. diff --git a/referencing/index.md b/referencing/index.md index fc9a8e8..e904697 100644 --- a/referencing/index.md +++ b/referencing/index.md @@ -10,7 +10,7 @@ has_toc: false OpenAPI Referencing is a powerful tool. It allows managing document size and complexity, and allows re-using shared components and avoiding copy-paste or change management errors. -However, the history of referencing and the `"$ref"` keyword is complex, leading to slightly different behavior depending on the version of the OpenAPI Specification (OAS) that you are using, and on where in your OpenAPI Description (OAD) the reference occurs. There are also other `"$ref"`-like keywords (`"operationRef"`, `"mapping"`) and behaviors (referencing by component name or operation ID) that are related but somwhat different. Referencing can also be challenging to use due to incomplete and inconsistent support across different tools, some of which require references to be pre-processed before they can read the OAD. +However, the history of referencing and the `"$ref"` keyword is complex, leading to slightly different behavior depending on the version of the OpenAPI Specification (OAS) that you are using, and on where in your OpenAPI Description (OAD) the reference occurs. There are also other `"$ref"`-like keywords (`"operationRef"`, `"mapping"`) and behaviors (referencing by component name or operation ID) that are related but somewhat different. Referencing can also be challenging to use due to incomplete and inconsistent support across different tools, some of which require references to be pre-processed before they can read the OAD. This guide explains how to use referencing, and what to look for when assessing the referencing support in your OpenAPI tools. From fc1d4f55ce14d4ed83725c3d2bd583aecd78bdec Mon Sep 17 00:00:00 2001 From: Lorna Jane Mitchell Date: Sat, 24 Feb 2024 19:38:14 +0000 Subject: [PATCH 3/3] Restructure content into topic pages --- referencing/dynamic.md | 16 +++++++ referencing/faq.md | 69 ---------------------------- referencing/index.md | 14 ++++-- referencing/{what.md => overview.md} | 4 +- referencing/remove.md | 38 +++++++++++++++ referencing/resolve.md | 23 ++++++++++ 6 files changed, 89 insertions(+), 75 deletions(-) create mode 100644 referencing/dynamic.md delete mode 100644 referencing/faq.md rename referencing/{what.md => overview.md} (98%) create mode 100644 referencing/remove.md create mode 100644 referencing/resolve.md diff --git a/referencing/dynamic.md b/referencing/dynamic.md new file mode 100644 index 0000000..907f26c --- /dev/null +++ b/referencing/dynamic.md @@ -0,0 +1,16 @@ +--- +layout: default +title: Dynamic References +nav_order: 4 +parent: Using References +--- + +## Dynamic References + +Dynamic referencing is a feature of JSON Schema draft 2020-12, and therefore included in OAS 3.1. It allows setting a placeholder reference target that can be overridden during validation depending on how the dynamic reference was reached. If that explanation didn't help you at all, don't worry, it boils down to two use cases: + +* [Implementing generic types](https://json-schema.org/blog/posts/dynamicref-and-generics#using-dynamic-references-to-support-generic-types) (a.k.a. templatized types), such as container data structures or payload envelopes +* Extending recursive schemas, as the OAS schema needed to do to [add OAS-specific keywords to the standard JSON Schema dialect](https://json-schema.org/blog/posts/validating-openapi-and-json-schema) + +While there may be other creative ways to use dynamic references, these are the situations most likely to show up in API descriptions. + diff --git a/referencing/faq.md b/referencing/faq.md deleted file mode 100644 index fcf5e97..0000000 --- a/referencing/faq.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -layout: default -title: Referencing FAQ -nav_order: 1 -parent: Using References ---- - -# Referencing FAQ - -A few quick answers to common questions when using references with multi-document OpenAPI Descriptions (OADs). - -## Is it possible to convert all OADs to a single-document form without any references? - -The short answer is "no", but there are many cases where it is possible to do such a thing. The difficult cases are: - -* Cyclic references in JSON Schema ***cannot*** be removed, although if the references cross documents, it is possible to reduce them to same-document references in a single document -* While dynamic references in OAS 3.1 Schema Objects technically can be converted to static references, it can [grow the schema size exponentially](https://dl.acm.org/doi/10.1145/3632891) ([pdf](https://dl.acm.org/doi/pdf/10.1145/3632891)) -* OAS 3.1 Schema Objects, OAS 3.0 Reference Objects, and both OAS 3.0 and 3.1 Path Item Objects all allow keywords alongside a`"$ref"`, each with slightly different semantics; generic reference removal tools, particularly ones that predate OAS 3.1, may fail to preserve the behavior while removing references -* In OAS 3.1 Schema Objects that use `"$id"`, reference removal tools that are not aware of `"$id"` will resolve references impacted by the `"$id"` incorrectly -* The Link Object's `"operationRef"` cannot be replaced with an inline Operation Object (although if the target Operation Object has a unique-within-the-OAD `"operationId"`, then the `"operationRef"` can be replaced by the `"operationId"`) -* The Discriminator Object's `"mapping"` field forbids inline Schema Objects; URI-reference values can be replaced with names from the `"schemas"` section of the Components Object, but when the Discriminator Object is in a referenced document, there are ambiguities involved in how to resolve those names -* When using components published by 3rd parties such as standards organizations, tools might rely on reference URIs to identify standard components; while this usage is currently uncommon, it may become more common as OpenAPI is more broadly adopted by standards bodies - -If your OAD does not run afoul of any of the above difficulties, you _can_ produce a reference-less single-document OAD, and there are [tools](https://tools.openapis.org) that will do so (but see the caveats in the next section). - -## Can I convert my multi-document OAD into a single document, even if I cannot remove all references? - -Yes. However, depending on exactly how your multi-document OAD is structured, this may be more complex than just removing references. Sadly, the reasons have to do with [painfully obscure and technical ambiguities](https://github.com/OAI/oascomply/blob/main/reports/processing-model-summary.md) regarding how multi-document OADs are intended to be parsed. The OpenAPI Initiative hopes to clarify this in future versions of the specification. - -* It is possible for a Link Object's `"operationId"` to point to an Operation Object in a Path Item Object that is never referenced; it is unclear how this case should handle it, which is one reason the OAS itself [advises against](https://spec.openapis.org/oas/v3.1.0#fixed-fields-16) using `"operationId"` in multi-document OADs -* Security Scheme Objects are never referenced, only named in Security Requirement Objects, and must be taken into account -* There is no standard place to put referenced Path Item Objects in OAS 3.0 -* The resolution of server configuration and security settings can be unclear in multi-document OADs - -Again, in many cases none of this is a problem, and there are [tools](tools.openapis.org) that will do what you want. - -## What is "dynamic referencing" and would I use it? - -Dynamic referencing is a feature of JSON Schema draft 2020-12, and therefore included in OAS 3.1. It allows setting a placeholder reference target that can be overridden during validation depending on how the dynamic reference was reached. If that explanation didn't help you at all, don't worry, it boils down to two use cases: - -* [Implementing generic types](https://json-schema.org/blog/posts/dynamicref-and-generics#using-dynamic-references-to-support-generic-types) (a.k.a. templatized types), such as container data structures or payload envelopes -* Extending recursive schemas, as the OAS schema needed to do to [add OAS-specific keywords to the standard JSON Schema dialect](https://json-schema.org/blog/posts/validating-openapi-and-json-schema) - -While there may be other creative ways to use dynamic references, these are the situations most likely to show up in API descriptions. - -## How is reference resolution different from reference removal? - -A reference is said to be _resolved_ within a tool if: - -* Its target has been identified -* Any modifications to the target required by the OAS (e.g. because of fields adjacent to `"$ref"`) have been performed on a copy of the target -* The resulting target value has been associated with the reference source in some way that the tool can easily use when needed - -A reference is said to be _removed_ if it has been replaced by its (possibly modified) target. - -Reference resolution usually preserves the referencing information such as the URI used to identify the target, while reference removal usually discards it. In many cases this is not significant, except that not knowing how the parsed OAD relates to the references in your JSON or YAML document may make debugging more difficult. - -While plain JSON documents form a [tree](https://en.wikipedia.org/wiki/Tree_%28data_structure%29) structure, an OpenAPI Description with resolved references is not _necessarily_ a tree, but a [graph](https://en.wikipedia.org/wiki/Graph_%28abstract_data_type%29). - -Tools that resolve references in-memory and work with the graph structure can process all OADs. Tools that rely on reference removal, either as part of the tool or by a separate pre-processing tool, can only support OADs that form trees. - -## Is anything being done to reduce this complexity in future OpenAPI Specifications? - -Yes! - -The [Moonwalk project](https://github.com/OAI/sig-moonwalk/discussions) is considering a different approach that imports complete documents, associates them with namespaces, and only supports referencing by component name (not `"$ref"`). A small example can be seen in the [Moonwalk deployments proposal](https://github.com/OAI/sig-moonwalk/blob/main/Deployments.md), and there are discussions around an [initial draft proposal for imports](https://github.com/OAI/sig-moonwalk/discussions/72) and a few ideas on [how to manage interactions with JSON Schema referencing](https://github.com/OAI/sig-moonwalk/discussions/73). - -The [proposed Workflows Specification](https://github.com/OAI/sig-workflows/blob/main/versions/1.0.0.md) is already using a `"sourceDescription"` field that is not unlike the Moonwalk proposal. - diff --git a/referencing/index.md b/referencing/index.md index e904697..9c69bc5 100644 --- a/referencing/index.md +++ b/referencing/index.md @@ -12,8 +12,14 @@ OpenAPI Referencing is a powerful tool. It allows managing document size and co However, the history of referencing and the `"$ref"` keyword is complex, leading to slightly different behavior depending on the version of the OpenAPI Specification (OAS) that you are using, and on where in your OpenAPI Description (OAD) the reference occurs. There are also other `"$ref"`-like keywords (`"operationRef"`, `"mapping"`) and behaviors (referencing by component name or operation ID) that are related but somewhat different. Referencing can also be challenging to use due to incomplete and inconsistent support across different tools, some of which require references to be pre-processed before they can read the OAD. -This guide explains how to use referencing, and what to look for when assessing the referencing support in your OpenAPI tools. +The resources in this section explain how to use referencing, and what to look for when assessing the referencing support in your OpenAPI tools. + +## The Future of References + +There are plans to reduce the complexity around references in future OpenAPI Specifications. + +The [Moonwalk project](https://github.com/OAI/sig-moonwalk/discussions) is considering a different approach that imports complete documents, associates them with namespaces, and only supports referencing by component name (not `"$ref"`). A small example can be seen in the [Moonwalk deployments proposal](https://github.com/OAI/sig-moonwalk/blob/main/Deployments.md), and there are discussions around an [initial draft proposal for imports](https://github.com/OAI/sig-moonwalk/discussions/72) and a few ideas on [how to manage interactions with JSON Schema referencing](https://github.com/OAI/sig-moonwalk/discussions/73). + +The [proposed Workflows Specification](https://github.com/OAI/sig-workflows/blob/main/versions/1.0.0.md) is already using a `"sourceDescription"` field that is not unlike the Moonwalk proposal. + -- [Referencing FAQ](faq) -- [What are References?](what) -- Resolving References (forthcoming) diff --git a/referencing/what.md b/referencing/overview.md similarity index 98% rename from referencing/what.md rename to referencing/overview.md index 401a173..9c8e49e 100644 --- a/referencing/what.md +++ b/referencing/overview.md @@ -1,7 +1,7 @@ --- layout: default -title: What Are References? -nav_order: 2 +title: References Overview +nav_order: 1 parent: Using References --- diff --git a/referencing/remove.md b/referencing/remove.md new file mode 100644 index 0000000..f8c952e --- /dev/null +++ b/referencing/remove.md @@ -0,0 +1,38 @@ +--- +layout: default +title: Remove References +nav_order: 3 +parent: Using References +--- + +# Remove References + +Sometimes it is useful to remove references in an OpenAPI Description (OAD). A common use case is working with tools that don't understand references well. This isn't always straightforward, however, and this page covers the main points to look out for. + +## Convert an OAD to a single-document form without any references + +Not all OADs can be converted but there are many cases where it is possible to do such a thing without problems. + +The difficult cases are: + +* Cyclic references in JSON Schema ***cannot*** be removed, although if the references cross documents, it is possible to reduce them to same-document references in a single document +* While [dynamic references](dynamic) in OAS 3.1 Schema Objects technically can be converted to static references, it can [grow the schema size exponentially](https://dl.acm.org/doi/10.1145/3632891) ([pdf](https://dl.acm.org/doi/pdf/10.1145/3632891)) +* OAS 3.1 Schema Objects, OAS 3.0 Reference Objects, and both OAS 3.0 and 3.1 Path Item Objects all allow keywords alongside a`"$ref"`, each with slightly different semantics; generic reference removal tools, particularly ones that predate OAS 3.1, may fail to preserve the behavior while removing references +* In OAS 3.1 Schema Objects that use `"$id"`, reference removal tools that are not aware of `"$id"` will resolve references impacted by the `"$id"` incorrectly +* The Link Object's `"operationRef"` cannot be replaced with an inline Operation Object (although if the target Operation Object has a unique-within-the-OAD `"operationId"`, then the `"operationRef"` can be replaced by the `"operationId"`) +* The Discriminator Object's `"mapping"` field forbids inline Schema Objects; URI-reference values can be replaced with names from the `"schemas"` section of the Components Object, but when the Discriminator Object is in a referenced document, there are ambiguities involved in how to resolve those names +* When using components published by 3rd parties such as standards organizations, tools might rely on reference URIs to identify standard components; while this usage is currently uncommon, it may become more common as OpenAPI is more broadly adopted by standards bodies + +If your OAD does not run afoul of any of the above difficulties, you _can_ produce a reference-less single-document OAD, and there are [tools](https://tools.openapis.org) that will do so (but see the caveats in the next section). + +## Convert a multi-document OAD into a single document + +It is possible to convert a multi-document OAD into a single document event if not all references can be removed. However, depending on exactly how your multi-document OAD is structured, this may be more complex than just removing references. Sadly, the reasons have to do with [painfully obscure and technical ambiguities](https://github.com/OAI/oascomply/blob/main/reports/processing-model-summary.md) regarding how multi-document OADs are intended to be parsed. The OpenAPI Initiative hopes to clarify this in future versions of the specification. + +* It is possible for a Link Object's `"operationId"` to point to an Operation Object in a Path Item Object that is never referenced; it is unclear how this case should handle it, which is one reason the OAS itself [advises against](https://spec.openapis.org/oas/v3.1.0#fixed-fields-16) using `"operationId"` in multi-document OADs +* Security Scheme Objects are never referenced, only named in Security Requirement Objects, and must be taken into account +* There is no standard place to put referenced Path Item Objects in OAS 3.0 +* The resolution of server configuration and security settings can be unclear in multi-document OADs + +Again, in many cases none of this is a problem, and there are [tools](https://tools.openapis.org) that will do what you want. + diff --git a/referencing/resolve.md b/referencing/resolve.md new file mode 100644 index 0000000..c22f9cc --- /dev/null +++ b/referencing/resolve.md @@ -0,0 +1,23 @@ +--- +layout: default +title: Resolve References +nav_order: 2 +parent: Using References +--- + +## Resolve References + +A reference is said to be _resolved_ within a tool if: + +* Its target has been identified +* Any modifications to the target required by the OAS (e.g. because of fields adjacent to `"$ref"`) have been performed on a copy of the target +* The resulting target value has been associated with the reference source in some way that the tool can easily use when needed + +A reference is said to be _removed_ if it has been replaced by its (possibly modified) target. + +Reference resolution usually preserves the referencing information such as the URI used to identify the target, while reference removal usually discards it. In many cases this is not significant, except that not knowing how the parsed OAD relates to the references in your JSON or YAML document may make debugging more difficult. + +While plain JSON documents form a [tree](https://en.wikipedia.org/wiki/Tree_%28data_structure%29) structure, an OpenAPI Description with resolved references is not _necessarily_ a tree, but a [graph](https://en.wikipedia.org/wiki/Graph_%28abstract_data_type%29). + +Tools that resolve references in-memory and work with the graph structure can process all OADs. Tools that rely on reference removal, either as part of the tool or by a separate pre-processing tool, can only support OADs that form trees. +