Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace PR workflow with a release draft. #211

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 0 additions & 49 deletions .github/workflows/build-single-file-specs.yml

This file was deleted.

45 changes: 45 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Build and Publish Spec

on:
push:
branches:
- main

jobs:
build:
name: Build and Publish
runs-on: ubuntu-latest
permissions: write-all

steps:
- name: Checkout the repo
uses: actions/checkout@v3

- name: Setup Node.js
uses: actions/setup-node@v2
with:
node-version: '20'

- name: Build
working-directory: ./tools
run: |-
mkdir -p ../build
npm install
export ROOT_PATH=../spec/OpenSearch.openapi.yaml
export OUTPUT_PATH=../build/OpenSearch.latest.yaml
npm run merge -- $ROOT_PATH $OUTPUT_PATH

- name: Extract Branch Name
id: branch
shell: bash
run: echo "name=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}" >> $GITHUB_OUTPUT

- uses: marvinpinto/[email protected]
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
automatic_release_tag: ${{ steps.branch.outputs.name }}
prerelease: true
title: OpenSearch OpenAPI Spec (${{ steps.branch.outputs.name }})
files: |
LICENSE.txt
build/*
15 changes: 8 additions & 7 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ jobs:
- name: Build Spec
working-directory: ./tools
run: |-
mkdir -p ../build
npm install
export ROOT_PATH=../spec/OpenSearch.openapi.yaml
export OUTPUT_PATH=../builds/OpenSearch.latest.yaml
export OUTPUT_PATH=../build/OpenSearch.latest.yaml
npm run merge -- $ROOT_PATH $OUTPUT_PATH
- name: Build and Run Docker Container
run: |
Expand All @@ -33,20 +34,20 @@ jobs:
curl -ks -u "admin:$OPENSEARCH_INITIAL_ADMIN_PASSWORD" https://localhost:9200/ | jq
- name: Dump and Compare API
run: |
curl -ks -u "admin:$OPENSEARCH_INITIAL_ADMIN_PASSWORD" https://localhost:9200/_plugins/api | jq > ./builds/OpenSearch.current.json
docker run --rm --mount type=bind,source=./builds,target=/builds openapitools/openapi-diff:latest /builds/OpenSearch.latest.yaml /builds/OpenSearch.current.json --json /builds/diff.json
curl -ks -u "admin:$OPENSEARCH_INITIAL_ADMIN_PASSWORD" https://localhost:9200/_plugins/api | jq > ./build/OpenSearch.current.json
docker run --rm --mount type=bind,source=./build,target=/build openapitools/openapi-diff:latest /build/OpenSearch.latest.yaml /build/OpenSearch.current.json --json /build/diff.json
- name: Show Diff
run: |
echo "-------- Missing APIs"
jq -r '.newEndpoints | group_by(.pathUrl)[] | "\(.[0].pathUrl): \([.[].method])"' builds/diff.json
jq -r '.newEndpoints | group_by(.pathUrl)[] | "\(.[0].pathUrl): \([.[].method])"' build/diff.json
echo "-------- Legacy APIs"
jq -r '.missingEndpoints | group_by(.pathUrl)[] | "\(.[0].pathUrl): \([.[].method])"' builds/diff.json
jq -r '.missingEndpoints | group_by(.pathUrl)[] | "\(.[0].pathUrl): \([.[].method])"' build/diff.json
- name: Gather Coverage
id: coverage
shell: bash
run: |
current=`docker run --rm -i mikefarah/yq:latest -r '.paths | keys | length' < builds/OpenSearch.latest.yaml`
total=`jq -r '.paths | keys | length' builds/OpenSearch.current.json`
current=`docker run --rm -i mikefarah/yq:latest -r '.paths | keys | length' < build/OpenSearch.latest.yaml`
total=`jq -r '.paths | keys | length' build/OpenSearch.current.json`
percent=$((current * 100 / total))
echo "API specs implemented for $current/$total ($percent%) APIs."
cat >>"$GITHUB_OUTPUT" <<EOL
Expand Down
4 changes: 2 additions & 2 deletions CLIENT_GENERATOR_GUIDE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Generate Clients for OpenSearch using OpenAPI Specification

OpenSearch Clients are available in multiple programming languages. The biggest challenge with this is keeping the clients up to date with the latest changes in OpenSearch. To solve this problem, we're automating the process of generating clients for OpenSearch using the OpenAPI specification. While OpenAPI comes with many well established off-the-shelf client generators for most languages, the OpenSearch APIs come with a lot of quirkiness that makes it near impossible to use these off-the-shelf generators. For this reason, we've opted to write our own client generators that is specifically tailored to the OpenSearch APIs. This document will walk you through the process of generating clients from [OpenSearch OpenAPI spec](builds/OpenSearch.latest.yaml), more specifically client API methods.
OpenSearch Clients are available in multiple programming languages. The biggest challenge with this is keeping the clients up to date with the latest changes in OpenSearch. To solve this problem, we're automating the process of generating clients for OpenSearch using the OpenAPI specification. While OpenAPI comes with many well established off-the-shelf client generators for most languages, the OpenSearch APIs come with a lot of quirkiness that makes it near impossible to use these off-the-shelf generators. For this reason, we've opted to write our own client generators that is specifically tailored to the OpenSearch APIs. This document will walk you through the process of generating clients from [the published OpenSearch OpenAPI spec](https://github.com/opensearch-project/opensearch-api-specification/releases), more specifically client API methods.

## The Grouping of API Operations
The OpenSearch clients, though written in different languages for different frameworks, share one distinct characteristic: API Operations are grouped into actions and namespaces. Operations serving the same functionality are often grouped together into a single API Action, which is represented in the client as an API method.
Expand All @@ -17,7 +17,7 @@ The **indices.get_field_mapping**, is consisted of 2 operations:

In a client, the `search` operations are grouped in to a single API method, `client.search(...)`, and the `indices.get_field_mapping` operations are grouped into a single API method, `get_field_mapping` of the `indices` namespace, `client.indices.get_field_mapping(...)`

In the [OpenAPI spec](./builds/OpenSearch.latest.yaml), this grouping is denoted by `x-operation-group` vendor extension in every operation definition. The value of this extension is the name of the API action (like `search` or `indices.get_field_mapping`). Operations with the same `x-operation-group` value are guaranteed to have the same query string parameters, response body, and request body (for PUT/POST/DELETE operations). Common path parameters are also guaranteed to be the same. The only differences between operations are the HTTP method and the path. With that in mind, below are rules on how to combine operations of different HTTP methods and path compositions.
In the [published OpenAPI spec](https://github.com/opensearch-project/opensearch-api-specification/releases), this grouping is denoted by `x-operation-group` vendor extension in every operation definition. The value of this extension is the name of the API action (like `search` or `indices.get_field_mapping`). Operations with the same `x-operation-group` value are guaranteed to have the same query string parameters, response body, and request body (for PUT/POST/DELETE operations). Common path parameters are also guaranteed to be the same. The only differences between operations are the HTTP method and the path. With that in mind, below are rules on how to combine operations of different HTTP methods and path compositions.

- If an operation is marked with `x-ignorable: "true"`, then ignore the operation. Such an operation has been deprecated and has been replaced by a newer one. As far as the clients are concerned, ignorable operations do not exist.
- If two operations have identical HTTP methods, but different paths: use the path that best matches the path parameters provided.
Expand Down
4 changes: 2 additions & 2 deletions PUBLISHING_SPECS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@

## Publishing OpenSearch API Specs

* The [build-single-file-specs](.github/workflows/build-single-file-specs.yml) workflow raises a PR for the changes to the [./build/OpenSearch.latest.yaml](builds/OpenSearch.latest.yaml) whenever a change is made to the [spec folder](./spec).
* The updated OpenAPI specs are hosted on GitHub pages at https://opensearch-project.github.io/opensearch-api-specification/.
* The [build](.github/workflows/build.yml) workflow publishes [a release draft](https://github.com/opensearch-project/opensearch-api-specification/releases) whenever a change is pushed to `main`.
* The updated OpenAPI specs are hosted on GitHub pages at https://opensearch-project.github.io/opensearch-api-specification/.
Loading
Loading