Skip to content

Commit

Permalink
kind/feat: passing artifacts between tasks
Browse files Browse the repository at this point in the history
  • Loading branch information
@ericzzzzzzz
ericzzzzzzz committed May 28, 2024
1 parent 14b416b commit 86b0a21
Show file tree
Hide file tree
Showing 35 changed files with 1,745 additions and 130 deletions.
358 changes: 312 additions & 46 deletions docs/artifacts.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ weight: 201
# Artifacts

- [Overview](#overview)
- [Artifact Provenance Data](#passing-artifacts-between-steps)
- [Artifact Provenance Data](#artifact-provenance-data)
- [Passing Artifacts between Steps](#passing-artifacts-between-steps)
- [Passing Artifacts between Tasks](#passing-artifacts-between-tasks)



Expand Down Expand Up @@ -104,6 +105,54 @@ spec:
EOF
```
The content is written by the `Step` to a file `$(artifacts.path)`:

```yaml
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
generateName: step-artifacts-
spec:
taskSpec:
description: |
A simple task that populates artifacts to TaskRun stepState
steps:
- name: artifacts-producer
image: bash:latest
script: |
cat > $(artifacts.path) << EOF
{
"inputs":[
{
"name":"source",
"values":[
{
"uri":"pkg:github/package-url/purl-spec@244fd47e07d1004f0aed9c",
"digest":{
"sha256":"b35cacccfdb1e24dc497d15d553891345fd155713ffe647c281c583269eaaae0"
}
}
]
}
],
"outputs":[
{
"name":"image",
"values":[
{
"uri":"pkg:oci/nginx:stable-alpine3.17-slim?repository_url=docker.io/library",
"digest":{
"sha256":"df85b9e3983fe2ce20ef76ad675ecf435cc99fc9350adc54fa230bae8c32ce48",
"sha1":"95588b8f34c31eb7d62c92aaa4e6506639b06ef2"
}
}
]
}
]
}
EOF
```

It is recommended to use [purl format](https://github.com/package-url/purl-spec/blob/master/PURL-SPECIFICATION.rst) for artifacts uri as shown in the example.

### Passing Artifacts between Steps
Expand Down Expand Up @@ -182,62 +231,279 @@ The resolved value of `$(steps.<step-name>.outputs.<artifact-category-name>)` or


Upon resolution and execution of the `TaskRun`, the `Status` will look something like:
```yaml
"steps": [

```json
{
"artifacts": {
"inputs": [
{
"container": "step-artifacts-producer",
"imageID": "docker.io/library/bash@sha256:5353512b79d2963e92a2b97d9cb52df72d32f94661aa825fcfa0aede73304743",
"inputs": [
"name": "source",
"values": [
{
"name": "source",
"values": [
{
"digest": {
"sha256": "b35cacccfdb1e24dc497d15d553891345fd155713ffe647c281c583269eaaae0"
},
"uri":"pkg:github/package-url/purl-spec@244fd47e07d1004f0aed9c",
}
]
"digest": {
"sha256": "b35cacccfdb1e24dc497d15d553891345fd155713ffe647c281c583269eaaae0"
},
"uri": "pkg:github/package-url/purl-spec@244fd47e07d1004f0aed9c"
}
],
"name": "artifacts-producer",
"outputs": [
]
}
],
"outputs": [
{
"name": "image",
"values": [
{
"name": "image",
"values": [
{
"digest": {
"sha1": "95588b8f34c31eb7d62c92aaa4e6506639b06ef2",
"sha256": "df85b9e3983fe2ce20ef76ad675ecf435cc99fc9350adc54fa230bae8c32ce48"
},
"uri":"pkg:oci/nginx:stable-alpine3.17-slim?repository_url=docker.io/library",
}
]
"digest": {
"sha1": "95588b8f34c31eb7d62c92aaa4e6506639b06ef2",
"sha256": "df85b9e3983fe2ce20ef76ad675ecf435cc99fc9350adc54fa230bae8c32ce48"
},
"uri": "pkg:oci/nginx:stable-alpine3.17-slim?repository_url=docker.io/library"
}
],
"terminated": {
"containerID": "containerd://010f02d103d1db48531327a1fe09797c87c1d50b6a216892319b3af93e0f56e7",
"exitCode": 0,
"finishedAt": "2024-03-18T17:05:06Z",
"message": "...",
"reason": "Completed",
"startedAt": "2024-03-18T17:05:06Z"
},
"terminationReason": "Completed"
]
}
]
},
"steps": [
{
"container": "step-artifacts-producer",
"imageID": "docker.io/library/bash@sha256:5353512b79d2963e92a2b97d9cb52df72d32f94661aa825fcfa0aede73304743",
"inputs": [
{
"name": "source",
"values": [
{
"digest": {
"sha256": "b35cacccfdb1e24dc497d15d553891345fd155713ffe647c281c583269eaaae0"
},
"uri": "pkg:github/package-url/purl-spec@244fd47e07d1004f0aed9c"
}
]
}
],
"name": "artifacts-producer",
"outputs": [
{
"name": "image",
"values": [
{
"digest": {
"sha1": "95588b8f34c31eb7d62c92aaa4e6506639b06ef2",
"sha256": "df85b9e3983fe2ce20ef76ad675ecf435cc99fc9350adc54fa230bae8c32ce48"
},
"uri": "pkg:oci/nginx:stable-alpine3.17-slim?repository_url=docker.io/library"
}
]
}
],
"terminated": {
"containerID": "containerd://010f02d103d1db48531327a1fe09797c87c1d50b6a216892319b3af93e0f56e7",
"exitCode": 0,
"finishedAt": "2024-03-18T17:05:06Z",
"message": "...",
"reason": "Completed",
"startedAt": "2024-03-18T17:05:06Z"
},
"terminationReason": "Completed"
},
{
"container": "step-artifacts-consumer",
"imageID": "docker.io/library/bash@sha256:5353512b79d2963e92a2b97d9cb52df72d32f94661aa825fcfa0aede73304743",
"name": "artifacts-consumer",
"terminated": {
"containerID": "containerd://42428aa7e5a507eba924239f213d185dd4bc0882b6f217a79e6792f7fec3586e",
"exitCode": 0,
"finishedAt": "2024-03-18T17:05:06Z",
"reason": "Completed",
"startedAt": "2024-03-18T17:05:06Z"
},
"terminationReason": "Completed"
}
]
}
```

### Passing Artifacts between Tasks
You can pass artifacts from one task to the another using:

- Specific Artifact: `$(tasks.<task-name>.inputs.<artifact-category-name>)` or `$(tasks.<task-name>.outputs.<artifact-category-name>)`
- Default (First) Artifact: `$(tasks.<task-name>.inputs)` or `$(tasks.<task-name>.outputs)` (if <artifact-category-name> is omitted)

The example below shows how to access the previous' task artifacts from another task in a pipeline

```yaml
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
generateName: pipelinerun-consume-tasks-artifacts
spec:
pipelineSpec:
tasks:
- name: produce-artifacts-task
taskSpec:
description: |
A simple task that produces artifacts
steps:
- name: produce-artifacts
image: bash:latest
script: |
#!/usr/bin/env bash
cat > $(artifacts.path) << EOF
{
"inputs":[
{
"name":"input-artifacts",
"values":[
{
"uri":"pkg:example.github.com/inputs",
"digest":{
"sha256":"b35cacccfdb1e24dc497d15d553891345fd155713ffe647c281c583269eaaae0"
}
}
]
}
],
"outputs":[
{
"name":"image",
"values":[
{
"uri":"pkg:github/package-url/purl-spec@244fd47e07d1004f0aed9c",
"digest":{
"sha256":"df85b9e3983fe2ce20ef76ad675ecf435cc99fc9350adc54fa230bae8c32ce48",
"sha1":"95588b8f34c31eb7d62c92aaa4e6506639b06ef2"
}
}
]
}
]
}
EOF
- name: consume-artifacts
runAfter:
- produce-artifacts-task
taskSpec:
steps:
- name: artifacts-consumer-python
image: python:latest
script: |
#!/usr/bin/env python3
import json
data = json.loads('$(tasks.produce-artifacts-task.outputs)')
if data[0]['uri'] != "pkg:github/package-url/purl-spec@244fd47e07d1004f0aed9c":
exit(1)
```


Similar to Step Artifacts. The resolved value of `$(tasks.<task-name>.outputs.<artifact-category-name>)` or `$(tasks.<task-name>.outputs)` is the values of an artifact. For this example,
`$(tasks.produce-artifacts-task.outputs)` is resolved to
```json
[
{
"uri":"pkg:example.github.com/inputs",
"digest":{
"sha256":"b35cacccfdb1e24dc497d15d553891345fd155713ffe647c281c583269eaaae0"
}
}
]
```
Upon resolution and execution of the `TaskRun`, the `Status` will look something like:
```json
{
"artifacts": {
"inputs": [
{
"name": "input-artifacts",
"values": [
{
"digest": {
"sha256": "b35cacccfdb1e24dc497d15d553891345fd155713ffe647c281c583269eaaae0"
},
"uri": "pkg:example.github.com/inputs"
}
]
}
],
"outputs": [
{
"name": "image",
"values": [
{
"digest": {
"sha1": "95588b8f34c31eb7d62c92aaa4e6506639b06ef2",
"sha256": "df85b9e3983fe2ce20ef76ad675ecf435cc99fc9350adc54fa230bae8c32ce48"
},
"uri": "pkg:github/package-url/purl-spec@244fd47e07d1004f0aed9c"
}
]
}
]
},
"completionTime": "2024-05-28T14:10:58Z",
"conditions": [
{
"container": "step-artifacts-consumer",
"imageID": "docker.io/library/bash@sha256:5353512b79d2963e92a2b97d9cb52df72d32f94661aa825fcfa0aede73304743",
"name": "artifacts-consumer",
"lastTransitionTime": "2024-05-28T14:10:58Z",
"message": "All Steps have completed executing",
"reason": "Succeeded",
"status": "True",
"type": "Succeeded"
}
],
"podName": "pipelinerun-consume-tasks-a41ee44e4f964e95adfd3aea417d52f90-pod",
"provenance": {
"featureFlags": {
"AwaitSidecarReadiness": true,
"Coschedule": "workspaces",
"DisableAffinityAssistant": false,
"DisableCredsInit": false,
"DisableInlineSpec": "",
"EnableAPIFields": "beta",
"EnableArtifacts": true,
"EnableCELInWhenExpression": false,
"EnableConciseResolverSyntax": false,
"EnableKeepPodOnCancel": false,
"EnableParamEnum": false,
"EnableProvenanceInStatus": true,
"EnableStepActions": true,
"EnableTektonOCIBundles": false,
"EnforceNonfalsifiability": "none",
"MaxResultSize": 4096,
"RequireGitSSHSecretKnownHosts": false,
"ResultExtractionMethod": "termination-message",
"RunningInEnvWithInjectedSidecars": true,
"ScopeWhenExpressionsToTask": false,
"SendCloudEventsForRuns": false,
"SetSecurityContext": false,
"VerificationNoMatchPolicy": "ignore"
}
},
"startTime": "2024-05-28T14:10:48Z",
"steps": [
{
"container": "step-produce-artifacts",
"imageID": "docker.io/library/bash@sha256:23f90212fd89e4c292d7b41386ef1a6ac2b8a02bbc6947680bfe184cbc1a2899",
"name": "produce-artifacts",
"terminated": {
"containerID": "containerd://42428aa7e5a507eba924239f213d185dd4bc0882b6f217a79e6792f7fec3586e",
"containerID": "containerd://1291ce07b175a7897beee6ba62eaa1528427bacb1f76b31435eeba68828c445a",
"exitCode": 0,
"finishedAt": "2024-03-18T17:05:06Z",
"finishedAt": "2024-05-28T14:10:57Z",
"message": "...",
"reason": "Completed",
"startedAt": "2024-03-18T17:05:06Z"
"startedAt": "2024-05-28T14:10:57Z"
},
"terminationReason": "Completed"
}
],
```
"taskSpec": {
"description": "A simple task that produces artifacts\n",
"steps": [
{
"computeResources": {},
"image": "bash:latest",
"name": "produce-artifacts",
"script": "#!/usr/bin/env bash\ncat > /tekton/artifacts/provenance.json << EOF\n{\n \"inputs\":[\n {\n \"name\":\"input-artifacts\",\n \"values\":[\n {\n \"uri\":\"pkg:example.github.com/inputs\",\n \"digest\":{\n \"sha256\":\"b35cacccfdb1e24dc497d15d553891345fd155713ffe647c281c583269eaaae0\"\n }\n }\n ]\n }\n ],\n \"outputs\":[\n {\n \"name\":\"image\",\n \"values\":[\n {\n \"uri\":\"pkg:github/package-url/purl-spec@244fd47e07d1004f0aed9c\",\n \"digest\":{\n \"sha256\":\"df85b9e3983fe2ce20ef76ad675ecf435cc99fc9350adc54fa230bae8c32ce48\",\n \"sha1\":\"95588b8f34c31eb7d62c92aaa4e6506639b06ef2\"\n }\n }\n ]\n }\n ]\n}\nEOF\n"
}
]
}
}
```
Loading

0 comments on commit 86b0a21

Please sign in to comment.