Skip to content

Commit

Permalink
feat(sdk): implement cloud.Bucket inflight method metadata for AWS ta…
Browse files Browse the repository at this point in the history
…rgets and Simulator (#4338)

Closes #4331
Closes #4330

This PR implements inflgiht method `Bucket.metadata` for AWS.
Other implementations are stubbed.

## Checklist

- [x] Title matches [Winglang's style guide](https://www.winglang.io/contributing/start-here/pull_requests#how-are-pull-request-titles-formatted)
- [x] Description explains motivation and solution
- [x] Tests added (always)
- [x] Docs updated (only required for features)
- [ ] Added `pr/e2e-full` label if this feature requires end-to-end testing

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Wing Cloud Contribution License](https://github.com/winglang/wing/blob/main/CONTRIBUTION_LICENSE.md)*.
  • Loading branch information
exoego authored Oct 11, 2023
1 parent cd97cfc commit d5fc08f
Show file tree
Hide file tree
Showing 14 changed files with 440 additions and 8 deletions.
75 changes: 75 additions & 0 deletions docs/docs/04-standard-library/01-cloud/bucket.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,7 @@ new cloud.Bucket(props?: BucketProps);
| <code><a href="#@winglang/sdk.cloud.IBucketClient.get">get</a></code> | Retrieve an object from the bucket. |
| <code><a href="#@winglang/sdk.cloud.IBucketClient.getJson">getJson</a></code> | Retrieve a Json object from the bucket. |
| <code><a href="#@winglang/sdk.cloud.IBucketClient.list">list</a></code> | Retrieve existing objects keys from the bucket. |
| <code><a href="#@winglang/sdk.cloud.IBucketClient.metadata">metadata</a></code> | Get the metadata of an object in the bucket. |
| <code><a href="#@winglang/sdk.cloud.IBucketClient.publicUrl">publicUrl</a></code> | Returns a url to the given file. |
| <code><a href="#@winglang/sdk.cloud.IBucketClient.put">put</a></code> | Put an object in the bucket. |
| <code><a href="#@winglang/sdk.cloud.IBucketClient.putJson">putJson</a></code> | Put a Json object in the bucket. |
Expand Down Expand Up @@ -401,6 +402,22 @@ Limits the response to keys that begin with the specified prefix.

---

##### `metadata` <a name="metadata" id="@winglang/sdk.cloud.IBucketClient.metadata"></a>

```wing
inflight metadata(key: str): ObjectMetadata
```

Get the metadata of an object in the bucket.

###### `key`<sup>Required</sup> <a name="key" id="@winglang/sdk.cloud.IBucketClient.metadata.parameter.key"></a>

- *Type:* str

Key of the object.

---

##### `publicUrl` <a name="publicUrl" id="@winglang/sdk.cloud.IBucketClient.publicUrl"></a>

```wing
Expand Down Expand Up @@ -723,6 +740,64 @@ Whether the bucket's objects should be publicly accessible.

---

### ObjectMetadata <a name="ObjectMetadata" id="@winglang/sdk.cloud.ObjectMetadata"></a>

Metadata of a bucket object.

#### Initializer <a name="Initializer" id="@winglang/sdk.cloud.ObjectMetadata.Initializer"></a>

```wing
bring cloud;
let ObjectMetadata = cloud.ObjectMetadata{ ... };
```

#### Properties <a name="Properties" id="Properties"></a>

| **Name** | **Type** | **Description** |
| --- | --- | --- |
| <code><a href="#@winglang/sdk.cloud.ObjectMetadata.property.lastModified">lastModified</a></code> | <code><a href="#@winglang/sdk.std.Datetime">datetime</a></code> | The time the object was last modified. |
| <code><a href="#@winglang/sdk.cloud.ObjectMetadata.property.size">size</a></code> | <code>num</code> | The size of the object in bytes. |
| <code><a href="#@winglang/sdk.cloud.ObjectMetadata.property.contentType">contentType</a></code> | <code>str</code> | The content type of the object, if it is known. |

---

##### `lastModified`<sup>Required</sup> <a name="lastModified" id="@winglang/sdk.cloud.ObjectMetadata.property.lastModified"></a>

```wing
lastModified: datetime;
```

- *Type:* <a href="#@winglang/sdk.std.Datetime">datetime</a>

The time the object was last modified.

---

##### `size`<sup>Required</sup> <a name="size" id="@winglang/sdk.cloud.ObjectMetadata.property.size"></a>

```wing
size: num;
```

- *Type:* num

The size of the object in bytes.

---

##### `contentType`<sup>Optional</sup> <a name="contentType" id="@winglang/sdk.cloud.ObjectMetadata.property.contentType"></a>

```wing
contentType: str;
```

- *Type:* str

The content type of the object, if it is known.

---

### SignedUrlOptions <a name="SignedUrlOptions" id="@winglang/sdk.cloud.SignedUrlOptions"></a>

Interface for signed url options.
Expand Down
17 changes: 17 additions & 0 deletions examples/tests/sdk_tests/bucket/metadata.test.w
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
bring cloud;

let b = new cloud.Bucket();

test "metadata" {
b.put("test1.txt", "Foo");

assert(b.metadata("test1.txt").size == 3);
assert(b.metadata("test1.txt").contentType == "application/octet-stream");
assert(b.metadata("test1.txt").lastModified.year >= 2023);

try {
b.metadata("no-such-file.txt").lastModified;
} catch e {
assert(e == "Object does not exist (key=no-such-file.txt).");
}
}
12 changes: 12 additions & 0 deletions libs/wingc/src/lsp/snapshots/completions/capture_in_test.snap
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,18 @@ source: libs/wingc/src/lsp/completions.rs
command:
title: triggerParameterHints
command: editor.action.triggerParameterHints
- label: metadata
kind: 2
detail: "inflight (key: str): ObjectMetadata"
documentation:
kind: markdown
value: "```wing\ninflight metadata: inflight (key: str): ObjectMetadata\n```\n---\nGet the metadata of an object in the bucket.\n\n\n*@Throws* *if there is no object with the given key.*"
sortText: ff|metadata
insertText: metadata($0)
insertTextFormat: 2
command:
title: triggerParameterHints
command: editor.action.triggerParameterHints
- label: publicUrl
kind: 2
detail: "inflight (key: str): str"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,12 @@ source: libs/wingc/src/lsp/completions.rs
kind: markdown
value: "```wing\nstruct GetSecretValueOptions\n```\n---\nOptions when getting a secret value.\n### Fields\n- `cache?` — Whether to cache the value."
sortText: hh|GetSecretValueOptions
- label: ObjectMetadata
kind: 22
documentation:
kind: markdown
value: "```wing\nstruct ObjectMetadata\n```\n---\nMetadata of a bucket object.\n### Fields\n- `contentType?` — The content type of the object, if it is known.\n- `lastModified` — The time the object was last modified.\n- `size` — The size of the object in bytes."
sortText: hh|ObjectMetadata
- label: OnDeployProps
kind: 22
documentation:
Expand Down Expand Up @@ -317,7 +323,7 @@ source: libs/wingc/src/lsp/completions.rs
kind: 8
documentation:
kind: markdown
value: "```wing\ninterface IBucketClient\n```\n---\nInflight interface for `Bucket`.\n### Methods\n- `delete` — Delete an existing object using a key from the bucket.\n- `exists` — Check if an object exists in the bucket.\n- `get` — Retrieve an object from the bucket.\n- `getJson` — Retrieve a Json object from the bucket.\n- `list` — Retrieve existing objects keys from the bucket.\n- `publicUrl` — Returns a url to the given file.\n- `put` — Put an object in the bucket.\n- `putJson` — Put a Json object in the bucket.\n- `signedUrl` — Returns a signed url to the given file.\n- `tryDelete` — Delete an object from the bucket if it exists.\n- `tryGet` — Get an object from the bucket if it exists.\n- `tryGetJson` — Gets an object from the bucket if it exists, parsing it as Json."
value: "```wing\ninterface IBucketClient\n```\n---\nInflight interface for `Bucket`.\n### Methods\n- `delete` — Delete an existing object using a key from the bucket.\n- `exists` — Check if an object exists in the bucket.\n- `get` — Retrieve an object from the bucket.\n- `getJson` — Retrieve a Json object from the bucket.\n- `list` — Retrieve existing objects keys from the bucket.\n- `metadata` — Get the metadata of an object in the bucket.\n- `publicUrl` — Returns a url to the given file.\n- `put` — Put an object in the bucket.\n- `putJson` — Put a Json object in the bucket.\n- `signedUrl` — Returns a signed url to the given file.\n- `tryDelete` — Delete an object from the bucket if it exists.\n- `tryGet` — Get an object from the bucket if it exists.\n- `tryGetJson` — Gets an object from the bucket if it exists, parsing it as Json."
sortText: ii|IBucketClient
- label: IBucketEventHandler
kind: 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,12 @@ source: libs/wingc/src/lsp/completions.rs
kind: markdown
value: "```wing\nstruct GetSecretValueOptions\n```\n---\nOptions when getting a secret value.\n### Fields\n- `cache?` — Whether to cache the value."
sortText: hh|GetSecretValueOptions
- label: ObjectMetadata
kind: 22
documentation:
kind: markdown
value: "```wing\nstruct ObjectMetadata\n```\n---\nMetadata of a bucket object.\n### Fields\n- `contentType?` — The content type of the object, if it is known.\n- `lastModified` — The time the object was last modified.\n- `size` — The size of the object in bytes."
sortText: hh|ObjectMetadata
- label: OnDeployProps
kind: 22
documentation:
Expand Down Expand Up @@ -317,7 +323,7 @@ source: libs/wingc/src/lsp/completions.rs
kind: 8
documentation:
kind: markdown
value: "```wing\ninterface IBucketClient\n```\n---\nInflight interface for `Bucket`.\n### Methods\n- `delete` — Delete an existing object using a key from the bucket.\n- `exists` — Check if an object exists in the bucket.\n- `get` — Retrieve an object from the bucket.\n- `getJson` — Retrieve a Json object from the bucket.\n- `list` — Retrieve existing objects keys from the bucket.\n- `publicUrl` — Returns a url to the given file.\n- `put` — Put an object in the bucket.\n- `putJson` — Put a Json object in the bucket.\n- `signedUrl` — Returns a signed url to the given file.\n- `tryDelete` — Delete an object from the bucket if it exists.\n- `tryGet` — Get an object from the bucket if it exists.\n- `tryGetJson` — Gets an object from the bucket if it exists, parsing it as Json."
value: "```wing\ninterface IBucketClient\n```\n---\nInflight interface for `Bucket`.\n### Methods\n- `delete` — Delete an existing object using a key from the bucket.\n- `exists` — Check if an object exists in the bucket.\n- `get` — Retrieve an object from the bucket.\n- `getJson` — Retrieve a Json object from the bucket.\n- `list` — Retrieve existing objects keys from the bucket.\n- `metadata` — Get the metadata of an object in the bucket.\n- `publicUrl` — Returns a url to the given file.\n- `put` — Put an object in the bucket.\n- `putJson` — Put a Json object in the bucket.\n- `signedUrl` — Returns a signed url to the given file.\n- `tryDelete` — Delete an object from the bucket if it exists.\n- `tryGet` — Get an object from the bucket if it exists.\n- `tryGetJson` — Gets an object from the bucket if it exists, parsing it as Json."
sortText: ii|IBucketClient
- label: IBucketEventHandler
kind: 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,12 @@ source: libs/wingc/src/lsp/completions.rs
kind: markdown
value: "```wing\nstruct GetSecretValueOptions\n```\n---\nOptions when getting a secret value.\n### Fields\n- `cache?` — Whether to cache the value."
sortText: hh|GetSecretValueOptions
- label: ObjectMetadata
kind: 22
documentation:
kind: markdown
value: "```wing\nstruct ObjectMetadata\n```\n---\nMetadata of a bucket object.\n### Fields\n- `contentType?` — The content type of the object, if it is known.\n- `lastModified` — The time the object was last modified.\n- `size` — The size of the object in bytes."
sortText: hh|ObjectMetadata
- label: OnDeployProps
kind: 22
documentation:
Expand Down Expand Up @@ -317,7 +323,7 @@ source: libs/wingc/src/lsp/completions.rs
kind: 8
documentation:
kind: markdown
value: "```wing\ninterface IBucketClient\n```\n---\nInflight interface for `Bucket`.\n### Methods\n- `delete` — Delete an existing object using a key from the bucket.\n- `exists` — Check if an object exists in the bucket.\n- `get` — Retrieve an object from the bucket.\n- `getJson` — Retrieve a Json object from the bucket.\n- `list` — Retrieve existing objects keys from the bucket.\n- `publicUrl` — Returns a url to the given file.\n- `put` — Put an object in the bucket.\n- `putJson` — Put a Json object in the bucket.\n- `signedUrl` — Returns a signed url to the given file.\n- `tryDelete` — Delete an object from the bucket if it exists.\n- `tryGet` — Get an object from the bucket if it exists.\n- `tryGetJson` — Gets an object from the bucket if it exists, parsing it as Json."
value: "```wing\ninterface IBucketClient\n```\n---\nInflight interface for `Bucket`.\n### Methods\n- `delete` — Delete an existing object using a key from the bucket.\n- `exists` — Check if an object exists in the bucket.\n- `get` — Retrieve an object from the bucket.\n- `getJson` — Retrieve a Json object from the bucket.\n- `list` — Retrieve existing objects keys from the bucket.\n- `metadata` — Get the metadata of an object in the bucket.\n- `publicUrl` — Returns a url to the given file.\n- `put` — Put an object in the bucket.\n- `putJson` — Put a Json object in the bucket.\n- `signedUrl` — Returns a signed url to the given file.\n- `tryDelete` — Delete an object from the bucket if it exists.\n- `tryGet` — Get an object from the bucket if it exists.\n- `tryGetJson` — Gets an object from the bucket if it exists, parsing it as Json."
sortText: ii|IBucketClient
- label: IBucketEventHandler
kind: 8
Expand Down
26 changes: 25 additions & 1 deletion libs/wingsdk/src/cloud/bucket.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { Topic } from "./topic";
import { fqnForType } from "../constants";
import { App } from "../core";
import { convertBetweenHandlers } from "../shared/convert";
import { Json, IResource, Node, Resource, Duration } from "../std";
import { Json, IResource, Node, Resource, Datetime, Duration } from "../std";

/**
* Global identifier for `Bucket`.
Expand Down Expand Up @@ -68,6 +68,7 @@ export abstract class Bucket extends Resource {
BucketInflightMethods.TRY_GET_JSON,
BucketInflightMethods.TRY_DELETE,
BucketInflightMethods.SIGNED_URL,
BucketInflightMethods.METADATA,
];
}

Expand Down Expand Up @@ -363,6 +364,28 @@ export interface IBucketClient {
* @inflight
*/
signedUrl(key: string, options?: SignedUrlOptions): Promise<string>;

/**
* Get the metadata of an object in the bucket.
* @param key Key of the object.
* @Throws if there is no object with the given key.
* @inflight
*/
metadata(key: string): Promise<ObjectMetadata>;
}

/**
* Metadata of a bucket object.
*/
export interface ObjectMetadata {
/** The size of the object in bytes. */
readonly size: number;

/** The time the object was last modified. */
readonly lastModified: Datetime;

/** The content type of the object, if it is known. */
readonly contentType?: string;
}

/**
Expand Down Expand Up @@ -467,4 +490,5 @@ export enum BucketInflightMethods {
TRY_DELETE = "tryDelete",

SIGNED_URL = "signedUrl",
METADATA = "metadata",
}
36 changes: 34 additions & 2 deletions libs/wingsdk/src/shared-aws/bucket.inflight.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,19 @@ import {
GetPublicAccessBlockCommandOutput,
S3Client,
GetObjectOutput,
NotFound,
NoSuchKey,
__Client,
} from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { BucketDeleteOptions, IBucketClient, SignedUrlOptions } from "../cloud";
import { Json } from "../std";
import {
BucketDeleteOptions,
IBucketClient,
SignedUrlOptions,
ObjectMetadata,
} from "../cloud";
import { Datetime, Json } from "../std";

export class BucketClient implements IBucketClient {
constructor(
private readonly bucketName: string,
Expand Down Expand Up @@ -298,6 +305,31 @@ export class BucketClient implements IBucketClient {
}
}

/**
* Get the metadata of an object in the bucket.
* @param key Key of the object.
*/
public async metadata(key: string): Promise<ObjectMetadata> {
const command = new HeadObjectCommand({
Bucket: this.bucketName,
Key: key,
});
try {
const resp = await this.s3Client.send(command);
return {
contentType: resp.ContentType,
lastModified: Datetime.fromIso(resp.LastModified!.toISOString()),
size: resp.ContentLength!,
};
} catch (error) {
// 403 is thrown if s3:ListObject is not granted.
if (error instanceof NotFound || (error as Error).name === "403") {
throw new Error(`Object does not exist (key=${key}).`);
}
throw error;
}
}

private async getLocation(): Promise<string> {
const command = new GetBucketLocationCommand({
Bucket: this.bucketName,
Expand Down
1 change: 1 addition & 0 deletions libs/wingsdk/src/shared-aws/permissions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ export function calculateBucketPermissions(
if (
ops.includes(cloud.BucketInflightMethods.GET) ||
ops.includes(cloud.BucketInflightMethods.GET_JSON) ||
ops.includes(cloud.BucketInflightMethods.METADATA) ||
ops.includes(cloud.BucketInflightMethods.LIST) ||
ops.includes(cloud.BucketInflightMethods.TRY_GET) ||
ops.includes(cloud.BucketInflightMethods.TRY_GET_JSON) ||
Expand Down
31 changes: 30 additions & 1 deletion libs/wingsdk/src/target-sim/bucket.inflight.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,13 @@ import {
IBucketClient,
ITopicClient,
SignedUrlOptions,
ObjectMetadata,
} from "../cloud";
import {
ISimulatorContext,
ISimulatorResourceInstance,
} from "../simulator/simulator";
import { Json } from "../std";
import { Datetime, Json } from "../std";

export class Bucket implements IBucketClient, ISimulatorResourceInstance {
private readonly objectKeys: Set<string>;
Expand Down Expand Up @@ -221,6 +222,34 @@ export class Bucket implements IBucketClient, ISimulatorResourceInstance {
});
}

/**
* Get the metadata of an object in the bucket.
* @param key Key of the object.
* @throws if the object does not exist.
*/
public async metadata(key: string): Promise<ObjectMetadata> {
return this.context.withTrace({
message: `Metadata (key=${key}).`,
activity: async () => {
const hash = this.hashKey(key);
const filename = join(this._fileDir, hash);
try {
const filestat = await fs.promises.stat(filename, {
bigint: false,
});
return {
size: filestat.size,
lastModified: Datetime.fromIso(filestat.mtime.toISOString()),
// fs does not provide a way to get the content-type
contentType: "application/octet-stream",
};
} catch (e) {
throw new Error(`Object does not exist (key=${key}).`);
}
},
});
}

private async addFile(key: string, value: string): Promise<void> {
const actionType: BucketEventType = this.objectKeys.has(key)
? BucketEventType.UPDATE
Expand Down
16 changes: 15 additions & 1 deletion libs/wingsdk/src/target-tf-azure/bucket.inflight.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,12 @@ import {
BlobServiceClient,
ContainerClient,
} from "@azure/storage-blob";
import { BucketDeleteOptions, IBucketClient, SignedUrlOptions } from "../cloud";
import {
BucketDeleteOptions,
IBucketClient,
SignedUrlOptions,
ObjectMetadata,
} from "../cloud";
import { Json } from "../std";

export class BucketClient implements IBucketClient {
Expand Down Expand Up @@ -225,6 +230,15 @@ export class BucketClient implements IBucketClient {
);
}

/**
* Get the metadata of an object in the bucket.
* @throws if the object does not exist.
* @param key Key of the object.
*/
public async metadata(key: string): Promise<ObjectMetadata> {
return Promise.reject(`metadata is not implemented: (key=${key})`);
}

/**
* Required helper function for node js only.
*
Expand Down
Loading

0 comments on commit d5fc08f

Please sign in to comment.