Skip to content

Commit

Permalink
feat(sdk)!: remove cloud.Queue initialMessages (#3780)
Browse files Browse the repository at this point in the history
Removes the `initialMessages` property from `cloud.Queue` to reduce API surface area in the standard library. This capability has only worked on the `sim` target to date, and it's possible to easily achieve the capability (across clouds) using `cloud.OnDeploy`:

```js
bring cloud;

let queue = new cloud.Queue();
new cloud.OnDeploy(inflight () => {
  queue.push("hello");
  queue.push("world");
});
```

This style of API would have more merit if it's something we could provided an optimized implementation for, but it doesn't appear there are dedicated Terraform resources available that represent AWS SQS messages or Google PubSub items. `OnDeploy` also offers more flexibility since it allows the user to dynamically calculate the queue's initial messages based on the results of other inflight operations.

Closes #281.

BREAKING CHANGE: The field `initialMessages` of `cloud.Queue` has been removed. Instead, use the `cloud.OnDeploy` resource to perform any deploy-time initialization logic.

## Checklist

- [x] Title matches [Winglang's style guide](https://www.winglang.io/contributing/start-here/pull_requests#how-are-pull-request-titles-formatted)
- [x] Description explains motivation and solution
- [x] Tests added (always)
- [x] Docs updated (only required for features)
- [ ] Added `pr/e2e-full` label if this feature requires end-to-end testing

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Wing Cloud Contribution License](https://github.com/winglang/wing/blob/main/CONTRIBUTION_LICENSE.md)*.
  • Loading branch information
Chriscbr authored Aug 11, 2023
1 parent 546663d commit 9399564
Show file tree
Hide file tree
Showing 13 changed files with 171 additions and 73 deletions.
14 changes: 0 additions & 14 deletions docs/docs/04-standard-library/01-cloud/queue.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,25 +239,11 @@ let QueueProps = cloud.QueueProps{ ... };

| **Name** | **Type** | **Description** |
| --- | --- | --- |
| <code><a href="#@winglang/sdk.cloud.QueueProps.property.initialMessages">initialMessages</a></code> | <code>MutArray&lt;str&gt;</code> | Initialize the queue with a set of messages. |
| <code><a href="#@winglang/sdk.cloud.QueueProps.property.retentionPeriod">retentionPeriod</a></code> | <code><a href="#@winglang/sdk.std.Duration">duration</a></code> | How long a queue retains a message. |
| <code><a href="#@winglang/sdk.cloud.QueueProps.property.timeout">timeout</a></code> | <code><a href="#@winglang/sdk.std.Duration">duration</a></code> | How long a queue's consumers have to process a message. |

---

##### `initialMessages`<sup>Optional</sup> <a name="initialMessages" id="@winglang/sdk.cloud.QueueProps.property.initialMessages"></a>

```wing
initialMessages: MutArray<str>;
```

- *Type:* MutArray&lt;str&gt;
- *Default:* []

Initialize the queue with a set of messages.

---

##### `retentionPeriod`<sup>Optional</sup> <a name="retentionPeriod" id="@winglang/sdk.cloud.QueueProps.property.retentionPeriod"></a>

```wing
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ source: libs/wingc/src/lsp/completions.rs
kind: 22
documentation:
kind: markdown
value: "```wing\nstruct QueueProps\n```\n---\nOptions for `Queue`.\n### Fields\n- `initialMessages?` — Initialize the queue with a set of messages.\n- `retentionPeriod?` — How long a queue retains a message.\n- `timeout?` — How long a queue's consumers have to process a message."
value: "```wing\nstruct QueueProps\n```\n---\nOptions for `Queue`.\n### Fields\n- `retentionPeriod?` — How long a queue retains a message.\n- `timeout?` — How long a queue's consumers have to process a message."
sortText: hh|QueueProps
- label: QueueSetConsumerProps
kind: 22
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ source: libs/wingc/src/lsp/completions.rs
kind: 22
documentation:
kind: markdown
value: "```wing\nstruct QueueProps\n```\n---\nOptions for `Queue`.\n### Fields\n- `initialMessages?` — Initialize the queue with a set of messages.\n- `retentionPeriod?` — How long a queue retains a message.\n- `timeout?` — How long a queue's consumers have to process a message."
value: "```wing\nstruct QueueProps\n```\n---\nOptions for `Queue`.\n### Fields\n- `retentionPeriod?` — How long a queue retains a message.\n- `timeout?` — How long a queue's consumers have to process a message."
sortText: hh|QueueProps
- label: QueueSetConsumerProps
kind: 22
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ source: libs/wingc/src/lsp/completions.rs
kind: 22
documentation:
kind: markdown
value: "```wing\nstruct QueueProps\n```\n---\nOptions for `Queue`.\n### Fields\n- `initialMessages?` — Initialize the queue with a set of messages.\n- `retentionPeriod?` — How long a queue retains a message.\n- `timeout?` — How long a queue's consumers have to process a message."
value: "```wing\nstruct QueueProps\n```\n---\nOptions for `Queue`.\n### Fields\n- `retentionPeriod?` — How long a queue retains a message.\n- `timeout?` — How long a queue's consumers have to process a message."
sortText: hh|QueueProps
- label: QueueSetConsumerProps
kind: 22
Expand Down
6 changes: 0 additions & 6 deletions libs/wingsdk/src/cloud/queue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,6 @@ export interface QueueProps {
* @default undefined
*/
readonly retentionPeriod?: Duration;

/**
* Initialize the queue with a set of messages.
* @default []
*/
readonly initialMessages?: string[];
}

/**
Expand Down
6 changes: 0 additions & 6 deletions libs/wingsdk/src/target-awscdk/queue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,6 @@ export class Queue extends cloud.Queue {
? Duration.seconds(props.retentionPeriod?.seconds)
: undefined,
});

if ((props.initialMessages ?? []).length) {
throw new Error(
"initialMessages not supported yet for AWS target - https://github.com/winglang/wing/issues/281"
);
}
}

public setConsumer(
Expand Down
8 changes: 0 additions & 8 deletions libs/wingsdk/src/target-sim/queue.inflight.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,6 @@ export class Queue
private readonly retentionPeriod: number;

constructor(props: QueueSchema["props"], context: ISimulatorContext) {
if (props.initialMessages) {
this.messages.push(
...props.initialMessages.map(
(message) => new QueueMessage(this.retentionPeriod, message)
)
);
}

this.timeout = props.timeout;
this.retentionPeriod = props.retentionPeriod;
this.intervalId = setInterval(() => this.processMessages(), 100); // every 0.1 seconds
Expand Down
4 changes: 0 additions & 4 deletions libs/wingsdk/src/target-sim/queue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ import { BaseResourceSchema } from "../testing/simulator";
export class Queue extends cloud.Queue implements ISimulatorResource {
private readonly timeout: Duration;
private readonly retentionPeriod: Duration;
private readonly initialMessages: string[] = [];
constructor(scope: Construct, id: string, props: cloud.QueueProps = {}) {
super(scope, id, props);

Expand All @@ -31,8 +30,6 @@ export class Queue extends cloud.Queue implements ISimulatorResource {
"Retention period must be greater than or equal to timeout"
);
}

this.initialMessages.push(...(props.initialMessages ?? []));
}

public setConsumer(
Expand Down Expand Up @@ -104,7 +101,6 @@ export class Queue extends cloud.Queue implements ISimulatorResource {
props: {
timeout: this.timeout.seconds,
retentionPeriod: this.retentionPeriod.seconds,
initialMessages: this.initialMessages,
},
attrs: {} as any,
};
Expand Down
2 changes: 0 additions & 2 deletions libs/wingsdk/src/target-sim/schema-resources.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,6 @@ export interface QueueSchema extends BaseResourceSchema {
readonly timeout: number;
/** How long a queue retains a message, in seconds */
readonly retentionPeriod: number;
/** Initial messages to be pushed to the queue. */
readonly initialMessages: string[];
};
}

Expand Down
6 changes: 0 additions & 6 deletions libs/wingsdk/src/target-tf-aws/queue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,6 @@ export class Queue extends cloud.Queue {
messageRetentionSeconds: props.retentionPeriod?.seconds,
name: ResourceNames.generateName(this, NAME_OPTS),
});

if ((props.initialMessages ?? []).length) {
throw new Error(
"initialMessages not supported yet for AWS target - https://github.com/winglang/wing/issues/281"
);
}
}

public setConsumer(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,6 @@ bucket: (function(env) {
"attrs": {},
"path": "root/HelloWorld/Queue",
"props": {
"initialMessages": [],
"retentionPeriod": 3600,
"timeout": 10,
},
Expand Down
148 changes: 134 additions & 14 deletions libs/wingsdk/test/target-sim/__snapshots__/queue.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ exports[`create a queue 1`] = `
"attrs": {},
"path": "root/my_queue",
"props": {
"initialMessages": [],
"retentionPeriod": 3600,
"timeout": 10,
},
Expand Down Expand Up @@ -136,7 +135,6 @@ async handle(message) {
"attrs": {},
"path": "root/my_queue",
"props": {
"initialMessages": [],
"retentionPeriod": 1,
"timeout": 2,
},
Expand Down Expand Up @@ -369,7 +367,6 @@ async handle(message) {
"attrs": {},
"path": "root/my_queue",
"props": {
"initialMessages": [],
"retentionPeriod": 3600,
"timeout": 1,
},
Expand Down Expand Up @@ -606,7 +603,6 @@ async handle(message) {
"attrs": {},
"path": "root/my_queue",
"props": {
"initialMessages": [],
"retentionPeriod": 3600,
"timeout": 1,
},
Expand Down Expand Up @@ -805,7 +801,6 @@ exports[`queue batch size of 2, purge the queue 2`] = `
"attrs": {},
"path": "root/my_queue",
"props": {
"initialMessages": [],
"retentionPeriod": 3600,
"timeout": 10,
},
Expand Down Expand Up @@ -867,8 +862,20 @@ exports[`queue with one subscriber, batch size of 5 1`] = `
"wingsdk.cloud.Function created.",
"wingsdk.cloud.Queue created.",
"wingsdk.sim.EventMapping created.",
"wingsdk.cloud.Function created.",
"Push (message=A).",
"Push (message=B).",
"Push (message=C).",
"Push (message=D).",
"Push (message=E).",
"Push (message=F).",
"Invoke (payload=\\"\\").",
"OnDeploy invoked.",
"wingsdk.cloud.OnDeploy created.",
"Sending messages (messages=[\\"A\\",\\"B\\",\\"C\\",\\"D\\",\\"E\\"], subscriber=sim-1).",
"Sending messages (messages=[\\"F\\"], subscriber=sim-1).",
"wingsdk.cloud.OnDeploy deleted.",
"wingsdk.cloud.Function deleted.",
"wingsdk.sim.EventMapping deleted.",
"wingsdk.cloud.Queue deleted.",
"wingsdk.cloud.Function deleted.",
Expand All @@ -878,6 +885,33 @@ exports[`queue with one subscriber, batch size of 5 1`] = `

exports[`queue with one subscriber, batch size of 5 2`] = `
{
".wing/function_c8ab799f.js": "exports.handler = async function(event) {
return await (new ((function(){
return class Handler {
constructor(clients) {
for (const [name, client] of Object.entries(clients)) {
this[name] = client;
}
}
async handle() {
await this.queue.push(\\"A\\");
await this.queue.push(\\"B\\");
await this.queue.push(\\"C\\");
await this.queue.push(\\"D\\");
await this.queue.push(\\"E\\");
await this.queue.push(\\"F\\");
}
};
})())({
queue: (function(env) {
let handle = process.env[env];
if (!handle) {
throw new Error(\\"Missing environment variable: \\" + env);
}
return $simulator.findInstance(handle);
})(\\"QUEUE_HANDLE_54fcf4cd\\")
})).handle(event);
};",
".wing/my_queue-setconsumer-e645076f_c8ddc1ce.js": "exports.handler = async function(event) {
return await (new (require(\\"[REDACTED]/wingsdk/src/target-sim/queue.setconsumer.inflight.js\\")).QueueSetConsumerHandlerClient({ handler: new ((function(){
return class Handler {
Expand Down Expand Up @@ -922,14 +956,6 @@ async handle(message) {
"attrs": {},
"path": "root/my_queue",
"props": {
"initialMessages": [
"A",
"B",
"C",
"D",
"E",
"F",
],
"retentionPeriod": 3600,
"timeout": 10,
},
Expand All @@ -947,6 +973,27 @@ async handle(message) {
},
"type": "wingsdk.sim.EventMapping",
},
{
"attrs": {},
"path": "root/my_queue_messages/Function",
"props": {
"environmentVariables": {
"QUEUE_HANDLE_54fcf4cd": "\${root/my_queue#attrs.handle}",
},
"sourceCodeFile": ".wing/function_c8ab799f.js",
"sourceCodeLanguage": "javascript",
"timeout": 60000,
},
"type": "wingsdk.cloud.Function",
},
{
"attrs": {},
"path": "root/my_queue_messages",
"props": {
"functionHandle": "\${root/my_queue_messages/Function#attrs.handle}",
},
"type": "wingsdk.cloud.OnDeploy",
},
],
"sdkVersion": "0.0.0",
},
Expand Down Expand Up @@ -982,6 +1029,22 @@ async handle(message) {
"id": "Handler",
"path": "root/Handler",
},
"OnDeployHandler": {
"attributes": {
"wing:resource:connections": [],
},
"constructInfo": {
"fqn": "constructs.Construct",
"version": "10.1.314",
},
"display": {
"description": "An inflight resource",
"hidden": true,
"title": "Inflight",
},
"id": "OnDeployHandler",
"path": "root/OnDeployHandler",
},
"cloud.TestRunner": {
"attributes": {
"wing:resource:connections": [],
Expand All @@ -1007,6 +1070,18 @@ async handle(message) {
"relationship": "setConsumer()",
"resource": "root/my_queue/my_queue-SetConsumer-e645076f",
},
{
"direction": "inbound",
"implicit": false,
"relationship": "push()",
"resource": "root/my_queue_messages/Function",
},
{
"direction": "inbound",
"implicit": false,
"relationship": "$inflight_init()",
"resource": "root/my_queue_messages/Function",
},
],
},
"children": {
Expand Down Expand Up @@ -1085,6 +1160,52 @@ async handle(message) {
"id": "my_queue",
"path": "root/my_queue",
},
"my_queue_messages": {
"attributes": {
"wing:resource:connections": [],
},
"children": {
"Function": {
"attributes": {
"wing:resource:connections": [
{
"direction": "outbound",
"implicit": false,
"relationship": "push()",
"resource": "root/my_queue",
},
{
"direction": "outbound",
"implicit": false,
"relationship": "$inflight_init()",
"resource": "root/my_queue",
},
],
},
"constructInfo": {
"fqn": "constructs.Construct",
"version": "10.1.314",
},
"display": {
"description": "A cloud function (FaaS)",
"sourceModule": "@winglang/sdk",
"title": "Function",
},
"id": "Function",
"path": "root/my_queue_messages/Function",
},
},
"constructInfo": {
"fqn": "constructs.Construct",
"version": "10.1.314",
},
"display": {
"description": "Run code during the app's deployment.",
"title": "OnDeploy",
},
"id": "my_queue_messages",
"path": "root/my_queue_messages",
},
},
"constructInfo": {
"fqn": "constructs.Construct",
Expand Down Expand Up @@ -1161,7 +1282,6 @@ async handle(message) {
"attrs": {},
"path": "root/my_queue",
"props": {
"initialMessages": [],
"retentionPeriod": 3600,
"timeout": 10,
},
Expand Down
Loading

0 comments on commit 9399564

Please sign in to comment.