Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pgmq_read_with_poll #64

Merged
merged 13 commits into from
Aug 18, 2023
Merged

Conversation

v0idpwn
Copy link
Collaborator

@v0idpwn v0idpwn commented Aug 17, 2023

For feature parity with SQS. The implementation is a bit naive: it simply will retry querying for messages on a small interval until the timeout is reached.

To be discussed:

  • should we make the interval configurable? maybe it should be a parameter to the function?
  • should we receive the timeout in seconds instead of ms, for consistency, considering vt is defined in seconds?

To be done:

  • add tests, i only tested manually
  • add in rust client without the extension (there we could use async instead of thread::sleep)
  • document

@ChuckHend
Copy link
Member

  • should we make the interval configurable? maybe it should be a parameter to the function?

I think we should make it configurable with a default. Postgres does let us have optional parameters in functions:

partitioned: default!(bool, false),

  • should we receive the timeout in seconds instead of ms, for consistency, considering vt is defined in seconds?

Seconds seems good to me, to be consistent with rest of pgmq. SQS is seconds too, right?

@@ -234,14 +235,14 @@ pub fn read(name: &str, vt: i32, limit: i32) -> Result<String, PgmqError> {
(
SELECT msg_id
FROM {PGMQ_SCHEMA}.{TABLE_PREFIX}_{name}
WHERE vt <= now()
WHERE vt <= clock_timestamp()
Copy link
Collaborator Author

@v0idpwn v0idpwn Aug 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now() uses transaction start time. This can be pretty useful, but in this case, if we repeatedly query it will always use the same timestamp. I also tried statement_timestamp(), but for some reason while using the SPI I got the same timestamp for different runs 🤔, so I resorted to clock_timestamp()

For feature parity with SQS. The implementation is a bit naive: it
simply will retry querying for messages on a small interval until
the timeout is reached.
@v0idpwn v0idpwn force-pushed the feat/read-with-poll branch 2 times, most recently from ca79a3e to 7a8f246 Compare August 17, 2023 23:25
@v0idpwn v0idpwn force-pushed the feat/read-with-poll branch 2 times, most recently from fd24f80 to 3738c57 Compare August 17, 2023 23:36
core/src/lib.rs Outdated Show resolved Hide resolved
@v0idpwn v0idpwn force-pushed the feat/read-with-poll branch 2 times, most recently from 730d554 to 591907c Compare August 18, 2023 11:34
core/src/lib.rs Outdated Show resolved Hide resolved
@ChuckHend
Copy link
Member

@v0idpwn , the Rust client library tests are run against the container image with pgmq installed in it - and the image is currently only built when the extension is released. So the Rust client library tests will fail since they are looking for the new long poll function, which wont be released yet.

@ChuckHend ChuckHend merged commit 4ef04f2 into tembo-io:main Aug 18, 2023
8 of 10 checks passed
@v0idpwn v0idpwn deleted the feat/read-with-poll branch August 18, 2023 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants