Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run setup when creating extension #38

Closed

Conversation

v0idpwn
Copy link
Collaborator

@v0idpwn v0idpwn commented Aug 10, 2023

Closes #32.

@v0idpwn v0idpwn changed the title Fix/run setup when creating extension Run setup when creating extension Aug 10, 2023
@ChuckHend
Copy link
Member

This looks good. I'll take a look at the tests. They have had issues ever since we made pg_partman an optional dependency.

@ChuckHend
Copy link
Member

Looks like CI is installing pg_partman into the pgrx managed postgres 15.3, but tests are now running on 15.4, which looks like was released today.

@ChuckHend
Copy link
Member

Looks like CI is installing pg_partman into the pgrx managed postgres 15.3, but tests are now running on 15.4, which looks like was released today.

hopefully fix that problem over in #39

@v0idpwn v0idpwn force-pushed the fix/run-setup-when-creating-extension branch from ec38010 to ad5ab37 Compare August 10, 2023 15:28
@v0idpwn v0idpwn force-pushed the fix/run-setup-when-creating-extension branch from ad5ab37 to 0348250 Compare August 10, 2023 15:30
@v0idpwn
Copy link
Collaborator Author

v0idpwn commented Aug 10, 2023

Rebased into main and tried to add a test. The toolchain is a bit new for me, so maybe I messed up :)

Cargo.toml Show resolved Hide resolved
@ChuckHend
Copy link
Member

This is going to be trickier than I expected :(

  • pgmq_meta its not currently owned by pgmq, but it should be. I'll have to think how we can handle that, it might be combination of a migration and the init?
  • there's two Rust client API, pgmq::PGMQueue and PGMQueueExt. PGMQueueExt is has a subset of features of the other, but can be run without the extension. PGMQueue still needs to create pgmq_meta, so some of the tests are probably failing for that reason. That part would go somewhere near here:
    let setup = query::init_queue(queue_name)?;
  • looks like one test is failing due to pg_partman extension not being created, thats a bug not related to this PR that slipped through. CREATE EXTENSION pg_partman just needs to happen somewhere before here
    let test_duration_queue = format!("test_duration_{test_num}");
    I can patch that

Comment on lines +20 to +23
CREATE TABLE IF NOT EXISTS public.pgmq_meta (
queue_name VARCHAR UNIQUE NOT NULL,
created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() at time zone 'utc') NOT NULL
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we won't be able to use the IF NOT EXISTS part since pgmq does not own the existing tables. Might be tricky, I'll have to think how we could handle that.

@v0idpwn
Copy link
Collaborator Author

v0idpwn commented Aug 10, 2023

pgmq_meta its not currently owned by pgmq, but it should be. I'll have to think how we can handle that, it might be combination of a migration and the init?

I don't really follow this. In my manual tests, it seemed to be owned by the extension (the queue tables, though, weren't). The only case I see where this would happen is if the pgmq_meta table was created manually (for example, by a previous version of the extension) and the extension was installed again. I can try to investigate further, though.

@ChuckHend
Copy link
Member

I don't really follow this. In my manual tests, it seemed to be owned by the extension (the queue tables, though, weren't). The only case I see where this would happen is if the pgmq_meta table was created manually (for example, by a previous version of the extension) and the extension was installed again. I can try to investigate further, though.

An extension may only use CREATE ... IF NOT EXISTS to skip object creation if the conflicting object is one that it already owns.

I think I ran into this once before and I did not had much luck using IF NOT EXISTS in the extension SQL if its in the schema that is owned by the extension. The desired behavior we want is for folks running create extension pgmq for the first time, pgmq_meta is created automatically. Users who already have pgmq installed, they should run ALTER EXTENSION pgmq UPDATE TO '<new-version>', and that would not break anything for them. So typing this out, maybe this just means the extension_sql! should just remove the if not exists part?

@v0idpwn
Copy link
Collaborator Author

v0idpwn commented Aug 11, 2023

Additionally, I think you probably want to drop all tables related to the extension if the extension is dropped. We'd need all the tables to belong to the extension so this is done automatically. Unless there's other way.

@ChuckHend
Copy link
Member

Additionally, I think you probably want to drop all tables related to the extension if the extension is dropped. We'd need all the tables to belong to the extension so this is done automatically. Unless there's other way.

Agreed. It'll be quite a bit more scope if we want to handle all of that right now. Alternatively, we could put a fix into list_queues() directly so it doesn't crash, and handle all of the object ownership in another body of work.

@v0idpwn
Copy link
Collaborator Author

v0idpwn commented Aug 11, 2023

I will give a stab to the "complete solution" in the weekend, maybe I can make it work. If not, I agree on reducing scope.

@ChuckHend
Copy link
Member

I will give a stab to the "complete solution" in the weekend, maybe I can make it work. If not, I agree on reducing scope.

Sounds good, and I appreciate the help! You can DM me in Slack if you want, https://join.slack.com/t/tembocommunity/shared_invite/zt-20dtnhcmo-pLNV7_Aobi50TdTLpfQ~EQ

@v0idpwn
Copy link
Collaborator Author

v0idpwn commented Aug 16, 2023

Closing this in favor of other changes by Adam.

@v0idpwn v0idpwn closed this Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pgmq_list_queues errors if no queues were created
2 participants