Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does table replication with row_filter require more memory? #477

Open
kostyay opened this issue May 29, 2024 · 3 comments
Open

Does table replication with row_filter require more memory? #477

kostyay opened this issue May 29, 2024 · 3 comments

Comments

@kostyay
Copy link

kostyay commented May 29, 2024

Hi
I'm trying to replicate a table with a row_filter defined (start_time > current_date - interval '4 months') and it causes memory usage to spike to 100% which ultimately causes the replication to go down as the process crashed with OOM.
Its a table with about 5 mil rows on a 6 vcpu 16 gig Postgres 14 machine.
If I try to sync the entire table without row_filter it finishes successfully.
I tried creating an index on start_time but it didn't help.
Is this the expected behaviour? Is there anything I can do to resolve this (besides adding more memory)?
Thanks!

@danrough
Copy link

We also experience this issue, in a similar set of circumstances - large table, which we're trying to replicate using a row filter, on an r7g.4xl postgres RDS, and the process brings the whole instance down - the main process gets OOM killed. Without a row filter however, the replication is fine.

Did you find any ways to optimise this, @kostyay?

@kostyay
Copy link
Author

kostyay commented Sep 4, 2024

Unfortunately not

@danrough
Copy link

danrough commented Sep 5, 2024

We moved to using the native replication with a row filter, rather than using the pglogical module, and we have been able to progress with this. So we've now abandoned the use of the pglogical module for anything other than replicating sequences.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants