-
-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Data lost when exceed batch size. #111
Comments
I wasn't able to reproduce the bug as the rows you mentioned seems already ok. |
In current version (tag v2.0.14) |
I'm trying to avoid saying "me too", because I'm struggling to understand the scope of the problem we're witnessing. It appears to be the case that "some" rows in one table "sometimes" aren't updated when the source rows are. The batch sizes in question are far far lower then our max batch size (10000 in config, actual tends to be less than 200) We haven't been able identify the criteria for failure as yet, and are not entirely sure it's limited to one table (that wouldn't make sense, but it does have a large amount of text in it, so that might issue some how). I am actively looking into the problem now we speak and hope to have more concrete information later today. |
OK... I think I might have found the problem, and agree with @wycwyx about the fix, but not the cause.
|
I've managed to get a copy of the binary log for an example of this occurring in our production environment and whilst I think my previous point holds true, the log doesn't support it being the cause. I'd like to share the log, but it's several hundred megabytes and contains personal information which I cannot disclose. What I see is very strange though: What I don't see in the log is anything in between the successful ones and failing ones that would be classed as Is it possible that I dunno... I'm really confused by this, so apologies for just brain dumping on here, just sharing what I'm finding. We have a steady trickle of the problem in our production environment, apparently only to one table and I can't for the life of me recreate it outside of that.. and all the tools, logs, statuses etc suggest it's not happening. |
We've now witnessed evidence of records missing (so an insert being dropped) from other tables, which suggests this is isn't limited scope problem. That makes sense as we could never explain why it would be. Unfortunately, we're also no closer to explaining why this is happening at all. |
@rascalDan did you end up finding a solution? e.g. adding some configs such as |
@FreCap I'm afraid not... we've never bottomed the problem and currently have a second process comparing/repairing data that's been missed... which I'd love to get rid of! (but are currently extending to cover other tables) |
Could you please share a fully runnable SQL and pg_chameleon example to reproduce so we can check on my side as well? I saw your previous description, but to get everybody on the same page and allow reproducing, I think it would be instrumental in finding a way to remove the compare/repair feature (which scares me out quite a bit). |
Honestly, not really. Most of our code is just |
…replica_size is reached rel-issue:the4thdoctor#111
Describe the bug
Data lost when the number of rows is exceed batch size.
To Reproduce
After debuged the code(v2.0.12 or master), I found the code snippet in mysql_lib. py.
code :
`
`
According the code, the rows in group_insert will not to be stored.
I changed them to
`
`
Then result was right.
The text was updated successfully, but these errors were encountered: