-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pools occasionally are not deleted from database even though loadbalancer status is 'ACTIVE' #725
Comments
@jlongstaf: check this out! |
And here is a sever.log capture during which the test failed. Here's a breakdown of the messages and the interesting times where you can find them: 15:21:59.892 --> api call to delete the pool Note that if we retry after a sleep, these tests succeed.
|
OpenStack Release
mitaka
Bug Severity
For bugs enter the bug severity level. Do not set any labels.
Severity: <Fill in level: 1 through 5>
Severity level definitions:
Description
There are a set of tests that intermittently fail in nightly due to an assertion error. I would guess that the failure rate is roughly 10-20% of the time. The tests are in
api/test_pools.py
. Both occasionally fail. I've reproduced the error with some debug information that is shown below. We delete the shared pool and then expect the loadbalancer status to go 'ACTIVE' once the agent communicates to the driver that the operation is complete. The test checks for the ACTIVE loadbalancer, and then retrieves the service object from the driver directly. Sometimes that service object has the pool still in PENDING_DELETE state. This causes the test to fail. I've also traced the logs and the right things appear to be happening in the agent and neutron. Both of the operations of setting the lb status and deleting the pool are performed within a database transaction.Here's the debug service object output. The steps are delete the pool, check the lb status for up to 1 minute until it goes ACTIVE, then check the service object:
The text was updated successfully, but these errors were encountered: