[release-1.10] eliminate short-circuiting in logic to add/remove finalizers

[release-1.10] eliminate short-circuiting in logic to add/remove finalizers #4151