Should Instrument.__del__() and close() not exist?

[donmr]

I have been getting somewhat random python errors due to dereferencing deleted objects. The answer seems to be not to have either __del__() or close() functions in the Instrument class (and thus not call usb.util.dispose_resources()) because Python deletes things in unpredictable orders. Just calling close() causes the same problems even with __del__() removed.

See: https://stackoverflow.com/questions/50574201/memory-corruption-on-exit-or-stupid-user-error

[alexforencich]

Is that actually the correct solution? Just don't do any clean up at all and hope things will just work out? Seems like the linked question says to use __enter__ and __exit__, but this then makes it extremely cumbersome to do much of anything as then the whole script that uses usbtmc has to be wrapped in a bunch of nested with statements and I believe this precludes any sort of GUI style operation where the connection is maintained in the background.

[alexforencich]

So, it looks like contextlib.ExitStack can alleviate some of the issues with opening multiple connections, but this still requires operating inside a with statement or explicit manual clean-up.

I also found https://stackoverflow.com/questions/36981177/python-with-statement-in-gui-pyqt-program , which seems to indicate that it may not be possible for python to do this sort of cleanup properly.

[alexforencich]

Hmm, I also found atexit, maybe that's a way to get some of this manual cleanup sorted out properly.

[alexforencich]

Looks like atexit might be ideal, with the one unfortunate problem being atexit does not support unregister in python 2, so some sort of callable weak reference is required for python 2 compatibility. This could solve a number of issues, though, for more than just python-usbtmc.

[donmr]

The place where I had problems was when close called usb.util.dispose_resources.
I tried removing the __del__ function and calling close from main. That produced similar errors when exit ran, it complained of trying to access things that had already been deleted.

It looks like pyusb goes to a lot of effort to support explicit freeing of resources, but as it notes, Python does not really provide for this.

I think one of the limitations of using a language that hides all of its memory management is that you have to trust it to handle things for you. Instead of freeing an object you have to clear anything that refers to it and the runtime should then free it.

[donmr]

I also wonder why I seem to be the only one seeing this. Am I doing something that different???

[alexforencich]

I have run in to some weirdness with python-usbtmc, but I always chalked it up to instruments not implementing the protocol correctly. I think you're not the only one who has seen this issue, but you're the first so far to figure out what's been going on.

I have had issues with python-vxi11 not properly closing links leading to instruments (namely certain Tektronix oscilloscopes) that don't free links when the TCP connection is closed to run out of resources and require a reboot and added the __del__ methods in an attempt to correct that, but apparently that isn't sufficient due to Python's somewhat crappy memory management.

[alexforencich]

Also, would you mind trying something?

Add

import atexit

at the top, and

atexit.register(self.close)

at the bottom of open. Let me know if you're still seeing these cleanup issues with that.

[donmr]

Using atexit prevents errors in a simple test where I just open and then call exit. If I access the device to read data then I still get warnings like this from down in pyUSB.

Exception ignored in: <bound method Device.del of <DEVICE ID 1ab1:04ce on Bus 003 Address 042>>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/usb/core.py", line 1029, in del
File "/usr/lib/python3/dist-packages/usb/core.py", line 225, in dispose
File "/usr/lib/python3/dist-packages/usb/core.py", line 220, in release_all_interfaces
TypeError: 'NoneType' object is not callable

Or like this:

Exception ignored in: <bound method _Device.del of <usb.backend.libusb1._Device object at 0x7fad44d4b390>>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/usb/backend/libusb1.py", line 561, in del
AttributeError: 'NoneType' object has no attribute 'libusb_unref_device'