Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

copy.deepcopy() doesn't work anymore #210

Open
moyogo opened this issue Nov 2, 2018 · 7 comments
Open

copy.deepcopy() doesn't work anymore #210

moyogo opened this issue Nov 2, 2018 · 7 comments

Comments

@moyogo
Copy link
Collaborator

moyogo commented Nov 2, 2018 •
edited
Loading

This now happens with 0.6.0, but was working before:

>>> import defcon                                                                                                                                                                            [42/78]
>>> font = defcon.Font()
>>> import copy
>>> copy.deepcopy(font)
<defcon.objects.font.Font object at 0x106be5a58>
>>> font.save('font060.ufo')
>>> copy.deepcopy(font)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle _thread.RLock objects
@anthrotype
Copy link
Member

that's because the Layer.save method stores a reference to the GlyphSet object that was used for writing, and the GlyphSet object in turns holds a filesystem object that uses some threading locks, which aren't pickleable, hence cannot be deepcopied.

https://github.com/typesupply/defcon/blob/5391d57703661ba29f607f6deacff02051cf5056/Lib/defcon/objects/layer.py#L568

However, when the output format is UFOZ, since the Font.save method needs to close the writer (to be able to actually compress the zip file), we cannot hold that reference in Layer._glyphSet attribute, so we have to set it to None

https://github.com/typesupply/defcon/blob/5391d57703661ba29f607f6deacff02051cf5056/Lib/defcon/objects/layerSet.py#L305-L315

I think the reason why Layer holds a ref to the GlyphSet after saving is to support the testForExternalChanges feature. But that probably needs to be rethought (see #209).

@anthrotype
Copy link
Member

maybe for the deepcopy, you could try to use the getDataForSerialization and setDataFromSerialization methods

@moyogo
Copy link
Collaborator Author

moyogo commented Nov 2, 2018

OK. Thanks for looking into it. We'll use getDataForSerialization() and setDataFromSerialization().

@typesupply
Copy link
Member

The _glyphSet objects need to be retained in the layers to facilitate lazy loading. I'm considering some new changes to speed up UFOZ read/write that would retain even more fs objects. So, we should solve this. Is there a way to exclude an attribute from being copied by deepcopy? If so, we could use that to prevent the attempt to copy the particular attributes. The retained fs objects won't be relevant to the copy anyway.

@anthrotype
Copy link
Member

we'd have to redefine __copy__ and __deepcopy__ methods I guess
https://docs.python.org/3/library/copy.html

@typesupply
Copy link
Member

Yeah. I looked at that. I need to experiment with it to see if there is some way to exclude a few things instead of listing a million things to include.

@anthrotype anthrotype mentioned this issue Jan 3, 2019
Merged
@anthrotype
Copy link
Member

how about we simply define __deepcopy__ on Font, or even on the base object itself, which reuses the existing getDataForSerialization and setDataFromSerialization methods? E.g.

def __deepcopy__(self, memo):
    del memo  # unused
    data = self.getDataForSerialization()
    copied = self.__class__()
    copied.setDataFromSerialization(data)
    return copied

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@moyogo @typesupply @anthrotype