Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix db.Setting.set_value (pin sqlalchemy>=1.3.12) #19

Closed
chrisjsewell opened this issue Feb 29, 2020 · 6 comments · Fixed by #21
Closed

Fix db.Setting.set_value (pin sqlalchemy>=1.3.12) #19

chrisjsewell opened this issue Feb 29, 2020 · 6 comments · Fixed by #21
Labels
bug Something isn't working

Comments

@chrisjsewell
Copy link
Member

First noted in #16. The value column in db.Setting is set as a value = Column(JSON()). However, this fails on directly storing integer/float values (actually its seems to serialize them correctly, but then fail on deserialize). Maybe store as {"value": value}.

@chrisjsewell chrisjsewell added the bug Something isn't working label Feb 29, 2020
@chrisjsewell
Copy link
Member Author

This is the error that @choldgraf got:

TypeError: the JSON object must be str, bytes or bytearray, not int
12:01 $ jcache execute
Executing: /c/Users/chold/Dropbox/github/forks/python/ebp/jupyter-cache/docs/viz.ipynb
Failed Commit: /c/Users/chold/Dropbox/github/forks/python/ebp/jupyter-cache/docs/viz.ipynb
Traceback (most recent call last):
  File "/c/Users/chold/Dropbox/github/forks/python/ebp/jupyter-cache/jupyter_cache/executors/basic.py", line 65, in run
    self.cache.commit_notebook_bundle(final_bundle, overwrite=True)
  File "/c/Users/chold/Dropbox/github/forks/python/ebp/jupyter-cache/jupyter_cache/cache/main.py", line 235, in commit_notebook_bundle
    self.truncate_commits()
  File "/c/Users/chold/Dropbox/github/forks/python/ebp/jupyter-cache/jupyter_cache/cache/main.py", line 101, in truncate_commits
    COMMIT_LIMIT_KEY, self.db, DEFAULT_COMMIT_LIMIT
  File "/c/Users/chold/Dropbox/github/forks/python/ebp/jupyter-cache/jupyter_cache/cache/db.py", line 63, in get_value
    result = session.query(Setting.value).filter_by(key=key).one_or_none()
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3257, in one_or_none
    ret = list(self)
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 105, in instances
    util.raise_from_cause(err)
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 89, in instances
    for row in fetch
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 89, in <listcomp>
    for row in fetch
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 88, in <listcomp>
    keyed_tuple([proc(row) for proc in process])
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/engine/result.py", line 107, in __getitem__
    return processor(self._row[index])
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/site-packages/sqlalchemy/sql/sqltypes.py", line 2255, in process
    return json_deserializer(value)
  File "/home/choldgraf/anaconda/envs/dev/lib/python3.7/json/__init__.py", line 341, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not int

Which seems to imply that the value has been converted from a string (to int), before
being parsed to json.loads.
Strangely I don't see this in testing:

def test_setting(tmp_path):
    db = create_db(tmp_path)
    Setting.set_value("a", 1, db)
    assert Setting.get_value("a", db) == 1
    assert Setting.get_dict(db) == {"a": 1}

@choldgraf
Copy link
Member

FWIW, this is being run in Windows Subsystem for Linux, so Ubuntu under the hood

@chrisjsewell
Copy link
Member Author

FWIW, this is being run in Windows Subsystem for Linux, so Ubuntu under the hood

Could you try running the unit tests on your system, specifically the one above: pytest -k test_setting, and let me know if this is failing or not?

@choldgraf
Copy link
Member

choldgraf commented Feb 29, 2020

ah - it fails with the same error I got in demo-ing it:

$  pytest -k test_setting

...

TypeError: the JSON object must be str, bytes or bytearray, not int

@chrisjsewell
Copy link
Member Author

chrisjsewell commented Feb 29, 2020

Ah looks like this has been fixed as of sqlalchemy>=1.3.12: sqlalchemy/sqlalchemy#5014.
Try upgrading and re-running. Guess I'll add this version restriction to install_requires

@choldgraf
Copy link
Member

test passes now!

@chrisjsewell chrisjsewell changed the title Fix db.Setting.set_value Fix db.Setting.set_value (pin sqlalchemy>=1.3.12) Feb 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants