Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MySQL -> Snowflake possibly wrong mapping for tinyint(1) #925

Open
SlavaLenskyy opened this issue Mar 25, 2022 · 2 comments
Open

MySQL -> Snowflake possibly wrong mapping for tinyint(1) #925

SlavaLenskyy opened this issue Mar 25, 2022 · 2 comments
Labels
invalid This doesn't seem right

Comments

@SlavaLenskyy
Copy link

Describe the bug
https://github.com/transferwise/pipelinewise/blob/master/pipelinewise/fastsync/mysql_to_snowflake.py
line: 61-63

'tinyint': 'BOOLEAN' if mysql_column_type and mysql_column_type.startswith('tinyint(1)') else 'NUMBER',

The problem we face is that according to mysql tinyint(1) does not necessarily mean boolean.
Please take a look at the example below for msql 5.7 on the screenshot

Expected behavior
tinyint(1) should convert to int, not boolean.

Screenshots
image

Your environment

  • Version of PipelineWise, e.g branch/commit #/release/tag
    from docker inspect:
    "Created": "2021-05-24T19:11:27.959157156Z" "Parent": "sha256:67eea1ebd28037d44a21ee755fab5afbb75d1646de45b63536a283954ecbc1b9" "Image": "sha256:67eea1ebd28037d44a21ee755fab5afbb75d1646de45b63536a283954ecbc1b9" "Layers": [ "sha256:02c055ef67f5904019f43a41ea5f099996d8e7633749b6e606c400526b2c4b33", "sha256:f9ef7f1bcb199f8d36e3d939a8b83c53aa79f7caa71d16f92e918a24c6fa45d6", "sha256:4a36e2652108deffd504c214cf252bed4f3469a56edd7a3735e0fcf6acc504c6", "sha256:8506f6dcbf82345293a52580ec2038279dd832c1b62a1097ad9052383aafa55f", "sha256:772762ad19a526c9ef62709e713734037c4fc044148915dcdb4286f639f06afd", "sha256:a2398b11591d9326273d5bc031e577efc6df214cbcfa8f028716e4b4cce19e00", "sha256:b1f662605817dc7c1a04bb22bb208ccf54ef7c2dd154083e22225f44c76e64ea", "sha256:a2ea233c7f7166023063741819753df5de138d9b37b49413f6bc5d4ff1e28c5b", "sha256:007be55eada46db6465487cd0c15ba266e6c554f8f0d7b63f6343e74e493247a" ]

  • Source type and setup
    Mysql
    mysql> select version();
    +------------+
    | version() |
    +------------+
    | 5.7.33-log |
    +------------+
    1 row in set (0.01 sec)

  • Target type and setup
    Snowflake

@SlavaLenskyy SlavaLenskyy added the bug Something isn't working label Mar 25, 2022
@SlavaLenskyy SlavaLenskyy changed the title MySQL -> Snowflake possibly wrong mapping MySQL -> Snowflake possibly wrong mapping for tinyint(1) Mar 25, 2022
@Samira-El
Copy link
Contributor

Samira-El commented Apr 4, 2022

Hey, Boolean and tinyint(1) are synonym in mysql, see doc
image

We cannot cater to your specific case, if you wish to store numbers then you'll have to change the type of the column or make the length > 1.

@Samira-El Samira-El added invalid This doesn't seem right and removed bug Something isn't working labels Apr 4, 2022
@koszti
Copy link
Contributor

koszti commented Apr 4, 2022

@SlavaLenskyy this is an expected behaviour. You can read more about the decision at singer-io/tap-mysql#82

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid This doesn't seem right
Projects
None yet
Development

No branches or pull requests

3 participants