Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: CLIENT: Sent message larger than max (64975458 vs. 64000000) with bulk upsert #460

Open
maximyurchuk opened this issue Jul 29, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@maximyurchuk
Copy link

Bug Report

YDB Python SDK version:

maxim-yurchuk@yurchuk-github:~/ydb/ydb$ pip show ydb
Name: ydb
Version: 3.11.3

Environment

maxim-yurchuk@yurchuk-github:~/ydb/ydb$ uname -a
Linux yurchuk-github 5.15.0-107-generic https://github.com/ydb-platform/ydb-python-sdk/issues/117-Ubuntu SMP Fri Apr 26 12:26:49 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Current behavior:

Huge bulk upsert (driver.table_client.bulk_upsert) gives the following error

2024-07-12T08:09:49.8675916Z Traceback (most recent call last):
2024-07-12T08:09:49.8677299Z   File "/home/runner/.local/lib/python3.10/site-packages/ydb/connection.py", line 458, in __call__
2024-07-12T08:09:49.8678065Z     response = rpc_state(
2024-07-12T08:09:49.8678879Z   File "/home/runner/.local/lib/python3.10/site-packages/ydb/connection.py", line 242, in __call__
2024-07-12T08:09:49.8679773Z     response, rendezvous = self.rpc.with_call(*args, **kwargs)
2024-07-12T08:09:49.8680722Z   File "/home/runner/.local/lib/python3.10/site-packages/grpc/_channel.py", line 1198, in with_call
2024-07-12T08:09:49.8681604Z     return _end_unary_response_blocking(state, call, True, None)
2024-07-12T08:09:49.8682681Z   File "/home/runner/.local/lib/python3.10/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
2024-07-12T08:09:49.8683773Z     raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
2024-07-12T08:09:49.8684656Z grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
2024-07-12T08:09:49.8685362Z 	status = StatusCode.RESOURCE_EXHAUSTED
2024-07-12T08:09:49.8686015Z 	details = "CLIENT: Sent message larger than max (64975458 vs. 64000000)"
2024-07-12T08:09:49.8687739Z 	debug_error_string = "UNKNOWN:Error received from peer  ***grpc_message:"CLIENT: Sent message larger than max (64975458 vs. 64000000)", grpc_status:8, created_time:"2024-07-12T08:09:49.841712345+00:00"***"
2024-07-12T08:09:49.8688817Z >
        """

Expected behavior:

No any error

Steps to reproduce:
Do huge bulk upsert

@vgvoleg
Copy link
Collaborator

vgvoleg commented Jul 30, 2024

64*10^6 is the default value for grpc channel params:
grpc.max_send_message_length
grpc.max_receive_message_length

It is possible to override them: DriverConfig has a special field channel_options. Place a dict with these keys and values you need (-1 could be used for no limits).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants