Client fault tolerant connection #12

DeKinci · 2021-12-13T14:44:12Z

Changelog category (leave one):

New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
...
Parameter host can accept multiple hosts. In case of unavailability of one of them, the client will try to connect to the next one.

Detailed description / Documentation draft:
...
When multiple addresses are passed to the host argument and the first one is unavailable, the console client will try to connect to the next one.

By adding documentation, you'll allow users to try your new feature immediately, not when someone else will have time to document it later. Documentation is necessary for all features that affect user experience in any way. You can add brief documentation draft above, or add documentation right into your patch as Markdown files in docs folder.

If you are doing this for the first time, it's recommended to read the lightweight Contributing to ClickHouse Documentation guide first.

Information about CI checks: https://clickhouse.tech/docs/en/development/continuous-integration/

PolyProgrammist · 2021-12-20T19:33:14Z

src/Client/ClientBase.h

+        String host;
+        std::optional<int> port{};
+        friend std::istream & operator>>(std::istream & in, HostPort & hostPort) {
+            String


Please use more common way:

String host_with_port; String delimiter = ":";

PolyProgrammist · 2021-12-20T20:00:42Z

src/Client/ClientBase.h

+            in >> host_with_port;
+            size_t delimiter_pos = host_with_port.find(delimiter);
+            hostPort.host = host_with_port.substr(0, delimiter_pos);
+            if (delimiter_pos < host_with_port.length())


delimiter_pos != std::string::npos

PolyProgrammist · 2021-12-20T20:03:13Z

src/Client/ClientBase.h

+                delimiter = ":";
+            in >> host_with_port;
+            size_t delimiter_pos = host_with_port.find(delimiter);
+            hostPort.host = host_with_port.substr(0, delimiter_pos);


You can move it also inside if and separately assign hostPort.host to host_with_port in else branch so that not to mess with host_with_port.substr(0, std::string::npos)

PolyProgrammist · 2021-12-20T20:19:29Z

src/Client/ClientBase.h

+            return in;
+        }
+    };
+    std::vector<HostPort> hosts_ports{};


I think it's better to delete the old host field

There is no host field in ClientBase. The reason why I add this field in ClientBase, you can read our discussion in old PR: #4 (comment)

PolyProgrammist · 2021-12-20T20:24:09Z

programs/client/Client.cpp

-        config().setString("host", options["host"].as<std::string>());
+    {
+        hosts_ports = options["host"].as<std::vector<HostPort>>();
+        config().setString("host", hosts_ports[0].host);


Is it better to set all the hosts and ports instead of just one?

config can't store std::vector , only String, Int and other simple types. I wrote about this in this PR: #4 (comment)

PolyProgrammist · 2021-12-20T20:26:13Z

programs/client/Client.cpp

    String server_name;
    UInt64 server_version_major = 0;
    UInt64 server_version_minor = 0;
    UInt64 server_version_patch = 0;

-    try
+    String tcp_port_config_key = is_secure ? "tcp_port_secure" : "tcp_port";


It may be easier to read with if instead of ternary operator

PolyProgrammist · 2021-12-20T20:31:22Z

programs/client/Client.cpp

@@ -512,48 +512,73 @@ catch (...)

 void Client::connect()


Also need rebase on new master and tests

fix the run command and add example

Before this patch current_user/current_address will be preserved from the previous query. Signed-off-by: Azat Khuzhin <[email protected]>

Co-authored-by: Azat Khuzhin <[email protected]>

…rite-buffer

Update list-versions.sh, update version_date.tsv

…ck Start Guide https://clickhouse.com/#quick-start

…-for-show-grants Fix checking grants for SHOW GRANTS

Cmake leftovers cleanup

…ssword Add 'clickhouse-client --password' comment to the scripts used in Qui…

Disable data skipping indexes by default for queries with FINAL

…put stream

…rays Fix consecutive backward seeks in seekable read buffers

…database-memory Fix wrong engine in SHOW CREATE DATABASE with engine Memory ClickHouse#34225

Add table function format(format_name, data)

Add options for clickhouse-format.

Add composability to casting and index operators

Fix clang tidy

…-fault-tolerant-connection

Support UUID in MsgPack format

…temp-tables-via-grpc Fix inserting to temporary tables via gRPC.

Update clickhouse-keeper.md

…metadata Better local metadata comparison with ZooKeeper metadata

Fix segfault in schema inference from url

add function addressToLineWithInlines

Fix various issues when projection is enabled by default

Method called on already moved

Use UTC in docker images

This reverts commit efd8044, reversing changes made to 4bb69bc.

Revert "Merge pull request ClickHouse#34373 from ClickHouse/docker-tz"

Add submodule minizip

…-fault-tolerant-connection

DeKinci requested a review from PolyProgrammist December 13, 2021 14:44

PolyProgrammist reviewed Dec 20, 2021

View reviewed changes

PolyProgrammist mentioned this pull request Dec 20, 2021

Add getting list of hosts with ports from --host param #10

Closed

PolyProgrammist approved these changes Dec 20, 2021

View reviewed changes

df530 force-pushed the client-fault-tolerant-connection branch 9 times, most recently from 94e48b0 to 962d851 Compare January 20, 2022 09:54

andycol and others added 17 commits February 2, 2022 14:32

Update clickhouse-keeper.md

93f9a9e

fix the run command and add example

Fix current_user/current_address for interserver mode

5472aef

Before this patch current_user/current_address will be preserved from the previous query. Signed-off-by: Azat Khuzhin <[email protected]>

Replace custom formats by short alias

744cb06

add query id for delayed interactive

eae0bc7

Add helper class to cache the result of checking access.

30557ae

Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp

1117e60

Co-authored-by: Azat Khuzhin <[email protected]>

Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-w…

ea044fc

…rite-buffer

Probably fix something

2003a96

support async inserts in clickhouse-client for queries with inlined data

5bb1b3c

correct config

f327e67

Update and add new team members (ClickHouse#34268)

9b0a8d5

Merge pull request ClickHouse#34240 from ClickHouse/list-versions

e49ff10

Update list-versions.sh, update version_date.tsv

Add 'clickhouse-client --password' comment to the scripts used in Qui…

a23edb4

…ck Start Guide https://clickhouse.com/#quick-start

Merge pull request ClickHouse#34225 from vitlibar/fix-checking-grants…

02c6cfb

…-for-show-grants Fix checking grants for SHOW GRANTS

Merge pull request ClickHouse#34167 from azat/cmake-cleanup

bbf2174

Cmake leftovers cleanup

Merge pull request ClickHouse#34270 from mfursov/clickhouse-client-pa…

75e4a47

…ssword Add 'clickhouse-client --password' comment to the scripts used in Qui…

Merge pull request ClickHouse#34243 from azat/use_skip_indexes_if_final

57d16ba

Disable data skipping indexes by default for queries with FINAL

df530 added 3 commits February 8, 2022 02:03

Fix error with IPv6 in host param and impove HostPort reading from in…

b035ad7

…put stream

Fix codestyle

87933e4

Fix error with IPv6 in host param

c1df291

df530 force-pushed the client-fault-tolerant-connection branch from 3253ae1 to c1df291 Compare February 7, 2022 23:04

alexey-milovidov and others added 26 commits February 8, 2022 02:27

Merge pull request ClickHouse#34376 from CurtizJ/fix-reading-empty-ar…

43ee8dd

…rays Fix consecutive backward seeks in seekable read buffers

Merge pull request ClickHouse#34345 from ClickHouse/fix-wrong-engine-…

51c767d

…database-memory Fix wrong engine in SHOW CREATE DATABASE with engine Memory ClickHouse#34225

Merge pull request ClickHouse#34125 from Avogar/table-from-literal

755c808

Add table function format(format_name, data)

Merge pull request ClickHouse#34349 from bigo-sg/format_parse_options

1c461e9

Add options for clickhouse-format.

Merge pull request ClickHouse#34229 from evillique/parser

b21adb8

Add composability to casting and index operators

Merge branch 'master' into addressToLineWithInlines

a33bd4c

use max instead of not any to check all rows

1d13c68

Method called on already moved

6b3adbb

Merge pull request ClickHouse#34388 from DevTeamBK/Fix-clang-tidy

3c0572b

Fix clang tidy

Merge branch 'master' of github.com:ClickHouse/ClickHouse into client…

c08bc7a

…-fault-tolerant-connection

Merge pull request ClickHouse#34065 from Avogar/msgpack

b4fec2a

Support UUID in MsgPack format

Delete not used include

e657219

Fix segfault in schema inference from url

e118c89

Merge pull request ClickHouse#34364 from vitlibar/fix-inserting-into-…

31fdd5c

…temp-tables-via-grpc Fix inserting to temporary tables via gRPC.

Merge pull request ClickHouse#34264 from andycol/patch-1

bb0cadc

Update clickhouse-keeper.md

Merge pull request ClickHouse#33933 from sunny19930321/fix-substr-zk-…

8e9ccbd

…metadata Better local metadata comparison with ZooKeeper metadata

Merge pull request ClickHouse#34405 from Avogar/fix-segfault-in-url

3e216ad

Fix segfault in schema inference from url

Merge pull request ClickHouse#33467 from cmsxbc/addressToLineWithInlines

496d77c

add function addressToLineWithInlines

Update Settings.h

428d6f1

Merge pull request ClickHouse#34305 from amosbird/projection-fix27

0f7c0c7

Fix various issues when projection is enabled by default

Merge pull request ClickHouse#34398 from DevTeamBK/input_format

4bb69bc

Method called on already moved

Merge pull request ClickHouse#34373 from ClickHouse/docker-tz

efd8044

Use UTC in docker images

Revert "Merge pull request ClickHouse#34373 from ClickHouse/docker-tz"

2d02eab

This reverts commit efd8044, reversing changes made to 4bb69bc.

Merge pull request ClickHouse#34413 from ClickHouse/docker-tz

69fa79a

Revert "Merge pull request ClickHouse#34373 from ClickHouse/docker-tz"

Merge pull request ClickHouse#34226 from vitlibar/add-submodule-minizip

9a8e458

Add submodule minizip

Merge branch 'master' of github.com:ClickHouse/ClickHouse into client…

cfefa57

…-fault-tolerant-connection

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Client fault tolerant connection #12

Client fault tolerant connection #12

DeKinci commented Dec 13, 2021

PolyProgrammist Dec 20, 2021

df530 Dec 31, 2021

PolyProgrammist Dec 20, 2021

df530 Dec 31, 2021

PolyProgrammist Dec 20, 2021

df530 Dec 31, 2021

PolyProgrammist Dec 20, 2021

df530 Dec 31, 2021

PolyProgrammist Dec 20, 2021

df530 Dec 31, 2021

PolyProgrammist Jan 10, 2022

PolyProgrammist Dec 20, 2021

PolyProgrammist Dec 20, 2021

Client fault tolerant connection #12

Are you sure you want to change the base?

Client fault tolerant connection #12

Conversation

DeKinci commented Dec 13, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment