集群配置问题，主机Agent分发失败 #333

yzhang921 · 2023-07-10T01:20:04Z

yzhang921
Jul 10, 2023

版本 datasophon-manager-1.1.1

work貌似正常分发启动，但是manager获取不到状态，主机Agent分发

$ jps
52995 DDHApplicationServer
82008 Jps
74683 WorkerApplicationServer

work信息如下：

[com.datasophon.common.model.StartWorkerMessage] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'
[INFO] 2023-07-09 21:05:51 com.datasophon.common.utils.ShellUtils:[102] - 脚本返回的数据如下： x86_64
[INFO] 2023-07-09 21:05:51 com.datasophon.common.utils.ShellUtils:[209] - stopping node
[INFO] 2023-07-09 21:05:54 com.datasophon.common.utils.ShellUtils:[209] - End stop node.
[INFO] 2023-07-09 21:05:54 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:05:54 com.datasophon.worker.WorkerApplicationServer:[166] - Worker server stopped
[INFO] 2023-07-09 21:06:08 com.datasophon.common.utils.ShellUtils:[102] - 脚本返回的数据如下： x86_64
[INFO] 2023-07-09 21:06:09 akka.event.slf4j.Slf4jLogger:[92] - Slf4jLogger started
[INFO] 2023-07-09 21:06:09 akka.remote.Remoting:[83] - Starting remoting
[INFO] 2023-07-09 21:06:09 akka.remote.Remoting:[83] - Remoting started; listening on addresses :[akka.tcp://datasophon@centos3-master:2552]
[INFO] 2023-07-09 21:06:09 akka.remote.Remoting:[83] - Remoting now listens on addresses: [akka.tcp://datasophon@centos3-master:2552]
[INFO] 2023-07-09 21:06:09 com.datasophon.common.utils.ShellUtils:[209] - no node to stop
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - starting node, logging to /opt/datasophon/datasophon-worker/node/x86/logs/node-centos3-master.out
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - nohup /opt/datasophon/datasophon-worker/node/x86/node_exporter > /opt/datasophon/datasophon-worker/node/x86/logs/node-centos3-master.out 2>&1 &
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - End restart node.
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - uid=1000(hive) gid=1000(hadoop) groups=1000(hadoop)
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - usermod: no changes
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - uid=1001(elastic) gid=1001(elastic) groups=1001(elastic)
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - usermod: no changes
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - uid=1002(hdfs) gid=1000(hadoop) groups=1000(hadoop)
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - usermod: no changes
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - uid=1003(yarn) gid=1000(hadoop) groups=1000(hadoop)
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - usermod: no changes
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - uid=1004(mapred) gid=1000(hadoop) groups=1000(hadoop)
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - usermod: no changes
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - uid=1005(hbase) gid=1000(hadoop) groups=1000(hadoop)
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[209] - usermod: no changes
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[127] - script execute success
[INFO] 2023-07-09 21:06:19 com.datasophon.common.utils.ShellUtils:[72] - 脚本返回的数据如下：{coreNum: 16, totalMem: 3.68164, totalDisk: 36.09}
[INFO] 2023-07-09 21:06:19 com.datasophon.worker.WorkerApplicationServer:[155] - host info collect result:com.datasophon.common.utils.ExecResult@5a1de7fb
[INFO] 2023-07-09 21:06:19 com.datasophon.worker.WorkerApplicationServer:[93] - start worker
[INFO] 2023-07-09 21:06:19 com.datasophon.worker.actor.RemoteEventActor:[39] - akka.tcp://datasophon@centos3-master:2552-->akka.tcp://datasophon@centos3-master:2551 associated
[WARN] 2023-07-09 21:06:19 akka.serialization.Serialization(akka://datasophon):[78] - Using the default Java serializer for class [com.datasophon.common.model.StartWorkerMessage] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'

datasophon · 2023-07-10T01:35:54Z

datasophon
Jul 10, 2023
Maintainer

主机名不要带 - 这类的特殊字符

14 replies

datasophon Jul 10, 2023
Maintainer

worker正常启动但分发失败的原因是worker发送给master的启动成功信息没有被master接收到，一般情况下都是主机名设置的问题，文档有说明如何设置主机名，当然不排除其他可能得原因，需要结合环境具体分析

yzhang921 Jul 10, 2023
Author

目前测试的4台机器
主机名如下，按照你第一次回复就已经改为没有任何特殊字符了

centos3master
centos4slave1
centos5slave2
centos6slave3

/etc/hosts配置, 不知道主机名和host是否需要一致，文档里面没有说明，只是建议的主机名，如果有硬性要求，建议可以明确说明

$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.10.3 node1
192.168.10.4 node2
192.168.10.5 node3
192.168.10.6 node4

文件是manger和work的全部日志，希望对分析有帮助
datasohpon-log.zip

还有一点要说明一下，测试环境数据库MySQL较高为8.x，项目中的驱动包为5.x，连接数据库的时候替换了一下jdbc jar包，刚才发现日志中有mybatis报错的信息，不知道是否有影响

datasophon Jul 10, 2023
Maintainer

你需要设置主机为node1 ... ，使用该命令设置hostnamectl set-hostname <主机名> ，同时需要配置/etc/hosts文件，与设置的保持一致，在你的日志中，我看到服务获取的主机名和你在页面上配置的主机名不一致

yzhang921 Jul 10, 2023
Author

修改host主机名和/etc/hosts一致之后Agent部署成功，建议把这两点写到文档中去
从项目解决的问题来看，确实是很棒的项目，希望项目越做越好:thumbsup:

datasophon Jul 10, 2023
Maintainer

感谢认可

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

集群配置问题，主机Agent分发失败 #333

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment 14 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

集群配置问题，主机Agent分发失败 #333

yzhang921 Jul 10, 2023

Replies: 1 comment · 14 replies

datasophon Jul 10, 2023 Maintainer

datasophon Jul 10, 2023 Maintainer

yzhang921 Jul 10, 2023 Author

datasophon Jul 10, 2023 Maintainer

yzhang921 Jul 10, 2023 Author

datasophon Jul 10, 2023 Maintainer

yzhang921
Jul 10, 2023

Replies: 1 comment 14 replies

datasophon
Jul 10, 2023
Maintainer

datasophon Jul 10, 2023
Maintainer

yzhang921 Jul 10, 2023
Author

datasophon Jul 10, 2023
Maintainer

yzhang921 Jul 10, 2023
Author

datasophon Jul 10, 2023
Maintainer