Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat] add http deployment method #365

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

liuminjian
Copy link

@liuminjian liuminjian commented Nov 24, 2023

Issue Number: #328

1.增加http daemon的部署方式,host增加protocol和http port字段
image
image

2.在每个host节点上部署http daemon
image

3.部署http daemon后能正常部署curve集群
image

image image

遗留问题:目前enter指令不支持http方式

@caoxianfei1
Copy link
Contributor

@liuminjian BTW, we can provide command curveadm http start/stop is better? And the http service commit has already done.
So you can refer the commit.

@liuminjian
Copy link
Author

@liuminjian BTW, we can provide command curveadm http start/stop is better? And the http service commit has already done. So you can refer the commit.

ok

@@ -52,7 +52,7 @@ func NewAddTargetTask(curveadm *cli.CurveAdm, cc *configure.ClientConfig) (*task
}

subname := fmt.Sprintf("host=%s volume=%s", options.Host, volume)
t := task.NewTask("Add Target", subname, hc.GetSSHConfig())
t := task.NewTask("Add Target", subname, hc.GetSSHConfig(), hc.GetHttpConfig())
Copy link
Collaborator

@Wine93 Wine93 Nov 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的 GetSSHConfigGetHttpConfig 可以合并成一个函数,比如 GetConnectConfig,然后在 NewTask 里面再做处理,这样后续有其他的连接方式控制,扩展性也会好一些。

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW:尽可能保持变量命名方式的一致性,比如用大写,那么可以用 SSHHTTP,或者都用 sshhttp

@@ -48,7 +48,7 @@ func NewBalanceTask(curveadm *cli.CurveAdm, dc *topology.DeployConfig) (*task.Ta

subname := fmt.Sprintf("host=%s role=%s containerId=%s",
dc.GetHost(), dc.GetRole(), tui.TrimContainerId(containerId))
t := task.NewTask("Balance Leader", subname, hc.GetSSHConfig())
t := task.NewTask("Balance Leader", subname, hc.GetSSHConfig(), hc.GetHttpConfig())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

@@ -0,0 +1,44 @@
/*
Copy link
Collaborator

@Wine93 Wine93 Nov 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个命令我认为叫 deamon 更好一些。
另外目前的做法是安装 pigeon 二进制,然后在 http start 时启动这个二进制,其实这是没必要的,我们可以把 pigeon 的 start 命令逻辑复制到 http start 这个命令中,详见:pigeon start

目前:

curveadm http start -> pigeon start 

期待:

curveadm deamon start

@@ -0,0 +1,71 @@
/*
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个 HTTP 的核心逻辑建议移动到 internal 目录下,建议叫 daemon



# generate http service config file
local httpConfpath="${g_curveadm_home}/conf/pigeon.yaml"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议在 curveadm 的主目录下新建一个 deamon 目录,配置文件 pigeon.yaml 以及一些 daemon 的日志都可以放到这个目录下

/*
* Project: Curveadm
* Created Date: 2023-03-31
* Author: wanghai (SeanHai)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的作者建议改一下~

func NewDaemonCommand(curveadm *cli.CurveAdm) *cobra.Command {
cmd := &cobra.Command{
Use: "daemon",
Short: "Manage http service",
Copy link
Collaborator

@Wine93 Wine93 Dec 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manage deamon service

@@ -66,6 +67,7 @@ func addSubCommands(cmd *cobra.Command, curveadm *cli.CurveAdm) {
target.NewTargetCommand(curveadm), // curveadm target ...
pfs.NewPFSCommand(curveadm), // curveadm pfs ...
monitor.NewMonitorCommand(curveadm), // curveadm monitor ...
daemon.NewDaemonCommand(curveadm), // curveadm http
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curveadm deamon ...

Makefile Outdated
@@ -21,6 +21,7 @@ GO := go

# output
OUTPUT := bin/curveadm
SERVER_OUTPUT := bin/pigeon
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是不需要了?

@liuminjian liuminjian force-pushed the feat/http branch 2 times, most recently from d51ef71 to a7b3b84 Compare December 13, 2023 08:35
/*
* Project: Curveadm
* Created Date: 2023-03-31
* Author: wanghai (SeanHai)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

/*
* Project: Curveadm
* Created Date: 2023-03-31
* Author: wanghai (SeanHai)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto


cmd := &cobra.Command{
Use: "stop",
Short: "Stop http service",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stop deamon service


cmd := &cobra.Command{
Use: "start [OPTIONS]",
Short: "Start http service",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Start deamon service

*/

/*
* Project: Curveadm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CurveAdm

*/

/*
* Project: Curveadm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

*/

/*
* Project: Curveadm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.


const (
STOP_EXAMPLR = `Examples:
$ curveadm daemon stop -c ~/.curveadm/daemon/config/pigeon.yaml # Stop an daemon http service`
Copy link
Collaborator

@Wine93 Wine93 Dec 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ curveadm daemon stop  # Stop daemon service


const (
START_EXAMPLR = `Examples:
$ curveadm daemon start -c ~/.curveadm/daemon/config/pigeon.yaml # Start an daemon http service to receive requests`
Copy link
Collaborator

@Wine93 Wine93 Dec 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ curveadm daemon start   # Start daemon service

func (hc *HostConfig) GetSSHConfig() *module.SSHConfig {

Copy link
Collaborator

@Wine93 Wine93 Dec 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

多余的空格.

@@ -103,3 +109,40 @@ func (hc *HostConfig) GetSSHConfig() *module.SSHConfig {
ConnectRetries: curveadm.GlobalCurveAdmConfig.GetSSHRetries(),
}
}

func (hc *HostConfig) GetHTTPConfig() *module.HTTPConfig {

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

}
}

func (hc *HostConfig) GetConnectConfig() *module.ConnectConfig {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

既然有了 GetConnectConfig 函数,GetSSHConfigGetHTTPConfig 这 2 个函数应该可以移除了.

* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议加上 Author 等相关信息。

* See the License for the specific language governing permissions and
* limitations under the License.
*/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

* See the License for the specific language governing permissions and
* limitations under the License.
*/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

* See the License for the specific language governing permissions and
* limitations under the License.
*/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

* See the License for the specific language governing permissions and
* limitations under the License.
*/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

@@ -0,0 +1,166 @@
/*
* Copyright (c) 2021 NetEase Inc.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2021 -> 2023.

return
}

if cfg.Protocol == SSH_PROTOCOL {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的逻辑可以简化一下:

if cfg.Protocol == HTTP_PROTOCOL {
    client, err = NewHTTPClient(*cfg.GetHTTPConfig())
} else {
    client, err = NewSSHClient(*cfg.GetSSHConfig())
}

if (err != nil) {
    ...
}
return client, err

@@ -65,6 +65,18 @@ timeout = 10

[database]
url = "${g_db_path}"
__EOF__
fi

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

多余的空格

}

func (client *HttpClient) Upload(localPath string, remotePath string) (err error) {
bodyBuf := &bytes.Buffer{}
Copy link
Collaborator

@Wine93 Wine93 Dec 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的 uploaddownload 都太麻烦了,可以看下 resty 库,这个模块只要准备要命令就可以了,客户端这些逻辑可以交给 resty 去处理。

}

func (client *HttpClient) RunCommand(ctx context.Context, command string) (out []byte, err error) {
data := make(map[string]interface{})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.



# generate http service config file
local httpConfpath="${g_curveadm_home}/daemon/conf/pigeon.yaml"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个逻辑可以单独拆成一个函数,包括上面的 curveadm.cfg 都可以单独拆一下。

@caoxianfei1
Copy link
Contributor

@liuminjian please fix the conflict.

@liuminjian liuminjian force-pushed the feat/http branch 2 times, most recently from 07fda7a to 36353d1 Compare December 26, 2023 02:15
@liuminjian
Copy link
Author

@caoxianfei1 fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants