Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#150][#178] feat(all): Add launching process and scripts #191

Merged
merged 27 commits into from
Aug 11, 2023
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/workflows/integration.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: integration

# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "main" branch
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

concurrency:
group: ${{ github.worklfow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_requests' }}

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
integration_test:
# The type of runner that the job will run on
runs-on: ubuntu-latest

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3

- uses: actions/setup-java@v3
with:
java-version: '8'
distribution: 'temurin'

- name: Setup Gradle
uses: gradle/gradle-build-action@v2
with:
gradle-version: '8.1.1'

- name: Show gradle version
run: gradle --version

- name: Package Graviton
run: |
gradle build
gradle compileDistribution
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,6 @@ gen
*.iml
out/**
*.iws

distribution
server/src/main/resources/project.properties
80 changes: 80 additions & 0 deletions bin/common.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
#!/bin/bash
#
# Copyright 2023 Datastrato.
# This software is licensed under the Apache License version 2.
#

if [ -L "${BASH_SOURCE-$0}" ]; then
FWDIR=$(dirname "$(readlink "${BASH_SOURCE-$0}")")
else
FWDIR=$(dirname "${BASH_SOURCE-$0}")
fi

echo ${GRAVITON_HOME}

if [[ -z "${GRAVITON_HOME}" ]]; then
export GRAVITON_HOME="$(cd "${FWDIR}/.." || exit; pwd)"
fi

if [[ -z "${GRAVITON_CONF_DIR}" ]]; then
export GRAVITON_CONF_DIR="${GRAVITON_HOME}/conf"
fi

if [[ -z "${GRAVITON_LOG_DIR}" ]]; then
export GRAVITON_LOG_DIR="${GRAVITON_HOME}/logs"
fi

if [[ -f "${GRAVITON_CONF_DIR}/graviton-env.sh" ]]; then
. "${GRAVITON_CONF_DIR}/graviton-env.sh"
fi

GRAVITON_CLASSPATH+=":${GRAVITON_CONF_DIR}"

function check_java_version() {
if [[ -n "${JAVA_HOME+x}" ]]; then
JAVA="$JAVA_HOME/bin/java"
fi
java_ver_output=$("${JAVA:-java}" -version 2>&1)
jvmver=$(echo "$java_ver_output" | grep '[openjdk|java] version' | awk -F'"' 'NR==1 {print $2}' | cut -d\- -f1)
JVM_VERSION=$(echo "$jvmver"|sed -e 's|^\([0-9][0-9]*\)\..*$|\1|')
if [ "$JVM_VERSION" = "1" ]; then
JVM_VERSION=$(echo "$jvmver"|sed -e 's|^1\.\([0-9][0-9]*\)\..*$|\1|')
fi

if [ "$JVM_VERSION" -lt 8 ] || { [ "$JVM_VERSION" -eq 8 ] && [ "${jvmver#*_}" -lt 151 ]; } ; then
echo "Graviton requires either Java 8 update 151 or newer"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Java 8 enough, do we require a Java version larger than 151? Also, I was worrying about the way you get Java version using regex, you may not get the version number from some customized java release like zuul, kona jdk?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Because after JDK 8u151, Oracle fixed a number of security vulnerabilities and issues to improve system stability and security. We suggest using after JDK 8u151.
    https://www.oracle.com/java/technologies/javase/8u151-relnotes.html

  2. Because we only test passed in Oracle JDK, Other JDK versions not tested, better limit JDK type & version.

Copy link
Member

@justinmclean justinmclean Aug 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest not using Oracle JDK 8 as it has a restrictive license. Oracle JDK Java 8 is under the OTN License Agreement for Java (https://www.oracle.com/cis/downloads/licenses/oracle-javase-license.html). This requires a fee / license for commercial use.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest using OpenJDK / Corretto.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi, @justinmclean thank you for your suggestion.
I create issue #215 to track this problem.

exit 1;
fi
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indention of this method is 4 spaces.

Copy link
Member Author

@xunliu xunliu Aug 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indention of this method is 4 spaces.

This is not fixed.


function addEachJarInDir(){
if [[ -d "${1}" ]]; then
for jar in "${1}"/*.jar ; do
GRAVITON_CLASSPATH="${jar}:${GRAVITON_CLASSPATH}"
done
fi
}

function addEachJarInDirRecursive(){
if [[ -d "${1}" ]]; then
for jar in "${1}"/**/*.jar ; do
GRAVITON_CLASSPATH="${jar}:${GRAVITON_CLASSPATH}"
done
fi
}

function addJarInDir(){
if [[ -d "${1}" ]]; then
GRAVITON_CLASSPATH="${1}/*:${GRAVITON_CLASSPATH}"
fi
}

if [[ -z "${GRAVITON_MEM}" ]]; then
export GRAVITON_MEM="-Xmx1024m"
fi

if [[ -n "${JAVA_HOME}" ]]; then
export JAVA_RUNNER="${JAVA_HOME}/bin/java"
else
export JAVA_RUNNER=java
fi
152 changes: 152 additions & 0 deletions bin/graviton.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
#!/bin/bash
#
# Copyright 2023 Datastrato.
# This software is licensed under the Apache License version 2.
#
#set -ex
USAGE="-e Usage: bin/graviton.sh [--config <conf-dir>]\n\t
{start|stop|restart|status}"

if [[ "$1" == "--config" ]]; then
shift
conf_dir="$1"
if [[ ! -d "${conf_dir}" ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be it is not necessary to add --config <conf-dir>? I think we can use GRAVITON_CONF_DIR or GRAVITON_HOME to infer the conf dir programatically.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --config parameter is an option that allows us to easily change the configuration directory in the start scripts.

  • When the --config parameter is not specified, GRAVITON_CONF_DIR is the default environment in the graviton-env.sh
  • When the --config parameter is specified, the GRAVITON_CONF_DIR environment is overwritten.

echo "ERROR : ${conf_dir} is not a directory"
echo ${USAGE}
exit 1
else
export GRAVITON_CONF_DIR="${conf_dir}"
fi
shift
fi

bin="$(dirname "${BASH_SOURCE-$0}")"
bin="$(cd "${bin}">/dev/null; pwd)"

. "${bin}/common.sh"

check_java_version

function check_process_status() {
local pid=$(found_graviton_server_pid)

if [[ -z "${pid}" ]]; then
echo "Graviton Server is not running"
else
echo "Graviton Server is running[PID:$pid]"
fi
}

function found_graviton_server_pid() {
process_name='GravitonServer';
RUNNING_PIDS=$(ps x | grep ${process_name} | grep -v grep | awk '{print $1}');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to write pid to a file to avoid same name process?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using PID file is a traditional operation, and I have implemented this function in another version.

  • Generation the PID file in the graviton.sh start command.
  • Deleted the PID file in the graviton.sh stop command.

However, when the Graviton Server itself exits abnormally, the PID file is not deleted properly.
You will not be able to restart Gravtion Server the next time because there is already an old PID file.
So I think might be better to get the directory PID in the operation system.

I suggest that we can keep this implementation, and consider modifying it later when we have a specific problem.
What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, sure.


if [[ -z "${RUNNING_PIDS}" ]]; then
return
fi

if ! kill -0 ${RUNNING_PIDS} > /dev/null 2>&1; then
echo "Graviton Server running but process is dead"
fi

echo "${RUNNING_PIDS}"
}

function wait_for_graviton_server_to_die() {
timeout=10
timeoutTime=$(date "+%s")
let "timeoutTime+=$timeout"
currentTime=$(date "+%s")
forceKill=1

while [[ $currentTime -lt $timeoutTime ]]; do
local pid=$(found_graviton_server_pid)
if [[ -z "${pid}" ]]; then
forceKill=0
break
fi

$(kill ${pid} > /dev/null 2> /dev/null)
if kill -0 ${pid} > /dev/null 2>&1; then
sleep 3
else
forceKill=0
break
fi
currentTime=$(date "+%s")
done

if [[ forceKill -ne 0 ]]; then
$(kill -9 ${pid} > /dev/null 2> /dev/null)
fi
}

function start() {
local pid=$(found_graviton_server_pid)

if [[ ! -z "${pid}" ]]; then
if kill -0 ${pid} >/dev/null 2>&1; then
echo "Graviton Server is already running"
return 0;
fi
fi

if [[ ! -d "${GRAVITON_LOG_DIR}" ]]; then
echo "Log dir doesn't exist, create ${GRAVITON_LOG_DIR}"
mkdir -p "${GRAVITON_LOG_DIR}"
fi

nohup ${JAVA_RUNNER} ${JAVA_OPTS} ${GRAVITON_DEBUG_OPTS} -cp ${GRAVITON_CLASSPATH} ${GRAVITON_SERVER_NAME} >> "${GRAVITON_OUTFILE}" 2>&1 < /dev/null &

pid=$!
if [[ -z "${pid}" ]]; then
echo "Graviton Server start error!"
return 1;
else
echo "Graviton Server start success!"
fi

sleep 2
check_process_status
}

function stop() {
local pid

pid=$(found_graviton_server_pid)

if [[ -z "${pid}" ]]; then
echo "Graviton Server is not running"
else
wait_for_graviton_server_to_die
echo "Graviton Server stop"
fi
}

HOSTNAME=$(hostname)
GRAVITON_OUTFILE="${GRAVITON_LOG_DIR}/graviton-${HOSTNAME}.out"
GRAVITON_SERVER_NAME=com.datastrato.graviton.server.GravitonServer

JAVA_OPTS+=" -Dfile.encoding=UTF-8"
JAVA_OPTS+=" -Dlog4j2.configurationFile=file://${GRAVITON_CONF_DIR}/log4j2.properties"
JAVA_OPTS+=" -Dgraviton.log.path=${GRAVITON_LOG_DIR} ${GRAVITON_MEM}"

addJarInDir "${GRAVITON_HOME}/lib"

case "${1}" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
status)
check_process_status
;;
*)
echo ${USAGE}
esac
Loading