Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BDD development mega issue #30

Open
6 of 7 tasks
tk3369 opened this issue Feb 14, 2021 · 3 comments · Fixed by #90
Open
6 of 7 tasks

BDD development mega issue #30

tk3369 opened this issue Feb 14, 2021 · 3 comments · Fixed by #90
Labels
enhancement New feature or request help wanted Extra attention is needed TODO Good for newcomers
Milestone

Comments

@tk3369
Copy link
Member

tk3369 commented Feb 14, 2021

This is a summary issue that keeps track of the work involved for implementing the BDD test suite for GraknClient.jl (for background, see issue #22). Note that ExecutableSpecifications.jl needs to be enhanced to support this project, and that's being tracked separately.

We should be able to largely model the work from Grakn's python-client. The links below point to that repo for reference.

@mkschulze mkschulze added enhancement New feature or request TODO Good for newcomers labels Feb 14, 2021
@alexjpwalker
Copy link

alexjpwalker commented Feb 15, 2021

This looks like a great summary of the various parts of the Grakn Python client's BDD test framework, @tk3369 !

The only notable omission is an automated workflow for executing the tests (say, in CI). In our Python client we have a CI job that clones the behaviour repo, downloads, unzips and runs Grakn, executes the tests storing the result to a variable, closes Grakn and returns the test result. I don't know if that would be applicable to this Julia project?

@tk3369
Copy link
Member Author

tk3369 commented Feb 15, 2021

That's a good point. Julia projects typically uses GitHub actions for CI. Is there any pre-made actions for Grakn? If not, we can probably set that up.

I was also a little curious about that because I see you use Bazel everywhere. We don't need to follow that, do we?

@alexjpwalker
Copy link

alexjpwalker commented Feb 16, 2021

We do use Bazel everywhere, however, protocol has been designed specifically with non-Bazel users in mind. It deploys a pip artifact for Python, and an npm artifact for JavaScript. As for behaviour, the build system is irrelevant because all a Grakn client needs to do is clone the behaviour repo and read the .feature files verbatim.

We don't use GitHub actions, so we don't have any "prebuilt" ones. For inspiration, here's the Bazel rule we're currently using to run BDD in client-python - which is mostly just a shell script (Bazel's syntax is based on Python!):

    # Store the path of the first feature file. It is recommended to only have one feature file.
    feats_dir = ctx.files.feats[0].dirname

    # behave requires a 'steps' folder to exist in the test root directory.
    steps_out_dir = ctx.files.feats[0].dirname + "/steps"

    grakn_distro = str(ctx.files.native_grakn_artifact[0].short_path)

    cmd = "set -e && GRAKN_DISTRO=%s" % grakn_distro
    cmd += """
           if test -d grakn_distribution; then
             echo Existing distribution detected. Cleaning.
             rm -rf grakn_distribution
           fi
           mkdir grakn_distribution
           echo Attempting to unarchive Grakn distribution from $GRAKN_DISTRO
           if [[ ${GRAKN_DISTRO: -7} == ".tar.gz" ]]; then
             tar -xf $GRAKN_DISTRO -C ./grakn_distribution
           else
             if [[ ${GRAKN_DISTRO: -4} == ".zip" ]]; then
               unzip -q $GRAKN_DISTRO -d ./grakn_distribution
             else
               echo Supplied artifact file was not in a recognised format. Only .tar.gz and .zip artifacts are acceptable.
               exit 1
             fi
           fi
           DIRECTORY=$(ls ./grakn_distribution)
           if [[ $GRAKN_DISTRO == *"cluster"* ]]; then
             PRODUCT=Cluster
           else
             PRODUCT=Core
           fi
           echo Successfully unarchived Grakn $PRODUCT distribution.
           RND=20001
           while [ $RND -gt 20000 ]  # Guarantee fair distribution of random ports
           do
             RND=$RANDOM
           done
           PORT=$((40000 + $RND))
           echo Starting Grakn $PRODUCT Server.
           mkdir ./grakn_distribution/"$DIRECTORY"/grakn_test
           if [[ $PRODUCT == "Core" ]]; then
             ./grakn_distribution/"$DIRECTORY"/grakn server --port $PORT --data grakn_test &
           else
             ./grakn_distribution/"$DIRECTORY"/grakn server --address "127.0.0.1:$PORT:$(($PORT+1))" --data grakn_test &
           fi
           POLL_INTERVAL_SECS=0.5
           MAX_RETRIES=60
           RETRY_NUM=0
           while [[ $RETRY_NUM -lt $MAX_RETRIES ]]; do
             RETRY_NUM=$(($RETRY_NUM + 1))
             if [[ $(($RETRY_NUM % 4)) -eq 0 ]]; then
               echo Waiting for Grakn $PRODUCT server to start \($(($RETRY_NUM / 2))s\)...
             fi
             lsof -i :$PORT && STARTED=1 || STARTED=0
             if [[ $STARTED -eq 1 ]]; then
               break
             fi
             sleep $POLL_INTERVAL_SECS
           done
           if [[ $STARTED -eq 0 ]]; then
             echo Failed to start Grakn $PRODUCT server
             exit 1
           fi
           echo Grakn $PRODUCT database server started
           """
    # TODO: If two step files have the same name, we should rename the second one to prevent conflict
    cmd += "cp %s %s" % (ctx.files.background[0].path, feats_dir)
    cmd += " && rm -rf " + steps_out_dir
    cmd += " && mkdir " + steps_out_dir + " && "
    cmd += " && ".join(["cp %s %s" % (step_file.path, steps_out_dir) for step_file in ctx.files.steps])
    cmd += " && behave %s -D port=$PORT && export RESULT=0 || export RESULT=1" % feats_dir
    cmd += """
           echo Tests concluded with exit value $RESULT
           echo Stopping server.
           kill $(lsof -i :$PORT | awk '/java/ {print $2}')
           exit $RESULT
           """

Hopefully, it should be fairly straightforward to port this to a GitHub action! The following lines:

    cmd += "cp %s %s" % (ctx.files.background[0].path, feats_dir)
    cmd += " && rm -rf " + steps_out_dir
    cmd += " && mkdir " + steps_out_dir + " && "
    cmd += " && ".join(["cp %s %s" % (step_file.path, steps_out_dir) for step_file in ctx.files.steps])
    cmd += " && behave %s -D port=$PORT && export RESULT=0 || export RESULT=1" % feats_dir

are specific to the behave test runner for Python, and are likely to work differently in Julia.

@mkschulze mkschulze added this to the Version 1.0 milestone Mar 7, 2021
@mkschulze mkschulze added the help wanted Extra attention is needed label Mar 13, 2021
@mkschulze mkschulze linked a pull request Aug 10, 2021 that will close this issue
@mkschulze mkschulze reopened this Sep 4, 2021
@mkschulze mkschulze changed the title GraknClient BDD development mega issue BDD development mega issue Sep 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed TODO Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants