Skip to content

Performance Benchmark with Scala Edition

Jump to bottom
Jerry.Wang edited this page May 8, 2021 · 9 revisions

UPDATE 2021-05-02: Tried some optimization in Rust edition, the result is updated.

UPDATE 2021-05-08: Tried more optimization in Rust edition.

Source code

The prototype of rho-runtime is out, it only supports a few keywords, including send/receive/par/if/int, with an in-memory tuplespace. Although the language feature is limited at this moment, no pattern matching, even error check is not all in place, still we can perform a test to compare its performance with the Scala edition.

Thanks to @zsluedem, he provides a special edition of Scala RNode, which gets rid of LMD, performs tuplespace operations in memory, and also outputs the reduction duration in console window. Without his assistance, we cannot have this test.

Test Environment

  • 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz, 8 cores
  • 16G RAM & SSD
  • Ubuntu 20 LTS, Switched to Performance Mode

The Scala and Rust edition both run in Docker Container.

Scala Edition

The RNode server is started as below.

sudo docker rm -f rnode
sudo docker run -d --name rnode --rm \
    --net=host \
    -p 40401:40401 \
    -p 40402:40402 \
    -p 40403:40403 \
    -p 40404:40404 \
    -p 40405:40405 \
    -v $HOME/var/rnode1:/var/lib/rnode \
    -v $CURRENT_DIR:/rholang \
    -v $CURRENT_DIR/wallets.txt:/var/lib/rnode/genesis/wallets.txt:ro \
    -v $CURRENT_DIR/bonds.txt:/var/lib/rnode/genesis/bonds.txt:ro \
    zsluedem/rnode:rholang-eval-inmem run --standalone -XX:MaxRAMPercentage=80 -XX:MaxDirectMemorySize=2g

And the RNode client sends EVAL commands to it

sudo docker run --rm --name rnode-client \
    --network host -it \
    -v $CURRENT_DIR:/rholang:ro \
    zsluedem/rnode:rholang-eval-inmem \
    --grpc-port 40402 eval /rholang/bench1.rho

Rust Edition

It is a single command to launch the docker container by mounting the source code as below.

sudo docker run -ti \
    -v $CURRENT_DIR/bench1.rho:/source.rho \
    wangjia184/rho_runtime:0.0.3 /source.rho

How test is performed?

Special designed rholang codes are sent to them to evaluate. And their console outputs the reduction duration. Given that JVM compiles into machine code just-in-time, hence for each code snippet, runs it three times and the fastest record is taken.

Test Result

Case 1 - Parallel Execution

This case is special designed for 8-cores.

new cnt, x, sumx, y, sumy, z, sumz, v, sumv, u, sumu, q, sumq, w, sumw, stdout(`rho:io:stdout`) in {
    cnt!(0) |
    sumx!(0) | sumy!(0) | sumz!(0) | sumv!(0) | sumu!(0) | sumq!(0) | sumw!(0) |
    for( num <= cnt ) {
        if( *num < 10000 ) {
            cnt!(*num+1) |
            x!(*num+1) | y!(*num+1) | z!(*num+1) | v!(*num+1) | u!(*num+1) | q!(*num+1) | w!(*num+1)
        } else {
            stdout!(*num)
        }
    } |
    for( a <= x; sum1 <= sumx ){
        if( *sum1+*a < 50005000 ) { sumx!(*sum1+*a) } else { stdout!(*sum1+*a) }
    } |
    for( a <= y; sum <= sumy ){
        if( *sum+*a < 50005000 ) { sumy!(*sum+*a) } else { stdout!(*sum+*a) }
    } |
    for( a <= z; sum <= sumz ){
        if( *sum+*a < 50005000 ) { sumz!(*sum+*a) } else { stdout!(*sum+*a) }
    } |
    for( a <= v; sum <= sumv ){
        if( *sum+*a < 50005000 ) { sumv!(*sum+*a) } else { stdout!(*sum+*a) }
    } |
    for( a <= u; sum <= sumu ){
        if( *sum+*a < 50005000 ) { sumu!(*sum+*a) } else { stdout!(*sum+*a) }
    } |
    for( a <= q; sum <= sumq ){
        if( *sum+*a < 50005000 ) { sumq!(*sum+*a) } else { stdout!(*sum+*a) }
    } |
    for( a <= w; sum <= sumw ){
       if( *sum+*a < 50005000 ) { sumw!(*sum+*a) } else { stdout!(*sum+*a) }
    }
}
Edition Round 1 Round 2 Round 3 Times
Scala 43538 ms 41709 ms 41623 ms x 1
Rust 0.0.1 563 ms 578ms 562 ms x 74
Rust 0.0.2 418 ms 410ms 389 ms x 107
Rust 0.0.3 348 ms 328ms 350 ms x 126

Case 2 - Sequence Execution

new countCh, sumCh, stdout(`rho:io:stdout`) in {
    countCh!(0) |
    sumCh!(0) |
    for( count <= countCh; sum <= sumCh ) {
        if(*count < 100000) {
            countCh!(*count + 1) |
            sumCh!(*sum + *count + 1)
        } else {
            stdout!(*sum)
        }
    }
}
Edition Round 1 Round 2 Round 3 Times
Scala 112195 ms 109148 ms 108708 ms x 1
Rust 0.0.1 3676 ms 3690ms 3393 ms x 32
Rust 0.0.2 3020 ms 3106 ms 3109 ms x 35
Rust 0.0.3 2248 ms 2224 ms 2236 ms x 48
Clone this wiki locally