Tutorial: Build A Scalable and Reliable Counter Service

We will continue using the counter example and turn it from a single node service to a partitioned and replicated service for scalability and reliability. A more serious case is RocksDB.

Many people asked about what kinds of replication protocol we're using. Currently, our protocol largely follows PacificA.

STEP 1. continue the counter example, implement the virtual methods required by our replication layer

This is the extra step we need for replication, mostly about writing helper functions for replication so that it can work smoothly and efficiently for the given application.

        # include <dsn/cpp/replicated_service_app.h>

        class counter_service_impl :
            public counter_service,
            public replicated_service_app_type_1
        {
        public:
            virtual ::dsn::error_code start(int argc, char** argv) override;

            virtual ::dsn::error_code stop(bool cleanup = false) override;

            virtual ::dsn::error_code sync_checkpoint(int64_t last_commit) override;

            virtual int64_t get_last_checkpoint_decree() override { return last_durable_decree(); }

            virtual ::dsn::error_code get_checkpoint(
                int64_t learn_start,
                int64_t local_commit,
                void*   learn_request,
                int     learn_request_size,
                app_learn_state& state
            ) override;

            virtual ::dsn::error_code apply_checkpoint(
                dsn_chkpt_apply_mode mode,
                int64_t local_commit,
                const dsn_app_learn_state& state
            ) override;
        ...

Implementation for our counter service example is here.

STEP 2. Register the implementation, compile and run

We register the new service implementation to rDSN in main (done by code generation), and compile should be ok (do not forget to include the header file for counter_service_impl).

// register replication application provider
dsn::register_app_with_type_1_replication_support< ::dsn::example::counter_service_impl>("counter");

Before running the system, we need to configure the meta server and the replica servers. See config.ini here. Note we also specify what service interfaces need to be replicated in configuration file.

[task.RPC_COUNTER_COUNTER_READ]
rpc_request_is_write_operation = true

Then go to the directory where 'counter.exe' is generated and run; make sure 'config.ini' is also copied there.

00:00:35.045(35045) replica3.replication0.0000000003c93bc8: 1.0 @ localhost:34803: mutation 14.39 ack_prepare_message
00:00:35.057(35057) replica2.replication0.0000000003c93ec8: 1.0 @ localhost:34802: mutation 14.39 on_append_log_completed, err = 0
00:00:35.057(35057) replica2.replication0.0000000003c93ec8: 1.0 @ localhost:34802: mutation 14.39 ack_prepare_message
00:00:35.078(35078) replica1.replication0.0000000003c932c8: 1.0 @ localhost:34801: mutation 14.39 on_prepare_reply from localhost:34803
00:00:35.091(35091) replica1.replication0.0000000003c93088: 1.0 @ localhost:34801: mutation 14.39 on_prepare_reply from localhost:34802
00:00:35.091(35091) replica1.replication0.0000000003c93088: TwoPhaseCommit, 1.0 @ localhost:34801: mutation 14.39 committed, err = 0
call RPC_COUNTER_COUNTER_ADD end, return ERR_SUCCESS
call RPC_COUNTER_COUNTER_READ end, return ERR_SUCCESS
00:00:35.624(35624) replica1.replication0.0000000003c8c1a0: 1.0 @ localhost:34801: mutation 14.40 init_prepare
00:00:35.624(35624) replica1.replication0.0000000003c8c1a0: 1.0 @ localhost:34801: mutation 14.40 send_prepare_message to localhost:34803 as PS_SECONDARY
00:00:35.624(35624) replica1.replication0.0000000003c8c1a0: 1.0 @ localhost:34801: mutation 14.40 send_prepare_message to localhost:34802 as PS_SECONDARY
00:00:35.676(35676) replica3.replication0.0000000003c93fa8: 1.0 @ localhost:34803: mutation 14.40 on_prepare
00:00:35.676(35676) replica3.replication0.0000000003c93fa8: TwoPhaseCommit, 1.0 @ localhost:34803: mutation 14.39 committed, err = 0
00:00:35.705(35705) replica2.replication0.0000000000eae958: 1.0 @ localhost:34802: mutation 14.40 on_prepare
00:00:35.705(35705) replica2.replication0.0000000000eae958: TwoPhaseCommit, 1.0 @ localhost:34802: mutation 14.39 committed, err = 0
00:00:35.724(35724) replica1.replication0.0000000003c93988: 1.0 @ localhost:34801: mutation 14.40 on_append_log_completed, err = 0
00:00:35.776(35776) replica3.replication0.0000000003c93d48: 1.0 @ localhost:34803: mutation 14.40 on_append_log_completed, err = 0
00:00:35.776(35776) replica3.replication0.0000000003c93d48: 1.0 @ localhost:34803: mutation 14.40 ack_prepare_message