Storage

Features

1. Single-primary mode

Normal mode

2. Dual-primary mode

GFS or OCFS2 file system. Two concurrent users can use it at the same time

3. Replication modes

   a. Protocol A

Asynchronous replication protocol:

local operations on the primary node are considered as completed when:

• Local write disk has finished

• Local data in TCP sender (ready to send to remote)

  b. Protocol B

Memory synchronous (semi-synchronous) replication protocol:

local operations on the primary node are considered as completed when:

• Local write disk has finished

• Replication packet has reached the target node

  c. Protocol C

Synchronous replication protocol:

local operations on the primary node are considered as completed when:

• Local write disk has finished
• Target write disk has finished

For Us the mode is in Replication Mode Protocol C

Synchronization is not replication: decoupled from I/O writes.

Dedicated direct link shall be used for the replication.

Configuration

DRBD uses ports 7788 and 7799

Configuration files:

• /etc/drbd.conf
• /etc/drbd.d/global_common.conf
• /etc/drbd.d/main.res

The DRBD device is /dev/drbd0

The DRBD volume attached to this device is /dev/vg_drbddata/lv_drbddata

cat /proc/drbd return the status of the drbd device

service drbd status as well but with the data state

drbdadm primary main is used to change the device's status

Idem with:

• drbdadm secondary main
• drbdadm primary main

Can be used to initialize and force the sync

service drbd status main
m:res   cs         ro                 ds                 p  mounted  fstype

0:main  Connected  Primary/Secondary  UpToDate/UpToDate  C

cat /proc/drbd
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----

    ns:0 nr:0 dw:0 dr:744 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

Meaning:

• Cs=connection state: should be connected, not StandAlone
• Res=Ressource
• Ro=role: primary has read and write, secondary: no read and no write
• Ds=Disk state: should be UpToDate (consistent as well)
• P=Replication Protocol: A, B or C

I/O state flags:

• d: I/O blocked for a reason internal to DRBD, such as a transient disk state.

• b: Backing device I/O is blocking.

• n: Congestion on the network socket.

• a: Simultaneous combination of blocking device I/O and network congestion.

• ns (network send). Volume of net data sent to the partner via the network connection; in Kibyte.

• nr (network receive). Volume of net data received by the partner via the network connection; in Kibyte.

• dw (disk write). Net data written on local hard disk; in Kibyte.

• dr (disk read). Net data read from local hard disk; in Kibyte.

• al (activity log). Number of updates of the activity log area of the meta data.

• bm (bit map). Number of updates of the bitmap area of the meta data.

• lo (local count). Number of open requests to the local I/O sub-system issued by DRBD.

• pe (pending). Number of requests sent to the partner, but that have not yet been answered by the latter.

• ua (unacknowledged). Number of requests received by the partner via the network connection, but that have not yet been answered.

• ap (application pending). Number of block I/O requests forwarded to DRBD, but not yet answered by DRBD.

• ep (epochs). Number of epoch objects. Usually 1. Might increase under I/O load when using either the barrieror the none write ordering method.

• wo (write order). Currently used write ordering method: b(barrier), f(flush), d(drain) or n(none).

• oos (out of sync). Amount of storage currently out of sync; in Kibibytes.

drbdadm up main
drbdadm down main

cat /etc/drbd.d/main.res

on NODE-A{

        device /dev/drbd0;

        disk /dev/vg_drbddata/lv_drbddata;

        meta-disk internal;

        address A.B.C.1:7788;

}

• /dev/drbd/by-disk/vg_drbddata/lv_drbddata -> /dev/drbd0
• /dev/drbd/by-res/main -> /dev/drbd0
• /dev/drbd0 should be mounted to /drbddata

drbdadm show-gi main


       +--<  Current data generation UUID  >-

       |               +--<  Bitmap's base data generation UUID  >-

       |               |                 +--<  younger history UUID  >-

       |               |                 |         +-<  older history  >-

       V               V                 V         V

55927E99142A9411:0000000000000000:9B4A3B34D7E9C769:9B493B34D7E9C769:1:1:1:1:0:0:0

                                                                                                    ^ ^ ^ ^ ^ ^ ^

                                                    -<  Data consistency flag  >--+ | | | | | |

                                -<  Data was/is currently up-to-date  >--+ | | | | |

                                      -<  Node was/is currently primary  >--+ | | | |

                                    -<  Node was/is currently connected  >--+ | | |

            -<  Node was in the progress of setting all bits in the bitmap  >--+ | |

                          -<  The peer's disk was out-dated or inconsistent  >--+ |

         -<  This node was a crashed primary, and has not seen its peer since   >--+

• flags: Primary, Connected, UpToDate
• meta-data: need apply-al

Sources

DRBD Documentation