diff --git a/docs/guides/mongodb/backup/kubestash/_index.md b/docs/guides/mongodb/backup/kubestash/_index.md deleted file mode 100644 index 1c38ebf73..000000000 --- a/docs/guides/mongodb/backup/kubestash/_index.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: Backup & Restore MongoDB | KubeStash -menu: - docs_{{ .version }}: - identifier: guides-mongodb-backup-stashv2 - name: KubeStash (aka Stash 2.0) - parent: guides-mongodb-backup - weight: 50 -menu_name: docs_{{ .version }} ---- \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/auto-backup/_index.md b/docs/guides/mongodb/backup/kubestash/auto-backup/_index.md deleted file mode 100644 index abde5fc78..000000000 --- a/docs/guides/mongodb/backup/kubestash/auto-backup/_index.md +++ /dev/null @@ -1,369 +0,0 @@ ---- -title: MongoDB Auto-Backup | KubeStash -description: Backup MongoDB using KubeStash Auto-Backup -menu: - docs_{{ .version }}: - identifier: guides-mongodb-backup-kubestash-auto-backup - name: Auto-Backup - parent: guides-mongodb-backup-stashv2 - weight: 30 -menu_name: docs_{{ .version }} -section_menu_id: guides ---- - -# Backup MongoDB using KubeStash Auto-Backup - -KubeStash can be configured to automatically backup any MongoDB database in your cluster. KubeStash enables cluster administrators to deploy backup blueprints ahead of time so that the database owners can easily backup their database with just a few annotations. - -In this tutorial, we are going to show how you can configure a backup blueprint for MongoDB databases in your cluster and backup them with few annotations. - -## Before You Begin - -- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. -- Install KubeStash Enterprise in your cluster following the steps [here](https://stash.run/docs/latest/setup/install/stash/). -- Install KubeDB in your cluster following the steps [here](/docs/setup/README.md). -- If you are not familiar with how Stash backup and restore MongoDB databases, please check the following guide [here](/docs/guides/mongodb/backup/stash/overview/index.md). -- If you are not familiar with how auto-backup works in Stash, please check the following guide [here](https://stash.run/docs/latest/guides/auto-backup/overview/). -- If you are not familiar with the available auto-backup options for databases in Stash, please check the following guide [here](https://stash.run/docs/latest/guides/auto-backup/database/). - -You should be familiar with the following `KubeStash` concepts: - -- [BackupBlueprint](https://stash.run/docs/latest/concepts/crds/backupblueprint/) -- [BackupConfiguration](https://stash.run/docs/latest/concepts/crds/backupconfiguration/) -- [BackupSession](https://stash.run/docs/latest/concepts/crds/backupsession/) -- [Repository](https://stash.run/docs/latest/concepts/crds/repository/) -- [Function](https://stash.run/docs/latest/concepts/crds/function/) -- [Task](https://stash.run/docs/latest/concepts/crds/task/) - -To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create `demo` namespace if you haven't created yet. - -```console -$ kubectl create ns demo -namespace/demo created -``` - -## Prepare Backup Blueprint - -`BackupBlueprint` is a non-namespaced Custom Resource that lets you define a common template for taking backup for all the similar applications. Then, you can just apply some annotations in the targeted application to enable backup. KubeStash will automatically resolve the template and create a BackupConfiguration for the targeted application. - -So we can also take backup any MongoDB database of any namespace just by adding few annotations to our MongoDB CRD. Then, KubeStash will automatically create a `BackupConfiguration` according to the template to backup the database. - -We are going to store our backed up data into a S3 bucket. At first, we need to create a secret with S3 credentials then we need to create a `BackupStorage` crd. If you want to use a different backend, please read the respective backend configuration doc from [here](https://stash.run/docs/latest/guides/backends/overview/). - -### Create Storage Secret: - -Let's create a secret called `s3-secret` with access credentials to our desired S3 bucket, - -```console -$ echo -n '' > AWS_ACCESS_KEY_ID -$ echo -n '' > AWS_SECRET_ACCESS_KEY -$ kubectl create secret generic -n demo s3-secret \ - --from-file=./AWS_ACCESS_KEY_ID \ - --from-file=./AWS_SECRET_ACCESS_KEY -secret/s3-secret created -``` - -### Create BackupStorage: - -Now, crete a `BackupStorage` using this secret. Below is the YAML of BackupStorage CRO we are going to create, - -```yaml -apiVersion: storage.kubestash.com/v1alpha1 -kind: BackupStorage -metadata: - name: s3-storage - namespace: demo -spec: - storage: - provider: s3 - s3: - endpoint: us-east-1.linodeobjects.com - bucket: kubestash-testing - region: us-east-1 - prefix: demo - secret: s3-secret - usagePolicy: - allowedNamespaces: - from: All - deletionPolicy: WipeOut -``` - -Let's create the `BackupStorage` we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/auto-backup/examples/backupstorage.yaml -backupstorage.storage.kubestash.com/s3-storage created -``` -We also need to create an secret for encrypt data and retention policy for `BackupBlueprint` to create `BackupConfiguration` - -### Create Encryption Secret: - -EncryptionSecret refers to the Secret containing the encryption key which will be used to encode/decode the backed up data. Let's create a secret called `encry-secret` - -```console -$ kubectl create secret generic encry-secret -n demo \ - --from-literal=RESTIC_PASSWORD='123' -n demo -secret/encry-secret created -``` - -### Create Retention Policy: - -`RetentionPolicy` specifies how the old Snapshots should be cleaned up. This is a namespaced CRD. However, we can refer it from other namespaces as long as it is permitted via `.spec.usagePolicy`. Below is the YAML of the `RetentionPolicy` called `backup-rp` - -```console -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All -``` - -Let's create the RetentionPolicy we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/auto-backup/examples/retentionpolicy.yaml -retentionpolicy.storage.kubestash.com/backup-rp created -``` - -Now we can create `BackupBlueprint`. Below is the YAML of `BackupBlueprint` object that we are going to use in this tutorial, - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupBlueprint -metadata: - name: sample-blueprint - namespace: demo -spec: - usagePolicy: - allowedNamespaces: - from: All - backupConfigurationTemplate: - deletionPolicy: OnDelete - backends: - - name: s3-backend - storageRef: - namespace: ${storageNamespace} - name: ${storageName} - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - schedule: ${scheduleTime} - repositories: - - name: ${repoName} - backend: s3-backend - directory: ${backupPath} - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup -``` - -Here, we define a template for `BackupConfiguration`. Notice the `backends` and `sessions` fields of `backupConfigurationTemplate` section. We have used some variables in form of `${VARIABLE_NAME}`. KubeStash will automatically resolve those variables from the database annotations information to make `BackupConfiguration` according to that databases need. - -Let's create the `BackupBlueprint` we have shown above, - -```bash -❯ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/auto-backup/examples/backupblueprint.yaml -backupblueprint.core.kubestash.com/sample-blueprint created -``` - -Now, we are ready to backup our MongoDB databases using few annotations. You can check available auto-backup annotations for a databases from [here](https://stash.run/docs/latest/guides/auto-backup/database/#available-auto-backup-annotations-for-database). - -### Create Database - -Now, we are going to create an MongoDB CRO in `demo` namespace. Below is the YAML of the MongoDB object that we are going to create, - -```yaml -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: mgo - namespace: demo - annotations: - blueprint.kubestash.com/name: sample-blueprint - blueprint.kubestash.com/namespace: demo - variables.kubestash.com/storageNamespace: demo - variables.kubestash.com/storageName: s3-storage - variables.kubestash.com/scheduleTime: "*/3 * * * *" - variables.kubestash.com/backupPath: /demo/mgo - variables.kubestash.com/repoName: s3-repo -spec: - version: "4.4.26" - storageType: Durable - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut -``` - -Notice the `annotations` section. We are pointing to the `BackupBlueprint` that we have created earlier through `blueprint.kubestash.com` annotation. Also we pass the value of the variable of different template through some `variables.kubestash.com` annotations .KubeStash will watch this annotation and a `BackupConfiguration` according to the `BackupBlueprint`. - -Let's create the above MongoDB CRO, - -```bash -❯ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongob/backup/kubestash/auto-backup/examples/sample-mongodb.yaml -mongodb.kubedb.com/sample-mongodb created -``` - -### Verify Auto-backup configured - -In this section, we are going to verify whether KubeStash has created the respective `BackupConfiguration` for our MongoDB database we have just deployed or not. - -#### Verify BackupConfiguration - -If everything goes well, KubeStash should create a `BackupConfiguration` for our MongoDB in `demo` namespace and the phase of that `BackupConfiguration` should be `Ready`. Verify the `BackupConfiguration` crd by the following command, - -```bash -❯ kubectl get backupconfiguration -n demo -NAME PHASE PAUSED AGE -appbinding-mgo Ready 102s -``` - -Now, let's check the YAML of the `BackupConfiguration`. - -```yaml -❯ kubectl get backupconfiguration -n demo appbinding-mgo -oyaml -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - creationTimestamp: "2023-12-08T11:48:27Z" - finalizers: - - kubestash.com/cleanup - generation: 1 - labels: - app.kubernetes.io/managed-by: kubestash.com - kubestash.com/invoker-name: sample-blueprint - kubestash.com/invoker-namespace: demo - name: appbinding-mgo - namespace: demo - resourceVersion: "323212" - uid: 903a354a-7bdb-4a90-a96a-ccdf465b33fd -spec: - backends: - - name: s3-backend - retentionPolicy: - name: backup-rp - namespace: demo - storageRef: - name: s3-storage - namespace: demo - sessions: - - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup - failurePolicy: Fail - name: frequent - repositories: - - backend: s3-backend - directory: /demo/mgo - encryptionSecret: - name: encry-secret - namespace: demo - name: s3-repo - scheduler: - jobTemplate: - template: - controller: {} - metadata: {} - spec: - resources: {} - schedule: '*/3 * * * *' - sessionHistoryLimit: 1 - target: - apiGroup: kubedb.com - kind: MongoDB - name: mgo - namespace: demo -status: - backends: - - name: s3-backend - ready: true - retentionPolicy: - found: true - ref: - name: backup-rp - namespace: demo - storage: - phase: Ready - ref: - name: s3-storage - namespace: demo - conditions: - - lastTransitionTime: "2023-12-08T11:48:57Z" - message: Validation has been passed successfully. - reason: ResourceValidationPassed - status: "True" - type: ValidationPassed - dependencies: - - found: true - kind: Addon - name: mongodb-addon - phase: Ready - repositories: - - name: s3-repo - phase: Ready - sessions: - - conditions: - - lastTransitionTime: "2023-12-08T11:49:07Z" - message: Scheduler has been ensured successfully. - reason: SchedulerEnsured - status: "True" - type: SchedulerEnsured - name: frequent - targetFound: true -``` - -Here we can see that, - -- `spec.storageRef` template variables are overwrite by the `storageNamespace` and `storageName` variable annotations of the `mgo` MongoDB -- `spec.sessions.scheduler.schedule` template variable is overwrite by the `scheduleTime` variable annotation of the `mgo` MongoDB -- `spec.sessions.repositories.name` template variable is overwrite by the `repoName` variable annotation of the `mgo` MongoDB -- `spec.sessions.repositories.directory` template variable is overwrite by the `backupPath` variable annotation of the `mgo` MongoDB - -#### Verify Backup - -Now, let's wait for a backup run to complete. You can watch for `BackupSession` as below, - -```bash -❯ kubectl get backupsession -n demo -NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE -appbinding-mgo-frequent-1702036262 BackupConfiguration appbinding-mgo Succeeded 3m5s -appbinding-mgo-frequent-1702036442 BackupConfiguration appbinding-mgo Running 5s -``` - -Once the backup has been completed successfully, you should see the backed up data has been stored in the bucket at the directory pointed by the `spec.sessions.repositories.directory` field of the `BackupConfiguration`. - -
- Backup data in S3 Bucket -
Fig: Backup data in S3 Bucket
-
- -## Cleanup - -To cleanup the resources crated by this tutorial, run the following commands, - -```bash -❯ kubectl delete -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/auto-backup/examples/ -backupblueprint.core.kubestash.com "sample-blueprint" deleted -backupstorage.storage.kubestash.com "s3-storage" deleted -retentionpolicy.storage.kubestash.com "backup-rp" deleted -mongodb.kubedb.com "mgo" deleted -``` diff --git a/docs/guides/mongodb/backup/kubestash/auto-backup/examples/backupblueprint.yaml b/docs/guides/mongodb/backup/kubestash/auto-backup/examples/backupblueprint.yaml deleted file mode 100644 index dc02d1841..000000000 --- a/docs/guides/mongodb/backup/kubestash/auto-backup/examples/backupblueprint.yaml +++ /dev/null @@ -1,36 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupBlueprint -metadata: - name: sample-blueprint - namespace: demo -spec: - usagePolicy: - allowedNamespaces: - from: All - backupConfigurationTemplate: - deletionPolicy: OnDelete - backends: - - name: s3-backend - storageRef: - namespace: ${storageNamespace} - name: ${storageName} - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - jobTemplate: - backoffLimit: 1 - schedule: ${scheduleTime} - repositories: - - name: ${repoName} - backend: s3-backend - directory: ${backupPath} - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/auto-backup/examples/backupstorage.yaml b/docs/guides/mongodb/backup/kubestash/auto-backup/examples/backupstorage.yaml deleted file mode 100644 index 4bed4eee3..000000000 --- a/docs/guides/mongodb/backup/kubestash/auto-backup/examples/backupstorage.yaml +++ /dev/null @@ -1,18 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: BackupStorage -metadata: - name: s3-storage - namespace: demo -spec: - storage: - provider: s3 - s3: - endpoint: us-east-1.linodeobjects.com - bucket: kubestash-testing - region: us-east-1 - prefix: demo - secretName: s3-secret - usagePolicy: - allowedNamespaces: - from: All - deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/auto-backup/examples/retentionpolicy.yaml b/docs/guides/mongodb/backup/kubestash/auto-backup/examples/retentionpolicy.yaml deleted file mode 100644 index 3d327f9a7..000000000 --- a/docs/guides/mongodb/backup/kubestash/auto-backup/examples/retentionpolicy.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/auto-backup/examples/sample-mongodb.yaml b/docs/guides/mongodb/backup/kubestash/auto-backup/examples/sample-mongodb.yaml deleted file mode 100644 index 91241db3a..000000000 --- a/docs/guides/mongodb/backup/kubestash/auto-backup/examples/sample-mongodb.yaml +++ /dev/null @@ -1,24 +0,0 @@ -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: mgo - namespace: demo - annotations: - blueprint.kubestash.com/name: sample-blueprint - blueprint.kubestash.com/namespace: demo - variables.kubestash.com/storageNamespace: demo - variables.kubestash.com/storageName: s3-storage - variables.kubestash.com/scheduleTime: "*/3 * * * *" - variables.kubestash.com/backupPath: /demo/mgo - variables.kubestash.com/repoName: s3-repo -spec: - version: "4.4.26" - storageType: Durable - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/auto-backup/images/sample-mongodb.png b/docs/guides/mongodb/backup/kubestash/auto-backup/images/sample-mongodb.png deleted file mode 100644 index c6adefe03..000000000 Binary files a/docs/guides/mongodb/backup/kubestash/auto-backup/images/sample-mongodb.png and /dev/null differ diff --git a/docs/guides/mongodb/backup/kubestash/customization/_index.md b/docs/guides/mongodb/backup/kubestash/customization/_index.md deleted file mode 100644 index 59549ce24..000000000 --- a/docs/guides/mongodb/backup/kubestash/customization/_index.md +++ /dev/null @@ -1,314 +0,0 @@ ---- -title: MongoDB Backup Customization | KubeStash -description: Customizing MongoDB Backup and Restore process with KubeStash -menu: - docs_{{ .version }}: - identifier: guides-mongodb-backup-kubestash-customization - name: Customizing Backup & Restore Process - parent: guides-mongodb-backup-stashv2 - weight: 40 -menu_name: docs_{{ .version }} -section_menu_id: guides ---- - -# Customizing Backup and Restore Process - -KubeStash provides rich customization supports for the backup and restore process to meet the requirements of various cluster configurations. This guide will show you some examples of these customizations. - -## Customizing Backup Process - -In this section, we are going to show you how to customize the backup process. Here, we are going to show some examples of providing arguments to the backup process, running the backup process as a specific user, ignoring some indexes during the backup process, etc. - -### Passing arguments to the backup process - -KubeStash MongoDB addon uses [mongoump](https://docs.mongodb.com/database-tools/mongodump/) for backup. You can pass arguments to the `mongodump` through `args` param under `spec.sessions.addon.task.params` section. - -The below example shows how you can pass the `--db testdb` to take backup for a specific mongodb databases named `testdb`. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mongodb - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - schedule: "*/5 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /mongodb - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup - params: - args: "--db=testdb" -``` - -> **WARNING**: Make sure that you have the specific database created before taking backup. In this case, Database `testdb` should exist before the backup job starts. - -### Running backup job as a specific user - -If your cluster requires running the backup job as a specific user, you can provide `securityContext` under `spec.sessions.addon.tasks.jobTemplate.spec` section. The below example shows how you can run the backup job as the root user. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mongodb - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - schedule: "*/5 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /mongodb - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup - jobTemplate: - spec: - securityContext: - runAsUser: 0 - runAsGroup: 0 -``` - -### Specifying Memory/CPU limit/request for the backup job - -If you want to specify the Memory/CPU limit/request for your backup job, you can specify `resources` field under `spec.sessions.addon.tasks.containerRuntimeSettings` section. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mongodb - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - schedule: "*/3 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /mongodb - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup - containerRuntimeSettings: - resources: - requests: - cpu: "200m" - memory: "1Gi" - limits: - cpu: "200m" - memory: "1Gi" -``` - -## Customizing Restore Process - -KubeStash also uses `mongorestore` during the restore process. In this section, we are going to show how you can pass arguments to the restore process, restore a specific snapshot, run restore job as a specific user, etc. - -### Passing arguments to the restore process - -Similar to the backup process, you can pass arguments to the restore process through the `args` params under `spec.addon.task.params` section. This example will restore data from database `testdb`. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-restore - namespace: demo -spec: - target: - name: restore-mongodb - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestoress - params: - args: "--db=testdb" -``` - -### Restore specific snapshot - -You can also restore a specific snapshot. At first, list the available snapshot as bellow, - -```bash -❯ kubectl get snapshots -n demo -NAMESPACE NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE -demo s3-repo-mg-frequent-1702291682 s3-repo frequent 2023-12-11T10:48:10Z Delete Succeeded 82m -demo s3-repo-mg-frequent-1702291685 s3-repo frequent 2023-12-11T10:49:10Z Delete Succeeded 82m -``` - ->You can also filter the snapshots as shown in the guide [here](https://stash.run/docs/latest/concepts/crds/snapshot/#working-with-snapshot). - -The below example shows how you can pass a specific snapshot id through the `snapshots` filed of `rules` section. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-restore - namespace: demo -spec: - target: - name: restore-mongodb - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: s3-repo-mg-frequent-1702291682 - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestoress -``` - ->Please, do not specify multiple snapshots here. Each snapshot represents a complete backup of your database. Multiple snapshots are only usable during file/directory restore. - -### Running restore job as a specific user - -You can provide `securityContext` under `spec.addon.tasks.jobTemplate.spec` section to run the restore job as a specific user. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-restore - namespace: demo -spec: - target: - name: restore-mongodb - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestoress - jobTemplate: - spec: - securityContext: - runAsUser: 0 - runAsGroup: 0 -``` - -### Specifying Memory/CPU limit/request for the restore job - -Similar to the backup process, you can also provide `resources` field under the `spec.addon.tasks.containerRuntimeSettings` section to limit the Memory/CPU for your restore job. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-restore - namespace: demo -spec: - target: - name: restore-mongodb - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestoress - containerRuntimeSettings: - resources: - requests: - cpu: "200m" - memory: "1Gi" - limits: - cpu: "200m" - memory: "1Gi" -``` - -## Cleanup -To cleanup the resources crated by this tutorial, run the following commands, - -```bash -❯ kubectl delete backupconfiguration -n demo -❯ kubectl delete restoresession -n demo -``` \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/customization/examples/backup/passing-args.yaml b/docs/guides/mongodb/backup/kubestash/customization/examples/backup/passing-args.yaml deleted file mode 100644 index c947f7428..000000000 --- a/docs/guides/mongodb/backup/kubestash/customization/examples/backup/passing-args.yaml +++ /dev/null @@ -1,38 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mongodb - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - schedule: "*/5 * * * *" - jobTemplate: - backoffLimit: 1 - repositories: - - name: s3-repo - backend: s3-backend - directory: /mongodb - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup - params: - args: "--db=testdb" \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/customization/examples/backup/resource-limit.yaml b/docs/guides/mongodb/backup/kubestash/customization/examples/backup/resource-limit.yaml deleted file mode 100644 index 8cdeee995..000000000 --- a/docs/guides/mongodb/backup/kubestash/customization/examples/backup/resource-limit.yaml +++ /dev/null @@ -1,44 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mongodb - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - jobTemplate: - backoffLimit: 1 - schedule: "*/3 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /mongodb - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup - containerRuntimeSettings: - resources: - requests: - cpu: "200m" - memory: "1Gi" - limits: - cpu: "200m" - memory: "1Gi" \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/customization/examples/backup/specific-user.yaml b/docs/guides/mongodb/backup/kubestash/customization/examples/backup/specific-user.yaml deleted file mode 100644 index b2698d459..000000000 --- a/docs/guides/mongodb/backup/kubestash/customization/examples/backup/specific-user.yaml +++ /dev/null @@ -1,41 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mongodb - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - jobTemplate: - backoffLimit: 1 - schedule: "*/5 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /mongodb - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup - jobTemplate: - spec: - securityContext: - runAsUser: 0 - runAsGroup: 0 \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/customization/examples/restore/restore.yaml b/docs/guides/mongodb/backup/kubestash/customization/examples/restore/restore.yaml deleted file mode 100644 index 9fa004803..000000000 --- a/docs/guides/mongodb/backup/kubestash/customization/examples/restore/restore.yaml +++ /dev/null @@ -1,26 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-restore - namespace: demo -spec: - target: - name: restore-mongodb - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestoress - jobTemplate: - spec: - securityContext: - runAsUser: 0 - runAsGroup: 0 \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/customization/examples/retentionpolicy.yaml b/docs/guides/mongodb/backup/kubestash/customization/examples/retentionpolicy.yaml deleted file mode 100644 index 3d327f9a7..000000000 --- a/docs/guides/mongodb/backup/kubestash/customization/examples/retentionpolicy.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/_index.md b/docs/guides/mongodb/backup/kubestash/logical/_index.md deleted file mode 100644 index fb5bfba79..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/_index.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: Logical Backup & Restore of MongoDB | KubeStash -menu: - docs_{{ .version }}: - identifier: guides-mongodb-backup-kubestash-logical - name: Logical Backup - parent: guides-mongodb-backup-stashv2 - weight: 20 -menu_name: docs_{{ .version }} ---- \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/replicaset/_index.md b/docs/guides/mongodb/backup/kubestash/logical/replicaset/_index.md deleted file mode 100644 index 970743731..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/replicaset/_index.md +++ /dev/null @@ -1,593 +0,0 @@ ---- -title: Backup & Restore Replicaset MongoDB | KubeStash -description: Backup and restore replicaset MongoDB database using KubeStash -menu: - docs_{{ .version }}: - identifier: guides-mongodb-backup-kubestash-logical-replicaset - name: MongoDB Replicaset Cluster - parent: guides-mongodb-backup-kubestash-logical - weight: 21 -menu_name: docs_{{ .version }} -section_menu_id: guides ---- - -# Backup and Restore MongoDB database using KubeStash - -KubeStash v0.1.0+ supports backup and restoration of MongoDB databases. This guide will show you how you can backup and restore your MongoDB database with KubeStash. - -## Before You Begin - -- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube. -- Install KubeDB in your cluster following the steps [here](/docs/setup/README.md). -- Install KubeStash Enterprise in your cluster following the steps [here(link needed)](). -- Install KubeStash `kubectl` plugin following the steps [here(link needed)](). -- If you are not familiar with how KubeStash backup and restore MongoDB databases, please check the following guide [here](/docs/guides/mongodb/backup/kubestash/overview/_index.md). - -You have to be familiar with following custom resources: - -- [AppBinding](/docs/guides/mongodb/concepts/appbinding.md) -- [Function](https://stash.run/docs/latest/concepts/crds/function/) -- [Task](https://stash.run/docs/latest/concepts/crds/task/) -- [BackupConfiguration](https://stash.run/docs/latest/concepts/crds/backupconfiguration/) -- [RestoreSession](https://stash.run/docs/latest/concepts/crds/restoresession/) - -To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create `demo` namespace if you haven't created yet. - -```console -$ kubectl create ns demo -namespace/demo created -``` - -## Backup MongoDB - -This section will demonstrate how to backup MongoDB database. Here, we are going to deploy a MongoDB database using KubeDB. Then, we are going to backup this database into a S3 bucket. Finally, we are going to restore the backed up data into another MongoDB database. - -### Deploy Sample MongoDB Database - -Let's deploy a sample MongoDB database and insert some data into it. - -**Create MongoDB CRD:** - -Below is the YAML of a sample MongoDB crd that we are going to create for this tutorial: - -```yaml -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mg-rs - namespace: demo -spec: - version: "4.2.24" - replicas: 3 - replicaSet: - name: rs0 - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut -``` - -Create the above `MongoDB` crd, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/mongodb-replicaset.yaml -mongodb.kubedb.com/sample-mg-rs created -``` - -KubeDB will deploy a MongoDB database according to the above specification. It will also create the necessary secrets and services to access the database. - -Let's check if the database is ready to use, - -```console -$ kubectl get mongodb -n demo sample-mg-rs -NAME VERSION STATUS AGE -sample-mg-rs 4.2.24 Ready 2m27s -``` - -The database is `Ready`. Verify that KubeDB has created a Secret and a Service for this database using the following commands, - -```console -$ kubectl get secret -n demo -l=app.kubernetes.io/instance=sample-mg-rs -NAME TYPE DATA AGE -sample-mg-rs-auth kubernetes.io/basic-auth 2 3m53s -sample-mg-rs-key Opaque 1 3m53s - -$ kubectl get service -n demo -l=app.kubernetes.io/instance=sample-mg-rs -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -sample-mg-rs ClusterIP 10.96.211.27 27017/TCP 4m38s -sample-mg-rs-pods ClusterIP None 27017/TCP 4m38s -``` - -Here, we have to use service `sample-mg-rs` and secret `sample-mg-rs-auth` to connect with the database. - -**Insert Sample Data:** - -> Note: You can insert data into this `MongoDB` database using our [KubeDB CLI](https://kubedb.com/docs/latest/setup/install/kubectl_plugin/). - -For simplicity, we are going to exec into the database pod and create some sample data. At first, find out the database pod using the following command, - -```console -$ kubectl get pods -n demo --selector="app.kubernetes.io/instance=sample-mg-rs" -NAME READY STATUS RESTARTS AGE -sample-mg-rs-0 2/2 Running 0 6m15s -sample-mg-rs-1 2/2 Running 0 5m36s -sample-mg-rs-2 2/2 Running 0 5m14s -``` - -Now, let's exec into the pod and create a table, - -```console -$ export USER=$(kubectl get secrets -n demo sample-mg-rs-auth -o jsonpath='{.data.\username}' | base64 -d) - -$ export PASSWORD=$(kubectl get secrets -n demo sample-mg-rs-auth -o jsonpath='{.data.\password}' | base64 -d) - -$ kubectl exec -it -n demo sample-mg-rs-0 -- mongo admin -u $USER -p $PASSWORD - -rs0:PRIMARY> show dbs -admin 0.000GB -config 0.000GB -kubedb-system 0.000GB -local 0.000GB - -rs0:PRIMARY> show users -{ - "_id" : "admin.root", - "userId" : UUID("12716b1d-9186-4710-a669-4a2050a548bc"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -rs0:PRIMARY> use newdb -switched to db newdb - -rs0:PRIMARY> db.movie.insert({"name":"batman"}); -WriteResult({ "nInserted" : 1 }) - -rs0:PRIMARY> db.movie.find().pretty() -{ "_id" : ObjectId("657182194f1f295758f449a5"), "name" : "batman" } - -rs0:PRIMARY> exit -bye - -``` - -Now, we are ready to backup this sample database. - -### Prepare Backend - -We are going to store our backed up data into a S3 bucket. At first, we need to create a secret with S3 credentials then we need to create a `BackupStorage` crd. If you want to use a different backend, please read the respective backend configuration doc from [here](https://stash.run/docs/latest/guides/backends/overview/). - -**Create Storage Secret:** - -Let's create a secret called `s3-secret` with access credentials to our desired S3 bucket, - -```console -$ echo -n '' > AWS_ACCESS_KEY_ID -$ echo -n '' > AWS_SECRET_ACCESS_KEY -$ kubectl create secret generic -n demo s3-secret \ - --from-file=./AWS_ACCESS_KEY_ID \ - --from-file=./AWS_SECRET_ACCESS_KEY -secret/s3-secret created -``` - -**Create BackupStorage:** - -Now, crete a `BackupStorage` using this secret. Below is the YAML of BackupStorage crd we are going to create, - -```yaml -apiVersion: storage.kubestash.com/v1alpha1 -kind: BackupStorage -metadata: - name: s3-storage-replicaset - namespace: demo -spec: - storage: - provider: s3 - s3: - endpoint: us-east-1.linodeobjects.com - bucket: kubestash-testing - region: us-east-1 - prefix: demo-replicaset - secret: s3-secret - usagePolicy: - allowedNamespaces: - from: All - deletionPolicy: WipeOut -``` - -Let's create the `BackupStorage` we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/backupstorage-replicaset.yaml -backupstorage.storage.kubestash.com/s3-storage-replicaset created -``` - -Now, we are ready to backup our database to our desired backend. - -### Backup - -We have to create a `BackupConfiguration` targeting respective MongoDB crd of our desired database. Then KubeStash will create a CronJob to periodically backup the database. Before that we need to create an secret for encrypt data and retention policy. - -**Create Encryption Secret:** - -EncryptionSecret refers to the Secret containing the encryption key which will be used to encode/decode the backed up data. Let's create a secret called `encry-secret` - -```console -$ kubectl create secret generic encry-secret -n demo \ - --from-literal=RESTIC_PASSWORD='123' -n demo -secret/encry-secret created -``` - -**Create Retention Policy:** - -`RetentionPolicy` specifies how the old Snapshots should be cleaned up. This is a namespaced CRD.However, we can refer it from other namespaces as long as it is permitted via `.spec.usagePolicy`. Below is the YAML of the `RetentionPolicy` called `backup-rp` - -```console -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All -``` - -Let's create the RetentionPolicy we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/retentionpolicy.yaml -retentionpolicy.storage.kubestash.com/backup-rp created -``` - -**Create BackupConfiguration:** - -As we just create our encryption secret and retention policy, we are now ready to apply `BackupConfiguration` crd to take backup out database. - -Below is the YAML for `BackupConfiguration` crd to backup the `sample-mg-rs` database we have deployed earlier., - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mg-rs - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage-replicaset - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - schedule: "*/3 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /replicaset - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup -``` - -Here, - -- `spec.target` specifies our targeted `MongoDB` database. -- `spec.backends` specifies `BackupStorage` information for storing data. -- `spec.sessions` specifies common session configurations for this backup -- `spec.sessions.schedule` specifies that we want to backup the database at 5 minutes interval. -- `spec.sessions.addon` refers to the `Addon` crd for backup task - -Let's create the `BackupConfiguration` crd we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/backupconfiguration-replicaset.yaml -backupconfiguration.core.kubestash.com/mg created -``` - -**Verify Backup Setup Successful:** - -If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, - -```console -$ kubectl get backupconfiguration -n demo -NAME PHASE PAUSED AGE -mg Ready 85s -``` - -**Verify CronJob:** - -KubeStash will create a CronJob with the schedule specified in `spec.sessions.schedule` field of `BackupConfiguration` crd. - -Verify that the CronJob has been created using the following command, - -```console -$ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -trigger-mg-frequent */3 * * * * False 0 101s -``` - -**Wait for BackupSession:** - -The `trigger-mg-frequent` CronJob will trigger a backup on each schedule by creating a `BackpSession` crd. - -Wait for the next schedule. Run the following command to watch `BackupSession` crd, - -```console -$ kubectl get backupsession -n demo -NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE -mg-frequent-1701940862 BackupConfiguration mg Succeeded 3m16s -mg-frequent-1701941042 BackupConfiguration mg Running 16s -``` - -We can see above that the backup session has succeeded. Now, we are going to verify that the backed up data has been stored in the backend. - -**Verify Backup:** - -Once a backup is complete, KubeStash will update the respective `Snapshot` crd to reflect the backup. It will be created when a backup is triggered. Check that the `Snapshot` Phase to verify backup. - -```console -$ kubectl get snapshot -n demo -NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE -s3-repo-mg-frequent-1701940862 s3-repo frequent 2023-12-07T09:21:07Z Delete Succeeded 3m53s -s3-repo-mg-frequent-1701941042 s3-repo frequent 2023-12-07T09:24:08Z Delete Succeeded 53s -``` - -KubeStash will also update the respective `Repository` crd to reflect the backup. Check that the repository `s3-repo` has been updated by the following command, - -```console -$ kubectl get repository -n demo s3-repo -NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE -s3-repo true 2 2.883 KiB Ready 55s 8m5s -``` - -Now, if we navigate to the S3 bucket, we are going to see backed up data has been stored in `demo/replicaset/` directory as specified by `spec.sessions.repositories.directory` field of `BackupConfiguration` crd. - -> Note: KubeStash keeps all the backed up data encrypted. So, data in the backend will not make any sense until they are decrypted. - -## Restore MongoDB -You can restore your data into the same database you have backed up from or into a different database in the same cluster or a different cluster. In this section, we are going to show you how to restore in the same database which may be necessary when you have accidentally deleted any data from the running database. - -#### Stop Taking Backup of the Old Database: - -It's important to stop taking any further backup of the old database so that no backup is stored in our repository during restore process. KubeStash operator will automatically pause the `BackupConfiguration` when a `RestoreSession` is running. However if we want to pause the `BackupConfiguration` manually, we can do that by patching or using KubeStash CLI. - -Let's pause the `mg` BackupConfiguration by patching, -```bash -$ kubectl patch backupconfiguration -n demo mg --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.core.kubestash.com/mg patched -``` - -Now, wait for a moment. KubeStash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, - -```console -$ kubectl get backupconfiguration -n demo mg -NAME PHASE PAUSED AGE -mg Ready true 11m -``` - -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. - -#### Deploy New MongoDB Database For Restoring: - -We are going to deploy a new mongodb replicaset database for restoring backed up data. - -Below is the YAML of a sample `MongoDB` crd that we are going to create - -```yaml -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mg-rs-restore - namespace: demo -spec: - version: "4.2.24" - replicas: 3 - replicaSet: - name: rs0 - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut -``` - -Create the above `MongoDB` crd, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/mongodb-replicaset-restore.yaml -mongodb.kubedb.com/sample-mg-rs-restore created -``` - -Let's check if the database is ready to use, - -```console -$ kubectl get mg -n demo sample-mg-rs-restore -NAME VERSION STATUS AGE -sample-mg-rs-restore 4.2.24 Ready 2m45s -``` - -Let's verify all the databases of this `sample-mg-rs-restore` by exec into its pod - -```console -$ export USER=$(kubectl get secrets -n demo sample-mg-rs-restore-auth -o jsonpath='{.data.\username}' | base64 -d) - -$ export PASSWORD=$(kubectl get secrets -n demo sample-mg-rs-restore-auth -o jsonpath='{.data.\password}' | base64 -d) - -$ kubectl exec -it -n demo sample-mg-rs-restore-0 -- mongo admin -u $USER -p $PASSWORD - -rs0:PRIMARY> show dbs -admin 0.000GB -config 0.000GB -kubedb-system 0.000GB -local 0.000GB - -rs0:PRIMARY> show users -{ - "_id" : "admin.root", - "userId" : UUID("b8a10e3f-05a1-421c-87ca-7bdf16f99563"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -rs0:PRIMARY> exit -bye -``` - -As we can see no database named `newdb` exist in this new `sample-mg-rs-restore` database. - -#### Create RestoreSession: - -Now, we need to create a `RestoreSession` crd pointing to the `sample-mg-rs-restore` database. -Below is the YAML for the `RestoreSession` crd that we are going to create to restore the backed up data. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-rs-restore - namespace: demo -spec: - target: - name: sample-mg-rs-restore - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestore -``` - -Here, - -- `spec.dataSource.repository` specifies the `Repository` crd that holds the backend information where our backed up data has been stored. -- `spec.target` refers to the `MongoDB` crd for the `sample-mg-rs-restore` database. -- `spec.dataSource.snapshot` specifies that we are restoring from the latest backup snapshot of the `spec.dataSource.repository`. - -Let's create the `RestoreSession` crd we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/restoresession-replicaset.yaml -restoresession.core.kubestash.com/mg-rs-restore created -``` - -Once, you have created the `RestoreSession` crd, KubeStash will create a job to restore. We can watch the `RestoreSession` phase to check if the restore process is succeeded or not. - -Run the following command to watch `RestoreSession` phase, - -```console -$ kubectl get restoresession -n demo mg-rs-restore -w -NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE -mg-rs-restore s3-repo Succeeded 9s 34s -``` - -So, we can see from the output of the above command that the restore process succeeded. - -#### Verify Restored Data: - -In this section, we are going to verify that the desired data has been restored successfully. We are going to connect to the `sample-mg-rs-restore` database and check whether the table we had created earlier is restored or not. - -Lets, exec into the database pod and list available tables, - -```console -$ kubectl exec -it -n demo sample-mg-rs-restore-0 -- mongo admin -u $USER -p $PASSWORD - -rs0:PRIMARY> show dbs -admin 0.000GB -config 0.000GB -kubedb-system 0.000GB -local 0.000GB -newdb 0.000GB - -rs0:PRIMARY> show users -{ - "_id" : "admin.root", - "userId" : UUID("b8a10e3f-05a1-421c-87ca-7bdf16f99563"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -rs0:PRIMARY> use newdb -switched to db newdb - -rs0:PRIMARY> db.movie.find().pretty() -{ "_id" : ObjectId("657182194f1f295758f449a5"), "name" : "batman" } - -rs0:PRIMARY> exit -bye -``` - -So, from the above output, we can see the database `newdb` that we had created earlier is restored into another new `MongoDB` database. - -## Cleanup - -To cleanup the Kubernetes resources created by this tutorial, run: - -```console -kubectl delete -n demo restoresession mg-rs-restore -kubectl delete -n demo backupconfiguration mg -kubectl delete -n demo mg sample-mg-rs -kubectl delete -n demo mg sample-mg-rs-restore -kubectl delete -n demo backupstorage s3-storage-replicaset -``` diff --git a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/backupconfiguration-replicaset.yaml b/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/backupconfiguration-replicaset.yaml deleted file mode 100644 index aa9ada7a1..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/backupconfiguration-replicaset.yaml +++ /dev/null @@ -1,36 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mg-rs - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage-replicaset - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - jobTemplate: - backoffLimit: 1 - schedule: "*/3 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /replicaset - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/backupstorage-replicaset.yaml b/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/backupstorage-replicaset.yaml deleted file mode 100644 index 81614714b..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/backupstorage-replicaset.yaml +++ /dev/null @@ -1,18 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: BackupStorage -metadata: - name: s3-storage-replicaset - namespace: demo -spec: - storage: - provider: s3 - s3: - endpoint: us-east-1.linodeobjects.com - bucket: kubestash-testing - region: us-east-1 - prefix: demo-replicaset - secret: s3-secret - usagePolicy: - allowedNamespaces: - from: All - deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/mongodb-replicaset-restore.yaml b/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/mongodb-replicaset-restore.yaml deleted file mode 100644 index 68555ef7c..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/mongodb-replicaset-restore.yaml +++ /dev/null @@ -1,18 +0,0 @@ -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mg-rs-restore - namespace: demo -spec: - version: "4.2.24" - replicas: 3 - replicaSet: - name: rs0 - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/mongodb-replicaset.yaml b/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/mongodb-replicaset.yaml deleted file mode 100644 index aaf24a6cb..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/mongodb-replicaset.yaml +++ /dev/null @@ -1,18 +0,0 @@ -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mg-rs - namespace: demo -spec: - version: "4.2.24" - replicas: 3 - replicaSet: - name: rs0 - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/restoresession-replicaset.yaml b/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/restoresession-replicaset.yaml deleted file mode 100644 index 56920b806..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/restoresession-replicaset.yaml +++ /dev/null @@ -1,21 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-rs-restore - namespace: demo -spec: - target: - name: sample-mg-rs-restore - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestore \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/retentionpolicy.yaml b/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/retentionpolicy.yaml deleted file mode 100644 index 3d327f9a7..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/replicaset/examples/retentionpolicy.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/sharding/_index.md b/docs/guides/mongodb/backup/kubestash/logical/sharding/_index.md deleted file mode 100644 index 69435b449..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/sharding/_index.md +++ /dev/null @@ -1,607 +0,0 @@ ---- -title: Backup & Restore Sharded MongoDB | KubeStash -description: Backup and restore sharded MongoDB database using KubeStash -menu: - docs_{{ .version }}: - identifier: guides-mongodb-backup-kubestash-logical-sharded - name: MongoDB Sharded Cluster - parent: guides-mongodb-backup-kubestash-logical - weight: 22 -menu_name: docs_{{ .version }} -section_menu_id: guides ---- - -# Backup and Restore MongoDB database using KubeStash - -KubeStash v0.1.0+ supports backup and restoration of MongoDB databases. This guide will show you how you can backup and restore your MongoDB database with KubeStash. - -## Before You Begin - -- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube. -- Install KubeDB in your cluster following the steps [here](/docs/setup/README.md). -- Install KubeStash Enterprise in your cluster following the steps [here(link needed)](). -- Install KubeStash `kubectl` plugin following the steps [here(link needed)](). -- If you are not familiar with how KubeStash backup and restore MongoDB databases, please check the following guide [here](/docs/guides/mongodb/backup/kubestash/overview/_index.md). - -You have to be familiar with following custom resources: - -- [AppBinding](/docs/guides/mongodb/concepts/appbinding.md) -- [Function](https://stash.run/docs/latest/concepts/crds/function/) -- [Task](https://stash.run/docs/latest/concepts/crds/task/) -- [BackupConfiguration](https://stash.run/docs/latest/concepts/crds/backupconfiguration/) -- [RestoreSession](https://stash.run/docs/latest/concepts/crds/restoresession/) - -To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create `demo` namespace if you haven't created yet. - -```console -$ kubectl create ns demo -namespace/demo created -``` - -## Backup MongoDB - -This section will demonstrate how to backup MongoDB database. Here, we are going to deploy a MongoDB database using KubeDB. Then, we are going to backup this database into a S3 bucket. Finally, we are going to restore the backed up data into another MongoDB database. - -### Deploy Sample MongoDB Database - -Let's deploy a sample MongoDB database and insert some data into it. - -**Create MongoDB CRD:** - -Below is the YAML of a sample MongoDB crd that we are going to create for this tutorial: - -```yaml -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mg-sh - namespace: demo -spec: - version: 4.2.24 - shardTopology: - configServer: - replicas: 3 - storage: - resources: - requests: - storage: 1Gi - storageClassName: standard - mongos: - replicas: 2 - shard: - replicas: 3 - shards: 3 - storage: - resources: - requests: - storage: 1Gi - storageClassName: standard - terminationPolicy: WipeOut - -``` - -Create the above `MongoDB` crd, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/mongodb-sharding.yaml -mongodb.kubedb.com/sample-mg-sh created -``` - -KubeDB will deploy a MongoDB database according to the above specification. It will also create the necessary secrets and services to access the database. - -Let's check if the database is ready to use, - -```console -$ kubectl get mongodb -n demo sample-mg-sh -NAME VERSION STATUS AGE -sample-mg-sh 4.2.24 Ready 5m39s -``` - -The database is `Ready`. Verify that KubeDB has created a Secret and a Service for this database using the following commands, - -```console -$ kubectl get secret -n demo -l=app.kubernetes.io/instance=sample-mg-sh -NAME TYPE DATA AGE -sample-mg-sh-auth kubernetes.io/basic-auth 2 21m -sample-mg-sh-key Opaque 1 21m - -$ kubectl get service -n demo -l=app.kubernetes.io/instance=sample-mg-sh -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -sample-mg-sh ClusterIP 10.96.80.43 27017/TCP 21m -sample-mg-sh-configsvr-pods ClusterIP None 27017/TCP 21m -sample-mg-sh-mongos-pods ClusterIP None 27017/TCP 21m -sample-mg-sh-shard0-pods ClusterIP None 27017/TCP 21m -sample-mg-sh-shard1-pods ClusterIP None 27017/TCP 21m -sample-mg-sh-shard2-pods ClusterIP None 27017/TCP 21m -``` - -Here, we have to use service `sample-mg-sh` and secret `sample-mg-sh-auth` to connect with the database. - -**Insert Sample Data:** - -> Note: You can insert data into this `MongoDB` database using our [KubeDB CLI](https://kubedb.com/docs/latest/setup/install/kubectl_plugin/). - -For simplicity, we are going to exec into the database pod and create some sample data. At first, find out the database mongos pod using the following command, - -```console -$ kubectl get pods -n demo --selector="mongodb.kubedb.com/node.mongos=sample-mg-sh-mongos" -NAME READY STATUS RESTARTS AGE -sample-mg-sh-mongos-0 1/1 Running 0 21m -sample-mg-sh-mongos-1 1/1 Running 0 21m -``` - -Now, let's exec into the pod and create a table, - -```console -$ export USER=$(kubectl get secrets -n demo sample-mg-sh-auth -o jsonpath='{.data.\username}' | base64 -d) - -$ export PASSWORD=$(kubectl get secrets -n demo sample-mg-sh-auth -o jsonpath='{.data.\password}' | base64 -d) - -$ kubectl exec -it -n demo sample-mg-sh-mongos-0 -- mongo admin -u $USER -p $PASSWORD - -mongos> show dbs -admin 0.000GB -config 0.002GB -kubedb-system 0.000GB - -mongos> show users -{ - "_id" : "admin.root", - "userId" : UUID("61a02236-1e25-4f58-9b92-9a41a80726bc"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -mongos> use newdb -switched to db newdb - -mongos> db.movie.insert({"name":"batman"}); -WriteResult({ "nInserted" : 1 }) - -mongos> exit -bye - -``` - -Now, we are ready to backup this sample database. - -### Prepare Backend - -We are going to store our backed up data into a S3 bucket. At first, we need to create a secret with S3 credentials then we need to create a `BackupStorage` crd. If you want to use a different backend, please read the respective backend configuration doc from [here](https://stash.run/docs/latest/guides/backends/overview/). - -**Create Storage Secret:** - -Let's create a secret called `s3-secret` with access credentials to our desired S3 bucket, - -```console -$ echo -n '' > AWS_ACCESS_KEY_ID -$ echo -n '' > AWS_SECRET_ACCESS_KEY -$ kubectl create secret generic -n demo s3-secret \ - --from-file=./AWS_ACCESS_KEY_ID \ - --from-file=./AWS_SECRET_ACCESS_KEY -secret/s3-secret created -``` - -**Create BackupStorage:** - -Now, crete a `BackupStorage` using this secret. Below is the YAML of BackupStorage crd we are going to create, - -```yaml -apiVersion: storage.kubestash.com/v1alpha1 -kind: BackupStorage -metadata: - name: s3-storage-sharding - namespace: demo -spec: - storage: - provider: s3 - s3: - endpoint: us-east-1.linodeobjects.com - bucket: kubestash-testing - region: us-east-1 - prefix: demo-sharding - secret: s3-secret - usagePolicy: - allowedNamespaces: - from: All - deletionPolicy: WipeOut -``` - -Let's create the `BackupStorage` we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/backupstorage-sharding.yaml -backupstorage.storage.kubestash.com/s3-storage-sharding created -``` - -Now, we are ready to backup our database to our desired backend. - -### Backup - -We have to create a `BackupConfiguration` targeting respective MongoDB crd of our desired database. Then KubeStash will create a CronJob to periodically backup the database. Before that we need to create an secret for encrypt data and retention policy. - -**Create Encryption Secret:** - -EncryptionSecret refers to the Secret containing the encryption key which will be used to encode/decode the backed up data. Let's create a secret called `encry-secret` - -```console -$ kubectl create secret generic encry-secret -n demo \ - --from-literal=RESTIC_PASSWORD='123' -n demo -secret/encry-secret created -``` - -**Create Retention Policy:** - -`RetentionPolicy` specifies how the old Snapshots should be cleaned up. This is a namespaced CRD.However, we can refer it from other namespaces as long as it is permitted via `.spec.usagePolicy`. Below is the YAML of the `RetentionPolicy` called `backup-rp` - -```console -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All -``` - -Let's create the RetentionPolicy we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/retentionpolicy.yaml -retentionpolicy.storage.kubestash.com/backup-rp created -``` - -**Create BackupConfiguration:** - -As we just create our encryption secret and retention policy, we are now ready to apply `BackupConfiguration` crd to take backup out database. - -Below is the YAML for `BackupConfiguration` crd to backup the `sample-mg-sh` database we have deployed earlier., - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mg-sh - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage-sharding - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - schedule: "*/3 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /sharding - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup -``` - -Here, - -- `spec.target` specifies our targeted `MongoDB` database. -- `spec.backends` specifies `BackupStorage` information for storing data. -- `spec.sessions` specifies common session configurations for this backup -- `spec.sessions.schedule` specifies that we want to backup the database at 5 minutes interval. -- `spec.sessions.addon` refers to the `Addon` crd for backup task - -Let's create the `BackupConfiguration` crd we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/backupconfiguration-sharding.yaml -backupconfiguration.core.kubestash.com/mg created -``` - -**Verify Backup Setup Successful:** - -If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, - -```console -$ kubectl get backupconfiguration -n demo -NAME PHASE PAUSED AGE -mg Ready 85s -``` - -**Verify CronJob:** - -KubeStash will create a CronJob with the schedule specified in `spec.sessions.schedule` field of `BackupConfiguration` crd. - -Verify that the CronJob has been created using the following command, - -```console -$ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -trigger-mg-frequent */3 * * * * False 0 101s -``` - -**Wait for BackupSession:** - -The `trigger-mg-frequent` CronJob will trigger a backup on each schedule by creating a `BackpSession` crd. - -Wait for the next schedule. Run the following command to watch `BackupSession` crd, - -```console -$ kubectl get backupsession -n demo -NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE -mg-frequent-1701950402 BackupConfiguration mg Succeeded 3m5s -mg-frequent-1701950582 BackupConfiguration mg Running 5s -``` - -We can see above that the backup session has succeeded. Now, we are going to verify that the backed up data has been stored in the backend. - -**Verify Backup:** - -Once a backup is complete, KubeStash will update the respective `Snapshot` crd to reflect the backup. It will be created when a backup is triggered. Check that the `Snapshot` Phase to verify backup. - -```console -$ kubectl get snapshot -n demo -NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE -s3-repo-mg-frequent-1701950402 s3-repo frequent 2023-12-07T12:00:11Z Delete Succeeded 3m37s -s3-repo-mg-frequent-1701950582 s3-repo frequent 2023-12-07T12:03:08Z Delete Succeeded 37s -``` - -KubeStash will also update the respective `Repository` crd to reflect the backup. Check that the repository `s3-repo` has been updated by the following command, - -```console -$ kubectl get repository -n demo s3-repo -NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE -s3-repo true 2 95.660 KiB Ready 41s 4m3s -``` - -Now, if we navigate to the S3 bucket, we are going to see backed up data has been stored in `demo/sharding/` directory as specified by `spec.sessions.repositories.directory` field of `BackupConfiguration` crd. - -> Note: KubeStash keeps all the backed up data encrypted. So, data in the backend will not make any sense until they are decrypted. - -## Restore MongoDB -You can restore your data into the same database you have backed up from or into a different database in the same cluster or a different cluster. In this section, we are going to show you how to restore in the same database which may be necessary when you have accidentally deleted any data from the running database. - -#### Stop Taking Backup of the Old Database: - -It's important to stop taking any further backup of the old database so that no backup is stored in our repository during restore process. KubeStash operator will automatically pause the `BackupConfiguration` when a `RestoreSession` is running. However if we want to pause the `BackupConfiguration` manually, we can do that by patching or using KubeStash CLI. - -Let's pause the `mg` BackupConfiguration by patching, -```bash -$ kubectl patch backupconfiguration -n demo mg --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.core.kubestash.com/mg patched -``` - -Now, wait for a moment. KubeStash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, - -```console -$ kubectl get backupconfiguration -n demo mg -NAME PHASE PAUSED AGE -mg Ready true 11m -``` - -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. - -#### Deploy New MongoDB Database For Restoring: - -We are going to deploy a new mongodb replicaset database for restoring backed up data. - -Below is the YAML of a sample `MongoDB` crd that we are going to create - -```yaml -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mg-sh-restore - namespace: demo -spec: - version: 4.2.24 - shardTopology: - configServer: - replicas: 3 - storage: - resources: - requests: - storage: 1Gi - storageClassName: standard - mongos: - replicas: 2 - shard: - replicas: 3 - shards: 3 - storage: - resources: - requests: - storage: 1Gi - storageClassName: standard - terminationPolicy: WipeOut -``` - -Create the above `MongoDB` crd, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/mongodb-sharding-restore.yaml -mongodb.kubedb.com/sample-mg-sh-restore created -``` - -Let's check if the database is ready to use, - -```console -$ kubectl get mg -n demo sample-mg-sh-restore -NAME VERSION STATUS AGE -sample-mg-sh-restore 4.2.24 Ready 7m47s -``` - -Let's verify all the databases of this `sample-mg-sh-restore` by exec into its mongos pod - -```console -$ export USER=$(kubectl get secrets -n demo sample-mg-sh-restore-auth -o jsonpath='{.data.\username}' | base64 -d) - -$ export PASSWORD=$(kubectl get secrets -n demo sample-mg-sh-restore-auth -o jsonpath='{.data.\password}' | base64 -d) - -$ kubectl exec -it -n demo sample-mg-sh-restore-mongos-0 -- mongo admin -u $USER -p $PASSWORD - -mongos> show dbs -admin 0.000GB -config 0.002GB -kubedb-system 0.000GB - -mongos> show users -{ - "_id" : "admin.root", - "userId" : UUID("4400d0cc-bba7-4626-bf5b-7521fca30ff9"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -mongos> exit -bye -``` - -As we can see no database named `newdb` exist in this new `sample-mg-sh-restore` database. - -#### Create RestoreSession: - -Now, we need to create a `RestoreSession` crd pointing to the `sample-mg-sh-restore` database. -Below is the YAML for the `RestoreSession` crd that we are going to create to restore the backed up data. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-sh-restore - namespace: demo -spec: - target: - name: sample-mg-sh-restore - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestore -``` - -Here, - -- `spec.dataSource.repository` specifies the `Repository` crd that holds the backend information where our backed up data has been stored. -- `spec.target` refers to the `MongoDB` crd for the `sample-mg-sh-restore` database. -- `spec.dataSource.snapshot` specifies that we are restoring from the latest backup snapshot of the `spec.dataSource.repository`. - -Let's create the `RestoreSession` crd we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/restoresession-sharding.yaml -restoresession.core.kubestash.com/mg-sh-restore created -``` - -Once, you have created the `RestoreSession` crd, KubeStash will create a job to restore. We can watch the `RestoreSession` phase to check if the restore process is succeeded or not. - -Run the following command to watch `RestoreSession` phase, - -```console -$ kubectl get restoresession -n demo mg-sh-restore -w -NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE -mg-sh-restore s3-repo Succeeded 15s 48s -``` - -So, we can see from the output of the above command that the restore process succeeded. - -#### Verify Restored Data: - -In this section, we are going to verify that the desired data has been restored successfully. We are going to connect to the `sample-mg-sh-restore` database and check whether the table we had created earlier is restored or not. - -Lets, exec into the database's mongos pod and list available tables, - -```console -$ kubectl exec -it -n demo sample-mg-sh-restore-mongos-0 -- mongo admin -u $USER -p $PASSWORD - -mongos> show dbs -admin 0.000GB -config 0.002GB -kubedb-system 0.000GB -newdb 0.000GB - -mongos> show users -{ - "_id" : "admin.root", - "userId" : UUID("4400d0cc-bba7-4626-bf5b-7521fca30ff9"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -mongos> use newdb -switched to db newdb - -mongos> db.movie.find().pretty() -{ "_id" : ObjectId("6571b0465bde128e0c4d7caa"), "name" : "batman" } - -mongos> exit -bye -``` - -So, from the above output, we can see the database `newdb` that we had created earlier is restored into another new `MongoDB` database. - -## Cleanup - -To cleanup the Kubernetes resources created by this tutorial, run: - -```console -kubectl delete -n demo restoresession mg-sh-restore -kubectl delete -n demo backupconfiguration mg -kubectl delete -n demo mg sample-mg-sh -kubectl delete -n demo mg sample-mg-sh-restore -kubectl delete -n demo backupstorage s3-storage-sharding -``` diff --git a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/backupconfiguration-sharding.yaml b/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/backupconfiguration-sharding.yaml deleted file mode 100644 index 27389a4bc..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/backupconfiguration-sharding.yaml +++ /dev/null @@ -1,36 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mg-sh - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage-sharding - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - jobTemplate: - backoffLimit: 1 - schedule: "*/3 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /sharding - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/backupstorage-sharding.yaml b/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/backupstorage-sharding.yaml deleted file mode 100644 index b88ea760a..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/backupstorage-sharding.yaml +++ /dev/null @@ -1,18 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: BackupStorage -metadata: - name: s3-storage-sharding - namespace: demo -spec: - storage: - provider: s3 - s3: - endpoint: us-east-1.linodeobjects.com - bucket: kubestash-testing - region: us-east-1 - prefix: demo-sharding - secretName: s3-secret - usagePolicy: - allowedNamespaces: - from: All - deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/mongodb-sharding-restore.yaml b/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/mongodb-sharding-restore.yaml deleted file mode 100644 index e38e173c4..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/mongodb-sharding-restore.yaml +++ /dev/null @@ -1,26 +0,0 @@ -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mg-sh-restore - namespace: demo -spec: - version: 4.2.24 - shardTopology: - configServer: - replicas: 3 - storage: - resources: - requests: - storage: 1Gi - storageClassName: standard - mongos: - replicas: 2 - shard: - replicas: 3 - shards: 3 - storage: - resources: - requests: - storage: 1Gi - storageClassName: standard - terminationPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/mongodb-sharding.yaml b/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/mongodb-sharding.yaml deleted file mode 100644 index f28723435..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/mongodb-sharding.yaml +++ /dev/null @@ -1,26 +0,0 @@ -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mg-sh - namespace: demo -spec: - version: 4.2.24 - shardTopology: - configServer: - replicas: 3 - storage: - resources: - requests: - storage: 1Gi - storageClassName: standard - mongos: - replicas: 2 - shard: - replicas: 3 - shards: 3 - storage: - resources: - requests: - storage: 1Gi - storageClassName: standard - terminationPolicy: WipeOut diff --git a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/restoresession-sharding.yaml b/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/restoresession-sharding.yaml deleted file mode 100644 index d0d8614d7..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/restoresession-sharding.yaml +++ /dev/null @@ -1,21 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-sh-restore - namespace: demo -spec: - target: - name: sample-mg-sh-restore - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestore \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/retentionpolicy.yaml b/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/retentionpolicy.yaml deleted file mode 100644 index 3d327f9a7..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/sharding/examples/retentionpolicy.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/standalone/_index.md b/docs/guides/mongodb/backup/kubestash/logical/standalone/_index.md deleted file mode 100644 index d364622dd..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/standalone/_index.md +++ /dev/null @@ -1,586 +0,0 @@ ---- -title: Backup & Restore Standalone MongoDB | KubeStash -description: Backup and restore standalone MongoDB database using KubeStash -menu: - docs_{{ .version }}: - identifier: guides-mongodb-backup-kubestash-logical-standalone - name: Standalone MongoDB - parent: guides-mongodb-backup-kubestash-logical - weight: 20 -menu_name: docs_{{ .version }} -section_menu_id: guides ---- - -# Backup and Restore MongoDB database using KubeStash - -KubeStash v0.1.0+ supports backup and restoration of MongoDB databases. This guide will show you how you can backup and restore your MongoDB database with KubeStash. - -## Before You Begin - -- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube. -- Install KubeDB in your cluster following the steps [here](/docs/setup/README.md). -- Install KubeStash Enterprise in your cluster following the steps [here(link needed)](). -- Install KubeStash `kubectl` plugin following the steps [here(link needed)](). -- If you are not familiar with how KubeStash backup and restore MongoDB databases, please check the following guide [here](/docs/guides/mongodb/backup/kubestash/overview/_index.md). - -You have to be familiar with following custom resources: - -- [AppBinding](/docs/guides/mongodb/concepts/appbinding.md) -- [Function](https://stash.run/docs/latest/concepts/crds/function/) -- [Task](https://stash.run/docs/latest/concepts/crds/task/) -- [BackupConfiguration](https://stash.run/docs/latest/concepts/crds/backupconfiguration/) -- [RestoreSession](https://stash.run/docs/latest/concepts/crds/restoresession/) - -To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create `demo` namespace if you haven't created yet. - -```console -$ kubectl create ns demo -namespace/demo created -``` - -## Backup MongoDB - -This section will demonstrate how to backup MongoDB database. Here, we are going to deploy a MongoDB database using KubeDB. Then, we are going to backup this database into a S3 bucket. Finally, we are going to restore the backed up data into another MongoDB database. - -### Deploy Sample MongoDB Database - -Let's deploy a sample MongoDB database and insert some data into it. - -**Create MongoDB CRD:** - -Below is the YAML of a sample MongoDB crd that we are going to create for this tutorial: - -```yaml -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mongodb - namespace: demo -spec: - version: "4.2.24" - storageType: Durable - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut -``` - -Create the above `MongoDB` crd, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/mongodb.yaml -mongodb.kubedb.com/sample-mongodb created -``` - -KubeDB will deploy a MongoDB database according to the above specification. It will also create the necessary secrets and services to access the database. - -Let's check if the database is ready to use, - -```console -$ kubectl get mg -n demo sample-mongodb -NAME VERSION STATUS AGE -sample-mongodb 4.2.24 Ready 2m9s -``` - -The database is `Ready`. Verify that KubeDB has created a Secret and a Service for this database using the following commands, - -```console -$ kubectl get secret -n demo -l=app.kubernetes.io/instance=sample-mongodb -NAME TYPE DATA AGE -sample-mongodb-auth Opaque 2 2m28s - -$ kubectl get service -n demo -l=app.kubernetes.io/instance=sample-mongodb -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -sample-mongodb ClusterIP 10.107.58.222 27017/TCP 2m48s -sample-mongodb-gvr ClusterIP None 27017/TCP 2m48s -``` - -Here, we have to use service `sample-mongodb` and secret `sample-mongodb-auth` to connect with the database. - -**Insert Sample Data:** - -> Note: You can insert data into this `MongoDB` database using our [KubeDB CLI](https://kubedb.com/docs/latest/setup/install/kubectl_plugin/). - -For simplicity, we are going to exec into the database pod and create some sample data. At first, find out the database pod using the following command, - -```console -$ kubectl get pods -n demo --selector="app.kubernetes.io/instance=sample-mongodb" -NAME READY STATUS RESTARTS AGE -sample-mongodb-0 1/1 Running 0 12m -``` - -Now, let's exec into the pod and create a table, - -```console -$ export USER=$(kubectl get secrets -n demo sample-mongodb-auth -o jsonpath='{.data.\username}' | base64 -d) - -$ export PASSWORD=$(kubectl get secrets -n demo sample-mongodb-auth -o jsonpath='{.data.\password}' | base64 -d) - -$ kubectl exec -it -n demo sample-mongodb-0 -- mongo admin -u $USER -p $PASSWORD - -> show dbs -admin 0.000GB -config 0.000GB -kubedb-system 0.000GB -local 0.000GB - -> show users -{ - "_id" : "admin.root", - "userId" : UUID("b82f8a34-1fc4-4ffe-b616-c6ffa278ecc8"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -> use newdb -switched to db newdb - -> db.movie.insert({"name":"batman"}); -WriteResult({ "nInserted" : 1 }) - -> db.movie.find().pretty() -{ "_id" : ObjectId("5d19d1cdc93d828f44e37735"), "name" : "batman" } - -> exit -bye -``` - -Now, we are ready to backup this sample database. - -### Prepare Backend - -We are going to store our backed up data into a S3 bucket. At first, we need to create a secret with S3 credentials then we need to create a `BackupStorage` crd. If you want to use a different backend, please read the respective backend configuration doc from [here](https://stash.run/docs/latest/guides/backends/overview/). - -**Create Storage Secret:** - -Let's create a secret called `s3-secret` with access credentials to our desired S3 bucket, - -```console -$ echo -n '' > AWS_ACCESS_KEY_ID -$ echo -n '' > AWS_SECRET_ACCESS_KEY -$ kubectl create secret generic -n demo s3-secret \ - --from-file=./AWS_ACCESS_KEY_ID \ - --from-file=./AWS_SECRET_ACCESS_KEY -secret/s3-secret created -``` - -**Create BackupStorage:** - -Now, crete a `BackupStorage` using this secret. Below is the YAML of BackupStorage crd we are going to create, - -```yaml -apiVersion: storage.kubestash.com/v1alpha1 -kind: BackupStorage -metadata: - name: s3-storage - namespace: demo -spec: - storage: - provider: s3 - s3: - endpoint: us-east-1.linodeobjects.com - bucket: kubestash-testing - region: us-east-1 - prefix: demo - secret: s3-secret - usagePolicy: - allowedNamespaces: - from: All - deletionPolicy: WipeOut -``` - -Let's create the `BackupStorage` we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/backupstorage.yaml -storage.kubestash.com/s3-storage created -``` - -Now, we are ready to backup our database to our desired backend. - -### Backup - -We have to create a `BackupConfiguration` targeting respective MongoDB crd of our desired database. Then KubeStash will create a CronJob to periodically backup the database. Before that we need to create an secret for encrypt data and retention policy. - -**Create Encryption Secret:** - -EncryptionSecret refers to the Secret containing the encryption key which will be used to encode/decode the backed up data. Let's create a secret called `encry-secret` - -```console -$ kubectl create secret generic encry-secret -n demo \ - --from-literal=RESTIC_PASSWORD='123' -n demo -secret/encry-secret created -``` - -**Create Retention Policy:** - -`RetentionPolicy` specifies how the old Snapshots should be cleaned up. This is a namespaced CRD.However, we can refer it from other namespaces as long as it is permitted via `.spec.usagePolicy`. Below is the YAML of the `RetentionPolicy` called `backup-rp` - -```console -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All -``` - -Let's create the RetentionPolicy we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/retentionpolicy.yaml -retentionpolicy.storage.kubestash.com/backup-rp created -``` - -**Create BackupConfiguration:** - -As we just create our encryption secret and retention policy, we are now ready to apply `BackupConfiguration` crd to take backup out database. - -Below is the YAML for `BackupConfiguration` crd to backup the `sample-mongodb` database we have deployed earlier., - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mongodb - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - schedule: "*/3 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /mongodb - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup -``` - -Here, - -- `spec.target` specifies our targeted `MongoDB` database. -- `spec.backends` specifies `BackupStorage` information for storing data. -- `spec.sessions` specifies common session configurations for this backup -- `spec.sessions.schedule` specifies that we want to backup the database at 5 minutes interval. -- `spec.sessions.addon` refers to the `Addon` crd for backup task - -Let's create the `BackupConfiguration` crd we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/backupconfiguration.yaml -backupconfiguration.core.kubestash.com/mg created -``` - -**Verify Backup Setup Successful:** - -If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, - -```console -$ kubectl get backupconfiguration -n demo -NAME PHASE PAUSED AGE -mg Ready 85s -``` - -**Verify CronJob:** - -KubeStash will create a CronJob with the schedule specified in `spec.sessions.schedule` field of `BackupConfiguration` crd. - -Verify that the CronJob has been created using the following command, - -```console -$ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -trigger-mg-frequent */3 * * * * False 0 101s -``` - -**Wait for BackupSession:** - -The `trigger-mg-frequent` CronJob will trigger a backup on each schedule by creating a `BackpSession` crd. - -Wait for the next schedule. Run the following command to watch `BackupSession` crd, - -```console -$ kubectl get backupsession -n demo -NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE -mg-frequent-1701923402 BackupConfiguration mg Succeeded 3m4s -mg-frequent-1701923582 BackupConfiguration mg Running 4s -``` - -We can see above that the backup session has succeeded. Now, we are going to verify that the backed up data has been stored in the backend. - -**Verify Backup:** - -Once a backup is complete, KubeStash will update the respective `Snapshot` crd to reflect the backup. It will be created when a backup is triggered. Check that the `Snapshot` Phase to verify backup. - -```console -$ kubectl get snapshot -n demo -NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE -s3-repo-mg-frequent-1701923402 s3-repo frequent 2023-12-07T04:30:10Z Delete Succeeded 3m25s -s3-repo-mg-frequent-1701923582 s3-repo frequent 2023-12-07T04:33:06Z Delete Succeeded 25s -``` - - -KubeStash will also update the respective `Repository` crd to reflect the backup. Check that the repository `s3-repo` has been updated by the following command, - -```console -$ kubectl get repository -n demo s3-repo -NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE -s3-repo true 2 2.613 KiB Ready 2m42s 8m38s -``` - -Now, if we navigate to the S3 bucket, we are going to see backed up data has been stored in `demo/mongodb/` directory as specified by `spec.sessions.repositories.directory` field of `BackupConfiguration` crd. - -> Note: KubeStash keeps all the backed up data encrypted. So, data in the backend will not make any sense until they are decrypted. - -## Restore MongoDB -You can restore your data into the same database you have backed up from or into a different database in the same cluster or a different cluster. In this section, we are going to show you how to restore in the same database which may be necessary when you have accidentally deleted any data from the running database. - -#### Stop Taking Backup of the Old Database: - -It's important to stop taking any further backup of the old database so that no backup is stored in our repository during restore process. KubeStash operator will automatically pause the `BackupConfiguration` when a `RestoreSession` is running. However if we want to pause the `BackupConfiguration` manually, we can do that by patching or using KubeStash CLI. - -Let's pause the `mg` BackupConfiguration by patching, -```bash -$ kubectl patch backupconfiguration -n demo mg --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.core.kubestash.com/mg patched -``` - -Now, wait for a moment. KubeStash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, - -```console -$ kubectl get backupconfiguration -n demo mg -NAME PHASE PAUSED AGE -mg Ready true 26m -``` - -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. - -#### Deploy New MongoDB Database For Restoring: - -We are going to deploy a new mongodb standalone database for restoring backed up data. - -Below is the YAML of a sample `MongoDB` crd that we are going to create - -```yaml -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: restore-mongodb - namespace: demo -spec: - version: "4.2.24" - storageType: Durable - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut -``` - -Create the above `MongoDB` crd, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/mongodb-restore.yaml -mongodb.kubedb.com/restore-mongodb created -``` - -Let's check if the database is ready to use, - -```console -$ kubectl get mg -n demo restore-mongodb -NAME VERSION STATUS AGE -restore-mongodb 4.2.24 Ready 3m30s -``` - -Let's verify all the databases of this `restore-mongodb` by exec into its pod - -```console -$ export USER=$(kubectl get secrets -n demo restore-mongodb-auth -o jsonpath='{.data.\username}' | base64 -d) - -$ export PASSWORD=$(kubectl get secrets -n demo restore-mongodb-auth -o jsonpath='{.data.\password}' | base64 -d) - -$ kubectl exec -it -n demo restore-mongodb-0 -- mongo admin -u $USER -p $PASSWORD - -> show dbs -admin 0.000GB -config 0.000GB -kubedb-system 0.000GB -local 0.000GB - -> show users -{ - "_id" : "admin.root", - "userId" : UUID("a4decc6b-959a-434d-9e4b-19cb9bfa783b"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -> exit -bye -``` - -As we can see no database named `newdb` exist in this new `restore-mongodb` database. - -#### Create RestoreSession: - -Now, we need to create a `RestoreSession` crd pointing to the `restore-mongodb` database. -Below is the YAML for the `RestoreSession` crd that we are going to create to restore the backed up data. - -```yaml -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-restore - namespace: demo -spec: - target: - name: restore-mongodb - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestore -``` - -Here, - -- `spec.dataSource.repository` specifies the `Repository` crd that holds the backend information where our backed up data has been stored. -- `spec.target` refers to the `MongoDB` crd for the `restore-mongodb` database. -- `spec.dataSource.snapshot` specifies that we are restoring from the latest backup snapshot of the `spec.dataSource.repository`. - -Let's create the `RestoreSession` crd we have shown above, - -```console -$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/restoresession.yaml -restoresession.core.kubestash.com/mg-restore created -``` - -Once, you have created the `RestoreSession` crd, KubeStash will create a job to restore. We can watch the `RestoreSession` phase to check if the restore process is succeeded or not. - -Run the following command to watch `RestoreSession` phase, - -```console -$ kubectl get restoresession -n demo sample-mongodb-restore -w -NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE -mg-restore s3-repo Succeeded 8s 49s -``` - -So, we can see from the output of the above command that the restore process succeeded. - -#### Verify Restored Data: - -In this section, we are going to verify that the desired data has been restored successfully. We are going to connect to the `restore-mongodb` database and check whether the table we had created earlier is restored or not. - -Lets, exec into the database pod and list available tables, - -```console -$ kubectl exec -it -n demo restore-mongodb-0 -- mongo admin -u $USER -p $PASSWORD - -> show dbs -admin 0.000GB -config 0.000GB -kubedb-system 0.000GB -local 0.000GB -newdb 0.000GB - -> show users -{ - "_id" : "admin.root", - "userId" : UUID("a4decc6b-959a-434d-9e4b-19cb9bfa783b"), - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ], - "mechanisms" : [ - "SCRAM-SHA-1", - "SCRAM-SHA-256" - ] -} - -> use newdb -switched to db newdb - -> db.movie.find().pretty() -{ "_id" : ObjectId("5d19d1cdc93d828f44e37735"), "name" : "batman" } - -> exit -bye -``` - -So, from the above output, we can see the database `newdb` that we had created earlier is restored into another new `MongoDB` database. - -## Cleanup - -To cleanup the Kubernetes resources created by this tutorial, run: - -```console -kubectl delete -n demo restoresession mg-restore -kubectl delete -n demo backupconfiguration mg -kubectl delete -n demo mg sample-mongodb -kubectl delete -n demo mg restore-mongodb -kubectl delete -n demo backupstorage s3-storage -``` diff --git a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/backupconfiguration.yaml b/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/backupconfiguration.yaml deleted file mode 100644 index f4b7178cc..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/backupconfiguration.yaml +++ /dev/null @@ -1,36 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: BackupConfiguration -metadata: - name: mg - namespace: demo -spec: - target: - apiGroup: kubedb.com - kind: MongoDB - namespace: demo - name: sample-mongodb - backends: - - name: s3-backend - storageRef: - namespace: demo - name: s3-storage - retentionPolicy: - name: backup-rp - namespace: demo - sessions: - - name: frequent - scheduler: - jobTemplate: - backoffLimit: 1 - schedule: "*/3 * * * *" - repositories: - - name: s3-repo - backend: s3-backend - directory: /mongodb - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackup \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/backupstorage.yaml b/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/backupstorage.yaml deleted file mode 100644 index 4bed4eee3..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/backupstorage.yaml +++ /dev/null @@ -1,18 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: BackupStorage -metadata: - name: s3-storage - namespace: demo -spec: - storage: - provider: s3 - s3: - endpoint: us-east-1.linodeobjects.com - bucket: kubestash-testing - region: us-east-1 - prefix: demo - secretName: s3-secret - usagePolicy: - allowedNamespaces: - from: All - deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/mongodb-restore.yaml b/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/mongodb-restore.yaml deleted file mode 100644 index eaa55e00f..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/mongodb-restore.yaml +++ /dev/null @@ -1,16 +0,0 @@ -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: restore-mongodb - namespace: demo -spec: - version: "4.2.24" - storageType: Durable - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/mongodb.yaml b/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/mongodb.yaml deleted file mode 100644 index abaed3c01..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/mongodb.yaml +++ /dev/null @@ -1,16 +0,0 @@ -apiVersion: kubedb.com/v1alpha2 -kind: MongoDB -metadata: - name: sample-mongodb - namespace: demo -spec: - version: "4.2.24" - storageType: Durable - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - terminationPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/restoresession.yaml b/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/restoresession.yaml deleted file mode 100644 index 1020635be..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/restoresession.yaml +++ /dev/null @@ -1,21 +0,0 @@ -apiVersion: core.kubestash.com/v1alpha1 -kind: RestoreSession -metadata: - name: mg-restore - namespace: demo -spec: - target: - name: restore-mongodb - namespace: demo - apiGroup: kubedb.com - kind: MongoDB - dataSource: - snapshot: latest - repository: s3-repo - encryptionSecret: - name: encry-secret - namespace: demo - addon: - name: mongodb-addon - tasks: - - name: LogicalBackupRestore \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/retentionpolicy.yaml b/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/retentionpolicy.yaml deleted file mode 100644 index 3d327f9a7..000000000 --- a/docs/guides/mongodb/backup/kubestash/logical/standalone/examples/retentionpolicy.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: storage.kubestash.com/v1alpha1 -kind: RetentionPolicy -metadata: - name: backup-rp - namespace: demo -spec: - maxRetentionPeriod: 2mo - successfulSnapshots: - last: 10 - usagePolicy: - allowedNamespaces: - from: All \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/overview/_index.md b/docs/guides/mongodb/backup/kubestash/overview/_index.md deleted file mode 100644 index b09365f90..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/_index.md +++ /dev/null @@ -1,164 +0,0 @@ ---- -title: MongoDB Backup & Restore Overview | KubeStash -menu: - docs_{{ .version }}: - identifier: guides-mongodb-backup-kubestash-overview - name: Overview - parent: guides-mongodb-backup-stashv2 - weight: 10 -menu_name: docs_{{ .version }} ---- - -> New to KubeDB? Please start [here](/docs/README.md). - -{{< notice type="warning" message="This is an Enterprise-only feature. Please install [KubeStash Enterprise Edition] to try this feature. You can use KubeDB Enterprise license to install KubeStash Enterprise edition. Database backup with KubeStash is already included in the KubeDB Enterprise license. So, you don't need a separate license for KubeStash." >}} - -# MongoDB Backup & Restore Overview - -KubeDB can also uses [KubeStash](https://kubestash.com/) to backup and restore databases. KubeStash by AppsCode is a cloud native data backup and recovery solution for Kubernetes workloads. KubeStash utilizes [restic](https://github.com/restic/restic) to securely backup stateful applications to any cloud or on-prem storage backends (for example, S3, GCS, Azure Blob storage, Minio, NetApp, Dell EMC etc.). - -## How Backup Works - -The following diagram shows how KubeStash takes backup of a MongoDB database. Open the image in a new tab to see the enlarged version. - -
- MongoDB Backup Overview -
Fig: MongoDB Backup Overview
-
- -The backup process consists of the following steps: - -1. At first, a user creates a secret with access credentials of the backend where the backed up data will be stored. - -2. Then, she creates a `BackupStorage` crd that specifies the backend information along with the secret that holds the credentials to access the backend. - -3. Then, she creates a `BackupConfiguration` crd targeting the crd of the desired `MongoDB` database. The `BackupConfiguration` object also specifies the `Sessions` to use to backup the database. - -4. KubeStash operator watches for `BackupConfiguration` crd. - -5. Once KubeStash operator finds a `BackupConfiguration` crd, it creates a CronJob with the schedule specified in `BackupConfiguration` object to trigger backup periodically. - -6. On the next scheduled slot, the CronJob triggers a backup by creating a `BackupSession` crd. - -7. KubeStash operator also watches for `BackupSession` crd. - -8. When it finds a `BackupSession` object, It creates a `Snapshot` for holding backup information. - -9. KubeStash resolves the respective `Addon` and `Function` and prepares a Job definition to backup. - -10. Then, it creates the Job to backup the targeted database. - -11. The backup Job reads necessary information to connect with the database from the `AppBinding` crd of the targated `MongoDB` database. It also reads backend information and access credentials from `BackupStorage` crd and Storage Secret respectively through `Backend` section of `BackupConfiguration` crd - -12. Then, the Job dumps the targeted database and uploads the output to the backend. KubeStash pipes the output of dump command to uploading process. Hence, backup Job does not require a large volume to hold the entire dump output. - -13. Finally, when the backup is complete, the Job sends Prometheus metrics to the Pushgateway running inside KubeStash operator pod. It also updates the `BackupSession` and `Snapshot` status to reflect the backup procedure. - -### Backup Different MongoDB Configurations - -This section will show you how backup works for different MongoDB configurations. - -#### Standalone MongoDB - -For a standalone MongoDB database, the backup job directly dumps the database using `mongodump` and pipe the output to the backup process. - -
- Standalone MongoDB Backup Overview -
Fig: Standalone MongoDB Backup
-
- -#### MongoDB ReplicaSet Cluster - -For MongoDB ReplicaSet cluster, KubeStash takes backup from one of the secondary replicas. The backup process consists of the following steps: - -1. Identify a secondary replica. -2. Lock the secondary replica. -3. Backup the secondary replica. -4. Unlock the secondary replica. - -
- MongoDB ReplicaSet Cluster Backup Overview -
Fig: MongoDB ReplicaSet Cluster Backup
-
- -#### MongoDB Sharded Cluster - -For MongoDB sharded cluster, KubeStash takes backup of the individual shards as well as the config server. KubeStash takes backup from a secondary replica of the shards and the config server. If there is no secondary replica then KubeStash will take backup from the primary replica. The backup process consists of the following steps: - -1. Disable balancer. -2. Lock config server. -3. Identify a secondary replica for each shard. -4. Lock the secondary replica. -5. Run backup on the secondary replica. -6. Unlock the secondary replica. -7. Unlock config server. -8. Enable balancer. - -
- MongoDB Sharded Cluster Backup Overview -
Fig: MongoDB Sharded Cluster Backup
-
- -## How Restore Process Works - -The following diagram shows how KubeStash restores backed up data into a MongoDB database. Open the image in a new tab to see the enlarged version. - -
- Database Restore Overview -
Fig: MongoDB Restore Process Overview
-
- -The restore process consists of the following steps: - -1. At first, a user creates a `RestoreSession` crd targeting the `AppBinding` of the desired database where the backed up data will be restored. It also specifies the `Repository` crd which holds the backend information and the `Task` to use to restore the target. - -2. KubeStash operator watches for `RestoreSession` object. - -3. Once it finds a `RestoreSession` object, it resolves the respective `Task` and `Function` and prepares a Job definition to restore. - -4. Then, it creates the Job to restore the target. - -5. The Job reads necessary information to connect with the database from respective `AppBinding` crd. It also reads backend information and access credentials from `Repository` crd and Storage Secret respectively. - -6. Then, the job downloads the backed up data from the backend and injects into the desired database. KubeStash pipes the downloaded data to the respective database tool to inject into the database. Hence, restore job does not require a large volume to download entire backup data inside it. - -7. Finally, when the restore process is complete, the Job sends Prometheus metrics to the Pushgateway and update the `RestoreSession` status to reflect restore completion. - -### Restoring Different MongoDB Configurations - -This section will show you restore process works for different MongoDB configurations. - -#### Standalone MongoDB - -For a standalone MongoDB database, the restore job downloads the backed up data from the backend and pipe the downloaded data to `mongorestore` command which inserts the data into the desired MongoDB database. - -
- Standalone MongoDB Restore Overview -
Fig: Standalone MongoDB Restore
-
- -#### MongoDB ReplicaSet Cluster - -For MongoDB ReplicaSet cluster, KubeStash identifies the primary replica and restore into it. - -
- MongoDB ReplicaSet Cluster Restore Overview -
Fig: MongoDB ReplicaSet Cluster Restore
-
- -#### MongoDB Sharded Cluster - -For MongoDB sharded cluster, KubeStash identifies the primary replica of each shard as well as the config server and restore respective backed up data into them. - -
- MongoDB Sharded Cluster Restore -
Fig: MongoDB Sharded Cluster Restore
-
- -## Next Steps - -- Backup a standalone MongoDB databases using KubeStash following the guide from [here](/docs/guides/mongodb/backup/kubestash/logical/standalone/_index.md). -- Backup a MongoDB Replicaset cluster using Stash following the guide from [here](/docs/guides/mongodb/backup/kubestash/logical/replicaset/_index.md). -- Backup a sharded MongoDB cluster using KubeStash following the guide from [here](/docs/guides/mongodb/backup/kubestash/logical/sharding/_index.md). - - diff --git a/docs/guides/mongodb/backup/kubestash/overview/images/backup_overview.svg b/docs/guides/mongodb/backup/kubestash/overview/images/backup_overview.svg deleted file mode 100644 index d2c6722b0..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/images/backup_overview.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/overview/images/replicaset_backup.svg b/docs/guides/mongodb/backup/kubestash/overview/images/replicaset_backup.svg deleted file mode 100644 index e41cdcc77..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/images/replicaset_backup.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
KubeStash
Backup Job
KubeStash...
R1
R1
R2
R2
R3
R3
MongoDB
ReplicaSet
Cluster
MongoDB...
Primary
Primary
Secondary
Secondary
Secondary
Secondary
mongodump
mongodump
Text is not SVG - cannot display
\ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/overview/images/replicaset_restore.svg b/docs/guides/mongodb/backup/kubestash/overview/images/replicaset_restore.svg deleted file mode 100644 index c6cfa868b..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/images/replicaset_restore.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
KubeStash
Backup Job
KubeStash...
R1
R1
R2
R2
R3
R3
MongoDB
ReplicaSet
Cluster
MongoDB...
Primary
Primary
Secondary
Secondary
Secondary
Secondary
mongodump
mongodump
Text is not SVG - cannot display
\ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/overview/images/restore_overview.svg b/docs/guides/mongodb/backup/kubestash/overview/images/restore_overview.svg deleted file mode 100644 index 2b587b058..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/images/restore_overview.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/overview/images/sharded_backup.svg b/docs/guides/mongodb/backup/kubestash/overview/images/sharded_backup.svg deleted file mode 100644 index 3e0b4cb46..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/images/sharded_backup.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
KubeStash
Backup Job
KubeStash...
R1
R1
R2
R2
R3
R3
MongoDB
ReplicaSet
Cluster
MongoDB...
Primary
Primary
Secondary
Secondary
Secondary
Secondary
mongodump
mongodump
Text is not SVG - cannot display
\ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/overview/images/sharded_restore.svg b/docs/guides/mongodb/backup/kubestash/overview/images/sharded_restore.svg deleted file mode 100644 index 25b7126c0..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/images/sharded_restore.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
KubeStash
Backup Job
KubeStash...
R1
R1
R2
R2
R3
R3
MongoDB
ReplicaSet
Cluster
MongoDB...
Primary
Primary
Secondary
Secondary
Secondary
Secondary
mongodump
mongodump
Text is not SVG - cannot display
\ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/overview/images/standalone_backup.svg b/docs/guides/mongodb/backup/kubestash/overview/images/standalone_backup.svg deleted file mode 100644 index 47eb24665..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/images/standalone_backup.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
KubeStash
Backup Job
KubeStash...
MongoDB
MongoDB
mongodump
mongodump
Text is not SVG - cannot display
\ No newline at end of file diff --git a/docs/guides/mongodb/backup/kubestash/overview/images/standalone_restore.svg b/docs/guides/mongodb/backup/kubestash/overview/images/standalone_restore.svg deleted file mode 100644 index 1c72f07bb..000000000 --- a/docs/guides/mongodb/backup/kubestash/overview/images/standalone_restore.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
KubeStash
Backup Job
KubeStash...
R1
R1
R2
R2
R3
R3
MongoDB
ReplicaSet
Cluster
MongoDB...
Primary
Primary
Secondary
Secondary
Secondary
Secondary
mongodump
mongodump
Text is not SVG - cannot display
\ No newline at end of file