Pull a Backup Image

By default, if the backup job is enabled, it will be launched using an image that is different from either the Operator image or the Node image stored at Docker Hub.

If you opt to let the backup job use the Docker Hub image directly, nothing else needs to be done.

If your deployment environment does not allow images to be pulled from Docker Hub, you must manually download the backup image, push it to a container image registry that is accessible by your deployment, and configure the Operator to use this image for your backup job.

1. Run the following commands to pull the image from Docker Hub and push it to your internal registry.

   docker pull memsql/tools:centos-v1.11.6-1.6.4-4862f19001e0b8c02199171dc2cc905a0d7c380b
   

   Note: Be sure to change the actual internal image name.

   docker tag memsql/tools:centos-v1.11.6-1.6.4-4862f19001e0b8c02199171dc2cc905a0d7c380b \
   internal_registry/memsql/tools:centos-v1.11.6-1.6.4-4862f19001e0b8c02199171dc2cc905a0d7c380b
   

   docker push internal_registry/memsql/tools:centos-v1.11.6-1.6.4-4862f19001e0b8c02199171dc2cc905a0d7c380b
   

2. Add a spec.template.spec.containers.env environment variable to the deployment.yaml file and apply the change.

   - name: RELATED_IMAGE_BACKUP
                 value: "internal_registry/memsql/tools:centos-v1.11.6-1.6.4-4862f19001e0b8c02199171dc2cc905a0d7c380b"
   

   The deployment.yaml file will resemble the following.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: memsql-operator
   spec:
     replicas: 1
     selector:
       matchLabels:
         name: memsql-operator
     template:
       metadata:
         labels:
           name: memsql-operator
       spec:
         serviceAccountName: memsql-operator
         containers:
           - name: memsql-operator
             image: "OPERATOR_IMAGE"
             imagePullPolicy: Always
             args: [
               # including both non-backup related and 
               # backup related arguments
             ]
             env:
               - name: WATCH_NAMESPACE
                 valueFrom:
                   fieldRef:
                     fieldPath: metadata.namespace
               - name: POD_NAME
                 valueFrom:
                   fieldRef:
                     fieldPath: metadata.name
               - name: OPERATOR_NAME
                 value: "memsql-operator"
               - name: RELATED_IMAGE_BACKUP
                 value: "internal_registry/memsql/tools:centos...380b"
   

3. Run the following command to apply the changes in the deployment.yaml file.

   kubectl apply -f deployment.yaml
   

   Note: You must apply the changes in the memsql-cluster.yaml file if it has also been edited.

Backup Configurations

As of Operator 1.0.0, the deployment.yaml file can be configured so that the Operator will manage and perform backups. These new parameters are:

• backup-bucket-name: The AWS S3 bucket, including the path if desired
• backup-default-schedule: Default schedule for backup databases, in cron format. See the below for more information. If backup-incremental-default-schedule is specified, the scheduled job will perform a backup using BACKUP DATABASE ... WITH INIT ..., otherwise, it will perform a plain BACKUP DATABASE without using the WITH INIT option.
• backup-s3-endpoint: Provide compatible AWS S3 endpoint URL, otherwise leave blank (optional)
• backup-s3-region: Name of the AWS S3 region (optional)
• backup-secret-name: Name of the AWS Secret for the S3 backup

As of Operator 1.1.1, the deployment.yaml file may also include:

• backup-compatibility-mode: Disables certain optimizations that only work with native S3, the primary use case for which is backing up to GCS. (optional) Refer to BACKUP DATABASE for more information.

As of Operator 1.2.0, the deployment.yaml file may also include:

• backup-incremental-default-schedule: Default schedule for databases incremental backup, in cron format. If this is configured, it will perform an incremental backup using BACKUP DATABASE ... WITH DIFFERENTIAL ... option if there is already a backup created by using WITH INIT option, otherwise the first time the incremental backup is run, it will perform a backup using WITH INIT option. See the below for more information.
• backup-default-deadline-seconds: Default deadline in seconds that the backup job has to finish before it is terminated. Defaults to 3600 seconds if not specified.
• backup-incremental-default-deadline-seconds: Default deadline in seconds that the incremental backup job has to finish before it is terminated. Defaults to 3600 seconds if not specified. It’s suggested to make this value the same value as backup-default-deadline-seconds.

As of Operator 1.2.0, you may also specify backup details via backupSpec in the CR, which includes the following fields. These fields all have matching parts in the deployment.yaml file. Field specified in backupSpec in the CR takes precedence over the parameter specified in deployment.yaml.

• bucketName:
• s3Region:
• s3Endpoint:
• secretName:
• Schedule:
• deadlineSeconds:
• scheduleIncremental:
• deadlineSecondsIncremental:
• compatibilityMode:

You may define the --backup-default-schedule and/or backup-incremental-default-schedule using the following cron job scheduling format:

For example:

                              +---------------- MIN
                              | +------------- HOUR
                              | | +---------- DOM
                              | | | +------- MON
                              | | | | +---- DOW
                              | | | | |
"--backup-default-schedule", "0 1 * * *",

The following is an example deployment.yaml file that’s configured for backups using the parameters above:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: memsql-operator
  namespace: memsql
spec:
  replicas: 1
  selector:
    matchLabels:
      name: memsql-operator
  template:
    metadata:
      labels:
        name: memsql-operator
    spec:
      serviceAccountName: memsql-operator
      containers:
        - name: memsql-operator
          image: "memsql/operator"
          imagePullPolicy: Never
          args: [
            # Cause the operator to merge rather than replace annotations on services
            "--merge-service-annotations",
            # Allow the process inside the container to have read/write access to the volume.
            "--fs-group-id", "5555",
            # Configure operator to run backup daily at 1 AM
            "--backup-bucket-name", "<bucket-name/path>",
            "--backup-default-schedule", "0 1 * * *",
            "--backup-s3-endpoint", "",
            "--backup-s3-region", "<AWS region for example us-west-1>",
            "--backup-secret-name", "aws-credential"
          ]
          env:
            - name: WATCH_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: OPERATOR_NAME
              value: "memsql-operator"

The following is an example aws-secret.yaml file for using AWS S3 to store backups. It reflects an AWS S3 Access Secret configuration.

apiVersion: v1
kind: Secret
metadata:
  name: aws-credential
  namespace: memsql
type: Opaque
stringData:
  AWS_ACCESS_KEY: <AWS-Access-Key>
  AWS_SECRET_ACCESS_KEY: <AWS-Secret>

As of Operator 1.2.3, you may also specify the name of a Secret for the backup job’s user. The default resource pool associated with that user will be used for the backup job. The user can be specified either via userSecret in backupSpec, or via backup-user-secret Operator command line parameter in deployment.yaml. The field specified in backupSpec in the CR takes precedence over the parameter specified in deployment.yaml.

The following is an example backup-user-secret.yaml file.

apiVersion: v1
kind: Secret
metadata:
  name: backup-user-secret
  namespace: memsql
type: Opaque
stringData:
  BACKUP_USERNAME: <username>
  BACKUP_PASSWORD: <password>

Was this Article Helpful?

Thanks For Your Feedback!