Run ‘calibrate-blob-storage’ on S3-compatible object stores to measure blob storage performance.

Usage

Run 'calibrate-blob-storage' on S3-compatible object stores to measure blob storage performance.
This command measures the upload and download throughput of the cluster. 

A "calibrate_storage" database is created using the unlimited storage settings that are specified via command-line arguments and environment variables. The database is then populated with the dataset generated by this command.

The workload consists of the following steps:
 - Upload test
   - Create the "calibrate_storage" database
   - Generate the dataset
   - Introduce a high load on the cluster by maintaining multiple concurrent loads against the aggregator nodes for 10 minutes
 - Download test
   - Run aggregate queries against the data loaded during the upload test
 - Retrieve the results of the run and save them in a file on the Master Aggregator host

The maximum wait time for the upload queue during the upload test is divided into three main categories:
 - FAST: under 2 minutes
 - INTERMEDIATE: between 2 and 3 minutes
 - SLOW: over 3 minutes

The download test uses data that was loaded into the database during the upload test and calculates the average download throughput.

Examples:

# A simple example that runs a 'calibrate-blob-storage' workload with the root user of the cluster
sdb-report calibrate-blob-storage -r 's3://test/bucket?region=eu-central-1'

# Runs a 'calibrate-blob-storage' workload with a user 'user' and a password 'pass'
sdb-report calibrate-blob-storage -r 's3://test/bucket?region=eu-central-1' -u user -p pass

Usage:
  sdb-report calibrate-blob-storage [flags]

Flags:
      --blob-cache-eviction MINUTES   Set a timeout for the blob cache cleanup performed during the download test (ADVANCED) (default 20)
  -h, --help                          Help for calibrate-blob-storage
  -p, --password STRING               The database user's password for connecting to SingleStoreDB. If a password is specified on the command line, it must not contain an unescaped '$' character as it will be replaced by the shell
  -r, --repository STRING             Path to the repository in the URL format. See https://docs.singlestore.com/toolbox-redir/memsql-backup-repository for examples
  -u, --user string                   The database user for connecting to SingleStoreDB (default "root")

Global Flags:
      --backup-cache FILE_PATH                File path for the backup cache
      --cache-file FILE_PATH                  File path for the Toolbox node cache
  -c, --config FILE_PATH                      File path for the Toolbox configuration
      --disable-colors                        Disable color output in console, which some terminal sessions/environments may have difficulty with
      --disable-spinner                       Disable the progress spinner, which some terminal sessions/environments may have issues with
  -j, --json                                  Enable JSON output
      --parallelism POSITIVE_INTEGER          Maximum number of operations to run in parallel
      --runtime-dir DIRECTORY_PATH            Where to store Toolbox runtime data
      --ssh-control-persist SECONDS           Enable SSH ControlPersist and set it to the specified duration in seconds
      --ssh-max-sessions POSITIVE_INTEGER     Maximum number of SSH sessions to open per host, must be at least 3
      --ssh-strict-host-key-checking          Enable strict host key checking for SSH connections
      --ssh-user-known-hosts-file FILE_PATH   Path to the user known_hosts file for SSH connections. If not set, /dev/null will be used
      --state-file FILE_PATH                  Toolbox state file path
  -v, --verbosity count                       Increase logging verbosity: valid values are 1, 2, 3. Usage -v=count or --verbosity=count
  -y, --yes                                   Enable non-interactive mode and assume the user would like to move forward with the proposed actions by default

Remarks

This command is interactive unless you use either the --yes or --json flags to override interactive behavior.

Was this Article Helpful?

Thanks For Your Feedback!