Deploy a MemSQL cluster.

Usage

Deploy a SingleStore DB cluster on multiple machines

memsql-deploy setup-cluster conveniently runs various memsql-toolbox-config,
memsql-deploy, and memsql-admin commands as a single command to produce a
functional SingleStore DB cluster that can be connected to and queried. If you would
instead like to set up a local SingleStore DB "cluster" for testing purposes, see
'memsql-deploy cluster-in-a-box'.

Example usages:

    memsql-deploy setup-cluster \
      --master-host host1 \
      --aggregator-hosts host2 \
      --leaf-hosts host3,host4 \
      --identity-file ~/my_key.pem \
      --license AAAAAAa/Aaa/AA== \
      --password testpass

This will install the latest released version of memsql-server on each host and
deploy a master on host1; a child aggregator on host2; and leaf nodes on host3
and host4. The hosts host1, host2... must be accessible via SSH using the
current user and the key ~/my_key.pem. A valid license and password for the
SingleStore DB root user are also required. 
If you set a MEMSQL_LICENSE environment variable, you can omit the --license flag.

Other examples:

"High Availability" mode is enabled by default. To disable it, set
--high-availability to false:

    memsql-deploy setup-cluster --master-host host1 ... --high-availability=false


The command will deploy SingleStore DB on port 3306 on each node by default. To
configure it to a different port, set --memsql-port:

    memsql-deploy setup-cluster --master-host host1 ... --memsql-port 3307


If you want to install a version other than the latest released version, you can
specify a different released version, or the path to a memsql-server package:

    memsql-deploy setup-cluster --master-host host1 ... --version 6.7.0

    memsql-deploy setup-cluster --master-host host1 ... --file-path path/to/memsql-server.deb
    memsql-deploy setup-cluster --master-host host1 ... --file-path path/to/memsql-server.rpm


If you need to specify a host's SSH user or SSH port, you can add the
information to the host descriptor:

	memsql-deploy setup-cluster --master-host admin@host1:10022 --leaf-hosts admin@host2:10022,admin@host3:10022 ...


If you want to use a YAML file to define your cluster you can do so by using the
--cluster-file flag which takes in the path to a YAML file as input. Note that you have to
give all the configurations as part of the cluster file and cannot use any other flag along
with --cluster-file. If you want to learn about creating a cluster file please
refer to https://docs.singlestore.com/toolbox-redir/setup-cluster-file
Usage of the command with --cluster-file flag:
	memsql-deploy setup-cluster --cluster-file path/to/cluster-file.yml

Usage:
  memsql-deploy setup-cluster [flags]

Flags:
      --aggregator-hosts strings               SSH addresses of the child aggregator hosts
      --allow-duplicate-host-fingerprints      Whether to allow hosts with identical SSH host keys. (ADVANCED)
      --base-install-dir ABSOLUTE_PATH         The absolute path to the base install directory for all the nodes
      --cluster-file FILE_PATH                 The path to the cluster definition file
      --data-dir ABSOLUTE_PATH                 The absolute path to the data directory for all the nodes
      --file-path ABSOLUTE_PATH                The memsql-server package to install
      --force-package-format {deb, rpm, tar}   Specify 'deb', 'rpm', or 'tar' package format. Required if you have both dpkg and rpm package managers found on the host
  -h, --help                                   Help for setup-cluster
      --high-availability                      Indicate whether to deploy the cluster in high availability mode (default true)
      --hosts-file ABSOLUTE_PATH               The file specifying the host addresses for the master aggregator, child aggregators, and leaves, in INI format with [master], [aggregators], and [leaves] sections
  -i, --identity-file ABSOLUTE_PATH            The identity file to SSH to each host
      --leaf-hosts strings                     SSH addresses of each leaf host. Must specify at least one
      --license LICENSE                        The SingleStore DB license for the cluster
      --master-host string                     SSH address of the master aggregator host
      --memsql-port PORT                       The port each SingleStore DB node will run on
      --password STRING                        The password for the SingleStore DB root user on each node
      --preinstalled-path ABSOLUTE_PATH        The path to the unpacked memsql-server file. When unpacked, the folder name must be of the form 'memsql-server-<version>' (ADVANCED)
      --skip-validate-env                      Skip environment validation
      --version VERSION                        The version of memsql-server to install (default production:latest)

Global Flags:
      --backup-cache FILE_PATH              File path for the backup cache
      --cache-file FILE_PATH                File path for the Toolbox node cache
  -c, --config FILE_PATH                    Toolbox configuration file path
      --disable-spinner                     Disable the progress spinner, which some terminal sessions/environments may have issues with
  -j, --json                                Enable JSON output
      --parallelism POSITIVE_INTEGER        Maximum number of operations to run in parallel
      --runtime-dir DIRECTORY_PATH          Where to store Toolbox runtime data
      --ssh-max-sessions POSITIVE_INTEGER   Maximum number of SSH sessions to open per host, must be at least 3
      --state-file FILE_PATH                Toolbox state file path
  -v, --verbosity count                     Increase logging verbosity: valid values are 1, 2, 3. Usage -v=count or --verbosity=count
  -y, --yes                                 Enable non-interactive mode and assume the user would like to move forward with the proposed actions by default

YAML file configuration

As of MemSQL toolbox 1.3.0, the memsql-deploy setup-cluster command now accepts YAML configuration files, and validates the configuration file format before attempting to set up the specified cluster.

The command is designed to be consistent, where re-running the memsql-deploy setup-cluster command with the same cluster file will always produce the same cluster. It is also resilient, allowing errors encountered at any stage of the cluster construction process to be corrected, and memsql-deploy setup-cluster re-run, in order to generate the desired cluster.

Steps for using a configuration file

1. Install MemSQL toolbox onto the main deployment machine.

2. Using the template below, create a YAML file with your desired cluster configuration.

   license: <license-from-portal.memsql.com>
   memsql_server_version: <version>
   package_type: <type> 					      ← rpm, deb, or tar
   hosts:
   - hostname: <ip-address>
     localhost: true
     nodes:
     - register: false
       role: Master
       config:
         password: <secure-password>
         port: 3306
     - register: false
       role: Leaf
       config:
         password: <secure-password>
         port: 3307
   

3. Run the following with the path to the YAML file as input.

   memsql-deploy setup-cluster --cluster-file <path-to-cluster-file>
   

Complete cluster file template

license:                    <LICENSE> [Required to bootstrap Master Aggregator]
high_availability:          <true | false>
memsql_server_version:      <the version of memsql you want to install (6.7+)>
memsql_server_file_path:    <path to the downloaded memsql server file>
package_type:  		        <deb|rpm|tar> [Required if multiple package present]
root_password:		        <default password to be used for all nodes>
optimize:                   <true | false>
optimize_config:
  memory_percentage:        <percentage of memory you want memsql to use>
  no_numa: 			        <true|false>
hosts:
- hostname:                 <host-name> [Required]
  localhost:                <true | false> 
  memsqlctl_path:           <path to memsqlctl> [ADVANCED]
  memsqlctl_config_path:    <path to memsqlctl config> [ADVANCED]
  tar_install_dir:          <path to tar install dir> [ADVANCED]
  tar_install_state:        <path to tar install state> [ADVANCED]
  ssh:                      [Required for remote Hosts]
    host:                   <ssh host name>
    port:                   <ssh port>
    user:                   <ssh user>
    private_key:            <path to your identity key>
  nodes:
  - register: 	            <true | false>
    role:                   <Unknown | Master | Leaf | Aggregator> (case sensitive) [Required]
    availability_group:     <availability group>
    no_start:               <true | false>
    config: 
      auditlogsdir:         <path to auditlogs directory> [ADVANCED]
      baseinstalldir:       <path to base install directory> [ADVANCED]
      configpath:           <path to configuration path> [ADVANCED] [Required if register is true]
      datadir:              <path to data directory> [ADVANCED]
      disable_auto_restart: <true | false>
      password:             <password>
      plancachedir:		    <path to plancache directory> [ADVANCED]
      port:                 <port number> [Required for node creation]
      tracelogsdir:         <path to tracelogs directory> [ADVANCED]
      bind_address:         <bind address> [ADVANCED]

Example cluster files

The following cluster file template examples can be used to set up a cluster.

Copy an example into an example.yaml file and pass this file to the memsql-deploy setup-cluster command.

memsql-deploy setup-cluster --cluster-file example.yaml

Note: Depending on how you installed MemSQL toolbox (for example, via tarball-based install, you may need to include the full path to the cluster file.

./memsql-deploy setup-cluster --cluster-file /full-path/example.yaml

For Example 2, Example 3, and Example 4, you will need multiple hosts and the ability to ssh into each host from the main deployment machine.

Example 1

license: <license-from-portal.memsql.com>
memsql_server_version: 6.8.1
package_type: rpm
hosts:
- hostname: 127.0.0.1
  localhost: true
  nodes:
  - register: false
    role: Master
    config:
      password: <secure-password>
      port: 3306
  - register: false
    role: Leaf
    config:
      password: <secure-password>
      port: 3307

Using this cluster file, memsql-deploy setup-cluster:

1. Registers a local host to the cluster.

2. Installs memsql-server v6.7.1 on this local host.

3. Creates a Master Aggregator node on port 3306.

4. Creates a leaf node on port 3307.

Example 2

license: <license-from-portal.memsql.com>
memsql_server_version: 6.8.1
package_type: rpm
hosts:
- hostname: <hostname1>
  localhost: true
- hostname: <hostname2>
  localhost: false
  ssh: 
    host: <cluster-addressable-hostname>
    private_key: <path/to/private-key>

Using this cluster file, memsql-deploy setup-cluster:

1. Registers a local host to the cluster.

2. Registers a remote host to the cluster using the provided ssh credentials.

3. Installs memsql-server v6.7.1 on both hosts.

Example 3

license: <license-from-portal.memsql.com>
memsql_server_version: 6.8.1
package_type: rpm                
hosts:
- hostname: <hostname1>
  localhost: true
  nodes:
  - register: false
    role: Master
    config:
      password: <secure-password>
      port: 3306
  - register: false
    role: Leaf
    config:
      password: <secure-password>
      port: 3307
- hostname: <hostname2>
  localhost: false
  ssh: 
    host: <cluster-addressable-hostname>
    private_key: <path/to/private-key>
  nodes:
  - register: false
    role: Aggregator
    config:
      password: <secure-password>
      port: 3306
  - register: false
    role: Leaf
    config:
      password: <secure-password>
      port: 3307

Using this cluster file, memsql-deploy setup-cluster:

1. Registers a local host to the cluster.

2. Register a remote host to the cluster using the provided ssh credentials.

3. Installs memsql-server v6.7.1 on both the hosts.

4. On Host 1, creates a node on port 3306, bootstraps it as the Master Aggregator using the specified license, and sets the password to the value provided in the cluster file.

5. On Host 1, creates a node on port 3307, assign it as a leaf node, and sets the password to the value provided in the cluster file.

6. On Host 2, creates a node on port 3306, assign it as a Child Aggregator, and sets the password to the value provided in the cluster file.

7. On Host 2, creates a node on port 3307, assign it as a leaf node, and sets the password to the value provided in the cluster file.

Example 4

license: <license-from-portal.memsql.com>
high_availability: true
package_type: rpm
memsql_server_file_path: /home/ec2-user/memsql-server-7.0.16-5aa7127018.x86_64.rpm
hosts:
- hostname: <hostname1>
  localhost: true
  nodes:
  - role: Master
    config:
      password: <secure-password>
      port: 3306
- hostname: <hostname2>
  ssh:
    host: <cluster-addressable-hostname>
    private_key: <path/to/private-key>
  nodes:
  - role: Aggregator
    config:
      password: <secure-password>
- hostname: <hostname3>
  ssh:
    host: <cluster-addressable-hostname>
    private_key: <path/to/private-key>
  nodes:
  - role: Aggregator
    config:
      password: <secure-password>      
- hostname: <hostname4>
  ssh:
    host: <cluster-addressable-hostname>
    private_key: <path/to/private-key>
  nodes:
  - role: Leaf
    availability_group: 1  
    config:
      password: <secure-password>      
- hostname: <hostname5>
  ssh:
    host: <cluster-addressable-hostname>
    private_key: <path/to/private-key>    
  nodes:
  - role: Leaf
    availability_group: 1
    config:
      password: <secure-password>
- hostname: <hostname6>
  ssh:
    host: <cluster-addressable-hostname>
    private_key: <path/to/private-key>    
  nodes:
  - role: Leaf
    availability_group: 2
    config:
      password: <secure-password>  
- hostname: <hostname7>
  ssh:
    host: <cluster-addressable-hostname>
    private_key: <path/to/private-key>    
  nodes:
  - role: Leaf
    availability_group: 2
    config:
      password: <secure-password>

Using this cluster file, memsql-deploy setup-cluster:

1. Enables high availability.

2. Specifies the use of RPM packages.

3. Specifies the location of the memsql-server package which will be used directly instead of downloaded.

4. Installs memsql-server v7.0.16 on all hosts.

5. On Host 1, creates a node on port 3306, bootstraps it as the Master Aggregator using the specified license, and sets the password to the value provided in the cluster file.

6. On Host 2, creates a node on the default port (3306), assign it as a Child Aggregator, and sets the password to the value provided in the cluster file.

7. On Host 3, creates a node on the default port (3306), assign it as a Child Aggregator, and sets the password to the value provided in the cluster file.

8. On Host 4, creates a node on the default port (3306), assign it as a leaf node, sets the node’s availability group to 1, and sets the password to the value provided in the cluster file.

9. On Host 5, creates a node on the default port (3306), assign it as a leaf node, sets the node’s availability group to 1, and sets the password to the value provided in the cluster file.

10. On Host 6, creates a node on the default port (3306), assign it as a leaf node, sets the node’s availability group to 2, and sets the password to the value provided in the cluster file.

11. On Host 7, creates a node on the default port (3306), assign it as a leaf node, sets the node’s availability group to 2, and sets the password to the value provided in the cluster file.

Definitions

The following is a detailed explanation of each field that can be configured in the cluster file.

CLUSTER

The following two options are related to the version of MemSQL that will be installed for your cluster. Either option can be specified. If neither option is specified, the latest version of memsql-server is installed.

HOST

The following ssh-related fields are required for remote hosts:

NODE

CONFIG

Remarks

This command is interactive unless you use either the --yes or --json flags to override interactive behavior.

Host file structure

For large clusters, you can specify all of your host machine SSH information through an INI file that you reference using the --host-file flag. The format is below.

[master]
[ssh_user@]<master_host_IP_address|DNS_name>[:ssh_port]

[aggregators]
[ssh_user@]<agg_host1_IP_address|DNS_name>[:ssh_port]
...

[leaves]
[ssh_user@]<leaf_host1_IP_address|DNS_name>[:ssh_port]
...

Output

The following example uses the setup-cluster command to set up a simple cluster on two different host machines using a hosts file.

hosts file

[master]
172.18.1.4

[leaves]
172.18.1.5

Setting up a cluster with a hosts file

memsql-deploy setup-cluster -i clusterkey --license <license> --hosts-file hosts --password SecureKey

✓ Registered hosts
memsql-deploy will perform the following actions:
  · Install memsql-server 6.7.1 on hosts
    - 172.18.1.4
    - 172.18.1.5
  · Deploy a master aggregator on 172.18.1.4:3306
    - Enable high availability mode
  · Deploy a leaf node on 172.18.1.5:3306
  · Set root password on all nodes

Would you like to continue? [y/N]: y
✓ Downloaded memsql-server 6.7.1
Installing MemSQL on all hosts...
✓ Installed memsql-server6.7.1-e823135f6e on host 172.18.1.4 (1/2)
✓ Installed memsql-server6.7.1-e823135f6e on host 172.18.1.5 (2/2)
✓ Successfully installed on 2 hosts
✓ Created master node
✓ Successfully set license
✓ Bootstrapped master aggregator
✓ Enabled high availability mode
✓ Created aggregator nodes
✓ Added aggregators nodes to cluster
✓ Created leaf nodes
✓ Added leaf nodes to cluster

To view your cluster, run 'memsql-admin list-nodes'

Was this Article Helpful?

Thanks For Your Feedback!