When running MemSQL on machines with multiple Non-Uniform Memory Access (NUMA) nodes, you should configure MemSQL for NUMA with numactl for optimal performance. If you do not configure MemSQL this way, performance will be greatly degraded due to expensive cross-NUMA-node memory access. Configuring for NUMA should be done as part of the installation process; however, you can reconfigure your deployment later, if necessary.

To simplify the configuration process, MemSQL Ops can automatically detect if a host machine has multiple NUMA nodes and then configure MemSQL with numactl to bind individual MemSQL nodes to NUMA nodes.

Configuring a new cluster for NUMA

Before binding NUMA nodes to specific MemSQL leaf nodes, you must install numactl.

1. On each host, install the numactl package. For example, for Debian-based OSes:

   sudo apt-get install numactl
   

   For Redhat/CentOS, run the following:

   sudo yum install numactl
   

2. Check the number of NUMA nodes your machines by running numactl --hardware. For example:

   numactl --hardware
   available: 2 nodes (0-1)
   

   The output shows that there are 2 NUMA nodes on this machine, numbered 0 and 1.

Configuring for NUMA through the Ops web interface

If you are managing your cluster using MemSQL Ops, install Ops agents on host machines through the Ops web interface. After the Ops agent is installed on each host machine, you can install MemSQL nodes through the UI. The suggested leaf node count should align to the NUMA node count on each machine. Make sure to install one MemSQL leaf node per NUMA node.

MemSQL Ops will then configure the MemSQL nodes appropriately for NUMA on each machine during deployment.

Configuring for NUMA through the Ops CLI

1. On each host that will contain MemSQL leaf nodes, create one leaf per NUMA node by running memsql-ops memsql-deploy until you have the same number of leaf nodes as NUMA nodes. You will need to choose a different --port for each MemSQL node on the same host.

   For example, on a host machine with two NUMA nodes, you would deploy two leaf nodes.

   memsql-ops memsql-deploy -r leaf --port 3307
   memsql-ops memsql-deploy -r leaf --port 3308
   

2. Run memsql-ops memsql-optimize and confirm the prompt. This will configure numactl to bind MemSQL nodes to NUMA nodes, and configure MemSQL memory settings. If you have fewer leaf nodes on a machine than the number of NUMA nodes, the memsql-optimize command will display a message like:

   You should deploy 1 more leaf on this machine in order to maximize performance.

   See MEMSQL-OPTIMIZE for more information.

3. Restart the cluster for the configuration changes to take effect.

   memsql-ops memsql-restart --all
   

Reconfiguring an existing cluster for NUMA

1. Follow the steps in the previous section to deploy and optimize your leaf nodes.

2. Rebalance all MemSQL databases by running REBALANCE PARTITIONS on each database:

   REBALANCE PARTITIONS ON <database>
   

Was this Article Helpful?

Thanks For Your Feedback!