ERROR 2350: Leaf node could not be added because you are using the SingleStore DB free license which has a limit of 4 license units and after adding the node you would be using XX license units.

ERROR 2349: Code generation for new statements is disabled because the total number of license units of capacity used on all leaf nodes is XX, which is above the limit of 4 for the SingleStore DB free license.

Issue

Both errors 2350 and 2349 are caused because the total number of license units in your cluster is larger than the four unit limit imposed by the free license. You will encounter these errors when trying to add a new leaf node to the cluster, which can happen during initial deployment, or when expanding your cluster.

This error can occur when using MemSQL 6.7.13 or later with a unit-based free license. For previous RAM-based free licenses, a total memory limit of 128 GB of RAM for the cluster was used instead. See Memory Errors for more information on this limit.

See the FAQ for a definition of license units and how they are calculated.

Solutions

There are two potential solutions depending on what your needs are:

• If you need a cluster with more than four license units, you will need an Enterprise license. Sign up for a free 30-day trial and create an Enterprise License trial key before deploying a larger cluster.

• You can artificially constrain resources by deploying SingleStore DB onto a host machine (physical or virtual) with fewer resources. See the FAQ link above for more details.

WARN: The ready queue has not decreased (currently elements, pops) for seconds. This workload needs more threads.

Issue

Each query that runs on a cluster (including internal queries used by the nodes to communicate with each other) requires exactly one thread on an aggregator. This warning appears when the max_connection_threads limit is reached on an aggregator and the aggregator stops responding. The following is a brief description of each warning parameter:

• elements refers to the number of queued queries (queries which are waiting for execution threads)
• pops refers to the cumulative number of queries that have been scheduled. If the value of pops remains same, it means that the scheduler is not making progress because queries are still running.
• This workload needs more threads refers to the kernel threads dedicated to SingleStore DB, specifically to the value of max_connection_threads.

Solution

• Increase the value of max_connection_threads on aggregator nodes (maximum 8192). The default value for aggregators is 192. Increasing the value of max_connection_threads on aggregator nodes will allow more queries to run at the same time; however, this can increase resource pressure on the cluster.

  Note: The value on leaf nodes does not need to be changed from the default of 8192, which is the maximum value.

• The warning may also occur due to execution of BACKUP. If BACKUP is executed when there are long-running queries executing in the cluster, the backup operation will have to wait for the long-running queries to complete. During this time new queries executed on the cluster will also have to wait for the long-running query to finish. If this warning appears in close proximity to BACKUP entries in the master aggregator’s memsql.log, check to make sure backups are not being executed when long-running queries are present in the cluster.

Monitor Thread Utilization

Use the SHOW STATUS EXTENDED command to check the current number of running threads (threads_running) and number of connections in queue currently waiting (ready_queue). If a workload runs enough long-running concurrent queries that reaches the max_connection_threads threshold, and the queries are sufficiently long-running so that none have finished for over few seconds, the aggregator’s memsql.log file will have the WARN: This workload needs more threads. entry.

Was this Article Helpful?

Thanks For Your Feedback!