Limiting Requests to a Pool

All servers or applications use certain amounts of system resources to process a request. These resources can include threads, file descriptors, and an amount of memory. Each back-end server might be able to process a certain number of concurrent connections before system performance is affected. Furthermore, the request/response transactions performed within a connection can also negatively affect performance through handling too many resource-intensive tasks simultaneously. In extreme cases, the server can be seen to queue, drop, or refuse new connections while the overloaded system attempts to keep up with demand.

In cases such as these, placing hard limits on the number of concurrent connections that a server node receives, or on the number of transactions a node can process concurrently, can keep your pool nodes running at an optimal level while guaranteeing that they do not get overloaded.

Connection and transaction limiting can impact other Virtual Traffic Manager features. To understand the effect of using limits, read this section fully prior to implementation.

The Virtual Traffic Manager allows the imposition of limits on the number of concurrent connections or transactions a pool node can process simultaneously. Connection-based limiting is useful for protecting back-end server nodes running applications that consume resources on a per-connection basis. The Virtual Traffic Manager can also be configured with transaction limiting to queue up incoming HTTP requests if all the nodes in the chosen pool are already busy handling previous transactions.

Connection and transaction limiting are both configured at a pool level and are applied to each node within that pool. If, for example, you set a connection limit of 50, each node in the pool receives a maximum of 50 concurrent connections. Use of the limits assumes that all of the nodes within a pool are of the same specification and are providing the same application. If the same node is present in multiple pools, the potential concurrent connections to that node, or transactions handled by the node, are the sum of the rate limits configured for each corresponding pool.

Connections to back-end server nodes are not shared amongst Virtual Traffic Managers in a cluster. In an active-passive Virtual Traffic Manager pair, all connections are used by the active Virtual Traffic Manager and none by the passive Virtual Traffic Manager. In an active-active scenario, the connection limits effectively double. For an active-active-active Virtual Traffic Manager cluster, connection limits triple, and so on.

When a node is chosen by a load balancing algorithm, the Virtual Traffic Manager checks to determine if the number of concurrent connections has yet reached the defined limit, if one is set. Equally, the Virtual Traffic Manager checks if the number of transactions being processed by the node has reached the defined transaction limit. If either case is true, the Virtual Traffic Manager begins storing subsequent requests in a queue.

When demand continually exceeds the permitted limits for a long enough period of time, there is a risk (in the case of HTTP, for example) that the response to the client can time out. In order to help mitigate this risk, the TrafficScript function connection.checkLimits() can be used to detect when a pool has started queuing requests. The Virtual Traffic Manager can use this information, for example, to respond immediately to the client with a different page rather than letting the response time out. For more details, see the Virtual Traffic Manager: TrafficScript Guide.

Request limiting is a powerful capability, and can interact with several other features of the Virtual Traffic Manager. This section describes those interactions so that you can plan your implementation accordingly.

RFC 3261 - SIP: Session Initiation Protocol, section 17.2.1, defines specific behavior in regards to what must happen when the response time of a SIP server is going to take longer than 100ms. The Virtual Traffic Manager has been developed to adhere to this part of the SIP specification.

“The server transaction MUST generate a 100 (Trying) response unless it knows that the TU will generate a provisional or final response within 200 ms, in which case it MAY generate a 100 (Trying) response.” [http://www.faqs.org/rfcs/rfc3261.html]

The Virtual Traffic Manager's request rate shaping feature uses a queue mechanism that is different from the one used by the transaction and connection limiting features. Because a limit breach might cause the Virtual Traffic Manager to queue a request after being dequeued from a rate shaping class, it is possible that the rate of requests getting sent to a node might vary from what was specified by the rate shaping class.

The response time measured by a Service Level Monitoring (SLM) class is affected when requests sent to a node are queued due to the transaction or connection limit being breached; for SLM purposes, time spent in the request queue is counted as part of the response time.

To configure the limit for the maximum number of connections or transactions processed by a node before queuing starts, use the following steps:

You can configure the following additional options that apply to the request queue:

Because connection and transaction limiting involves queuing, requests can time out. Use the Current Activity graph to track the number of requests queued and the number of request that time out while queued:

The “zeusbench” utility can be used to check whether or not the limits you have configured are working properly. The zeusbench utility can be found in $ZEUSHOME/admin/bin. Running zeusbench in concurrency mode should be able to generate enough traffic to exhaust the free connections to your back-end nodes. If (for example) you have a pool of 2 nodes and you have configured a limit of 10 connections per node, the following zeusbench command should cause connections to be queued:

zeusbench will attempt to open 100 connections to the virtual server you specify (by URL) and send as many requests down each one as fast as possible. This should result in a similar number of active connections being made to the pool under test. Keep in mind that the more nodes you have, the more concurrent connections you will need to specify when invoking the zeusbench utility.