Proxying HTTP Traffic to a Group of Servers
Before start using Nginx open-source or Nginx Plus to load balance HTTP traffic to a group of servers, first, we need to define the group with the upstream directive. The directive is placed in the http context.
Servers in the group are configured using the server directive. Let's see an example, the following configuration defines a group named backend and consists of three server configurations that may resolve in more than three actual servers.
http {
upstream backend {
server backend1.example.com weight=5;
server backend2.example.com;
server 192.0.0.1 backup;
}
}
To pass the requests to a server group, the group name is specified in the proxy_pass directive. In the below example, a virtual server running on Nginx passes all requests to the upstream backend group.
server {
location / {
proxy_pass http://backend;
}
}
The following example combines the two snippets above and shows how to proxy HTTP request to the backend server group. The group consists of three servers, two of the instances of the same application while the third is a backup server. Because there is no load-balancing algorithm is specified in the upstream block, Nginx uses the default algorithm, Round Robin.
http {
upstream backend {
server backend1.example.com;
server backend2.example.com;
server 192.0.0.1 backup;
}
server {
location / {
proxy_pass http://backend;
}
}
}
Choosing a Load-Balancing Method
Nginx open-source supports four methods for load-balancing, and Nginx Plus adds two more methods:
1. Round Robin: In this method, requests are distributed equally across the servers, with server weights taken into consideration. There is no directive for enabling it; this method is used by default.
upstream backend {
# no load balancing method is defined for Round Robin
server backend1.example.com;
server backend2.example.com;
}
2. Least Connections: A request is sent to the server with the least number of active connections, with server weights taken into considerations.
upstream backend {
least_conn;
server backend1.example.com;
server backend2.example.com;
}
3. IP Hash: This method is used to determine what server should be selected for the next request. In this type of case, either the first three octets of the IPv4 address or the whole IPv6 address are used to calculate the hash value.
upstream backend {
ip_hash;
server backend1.example.com;
server backend2.example.com;
}
If one of the servers needs to be temporarily removed from the load-balancing rotation, it can be added with the down parameter to preserve the current hashing of client IP addresses.
upstream backend {
server backend1.example.com;
server backend2.example.com;
server backend3.example.com down;
}
4. Generic Hash: The server to which a request is sent is determined from a user-defined key which can be a string, text, variable or a combination. For example, the key may be a paired source IP address and port, or key may be a URI like the following example:
upstream backend {
hash $request_uri consistent;
server backend1.example.com;
server backend2.example.com;
}
Nginx Plus supports two more methods:
5. Least Time: For every request, Nginx selects the server with the lowest average latency and the lowest number of active connections, where the lowest average latency is calculated based on which of the following parameters to the least_time directive is included.
header: Time to receive the 1st byte from the server.
last_byte: Time to accept the full response from the server.
last_byte inflight: Time to receive the full response from the server, taking into account incomplete requests.
upstream backend {
least_time header;
server backend1.example.com;
server backend2.example.com;
}
6. Random: In this method, each request will be passed to a randomly selected server. If the 2 parameters are specified, 1st nginx randomly selects two servers taking into account server weights, and then choose one of these servers using the specified method:
Least_conn - least number of active connections.
least_time=header - The least average time to receive the response header from the server.
Least_time=last_byte - The least average time to receive the full response from the server.
upstream backend {
random two least_time=last_byte;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;
server backend4.example.com;
}
Server Weights
By default, Nginx uses weights to distribute requests among the servers in the group through the Round Robin method. The weight parameter to the server directive is used to set the weight of a server. The by default weight is 1.
upstream backend {
server backend1.example.com weight=5;
server backend2.example.com;
server 192.0.0.1 backup;
}
In the above example, the weight of backend1.example.com is 5, and the weights of the other two servers have the default weight 1, but the one with the IP address 192.0.0.1 is marked as a backup server and does not receive requests unless both of the other servers are unavailable. In this type of configuration of weights, out of every 6 requests, 5 are sent to backend1.example.com and 1 to backend2.example.com.
Server Slow-Start
The server slow start is used to prevent a recently recovered server from being overwhelmed by connections, which may time out and cause the server to be marked as failed again.
And in Nginx Plus, slow-start allows an upstream server to gradually recover its weight from 0 to its nominal value after it has been recovered or become available. To do this, use the slow_start parameter to the server directive.
upstream backend {
server backend1.example.com slow_start=30s;
server backend2.example.com;
server 192.0.0.1 backup;
}
Enabling Session Persistence
Nginx Plus identifies user sessions and routes all requests in a given session to the same upstream server. This is called session persistence.
Nginx Plus uses three-session persistence methods. The methods are set with the sticky directive.
Sticky cookie
Nginx Plus includes a session cookie to the first response from the upstream group and identifies the server that sent the response. Next request of the client contains the cookie value and Nginx Plus route the request to the upstream server that responded to the first request.
upstream backend {
server backend1.example.com;
server backend2.example.com;
sticky cookie srv_id expires=1h domain=.example.com path=/;
}
In the above example, the srv_id parameter is used to set the name of the cookie.
expires parameter is optional which sets the time for the browser to keep the cookie.
a domain which is also an optional parameter defines the domain for which the cookie is set.
the path is also an optional parameter. It defines the path for which the cookie is set.
A sticky cookie is the simplest session persistence method.
Sticky Route
When Nginx Plus receives the first request, it assigns a route to the client. All the subsequent requests are compared to the route parameter of the server directive to identify the server to which the request is proxied. The information of the route is taken from either a cookie or the request URI.
upstream backend {
server backend1.example.com route=a;
server backend2.example.com route=b;
sticky route $route_cookie $route_uri;
}
Cookie Learn
First of all, Nginx Plus finds session identifiers by inspecting requests and responses. Then Nginx Plus "learns" which upstream server corresponds to which session identifier. Generally, these identifiers are passed in an HTTP cookie. If a request has a session identifier already "learned", Nginx Plus forwards the request to the corresponding server.
upstream backend {
server backend1.example.com;
server backend2.example.com;
sticky learn
create=$upstream_cookie_examplecookie
lookup=$cookie_examplecookie
zone=client_sessions:1m
timeout=1h;
}
