Nova Nodes can stream HTTP or TCP "access logs" to a syslog server on your network. This may often be a graylog cluster, etc.
Nova ADCs log in two ways, TCP and HTTP. All ADCs will automatically use TCP logging when access logging is enabled, except for SSL Termination and HTTP ADCs. These will use HTTP logging.
HTTP logging has additional information, and can have additional custom log format flags included. Naturally, it it's only available on HTTP/S traffic.
Logging can often be seen as the enemy of high performance systems. Nova Nodes can generate hundreds of thousands of log lines per second (one line per request) which can slow the performance of the Node slightly, and also can be very difficult for you (as the client) to consume.
For this reason we also support log sampling - sending only a customizable percentage of the logs. For example a sample value of 5 will send 5 in 100 (or 1 in 20) log lines to your server, selected randomly. This is good for performance analytics, but bad for security and debugging of course.
By default Nova uses two formats, once for TCP and one for HTTP. You can also customize these to include whatever information is needed.
>>> Feb 6 12:12:56 localhost \
nova[14387]: 10.0.1.2:33313 [06/Feb/2009:12:12:51.443] fnt \
bck/srv1 0/0/5007 212 -- 0/0/0/0/3 0/0
Field Format Extract from the example above
1 process_name '[' pid ']:' nova[14387]:
2 client_ip ':' client_port 10.0.1.2:33313
3 '[' accept_date ']' [06/Feb/2019:12:12:51.443]
4 frontend_name fnt
5 backend_name '/' server_name bck/srv1
6 Tw '/' Tc '/' Tt* 0/0/5007
7 bytes_read* 212
8 termination_state --
9 actconn '/' feconn '/' beconn '/' srv_conn '/' retries* 0/0/0/0/3
10 srv_queue '/' backend_queue 0/0
>>> Feb 6 12:14:14 localhost \
nova[14389]: 10.0.1.2:33317 [06/Feb/2009:12:14:14.655] http-in \
static/srv1 10/0/30/69/109 200 2750 - - ---- 1/1/1/1/0 0/0 {nova-adc.com} \
{} "GET /index.html HTTP/1.1"
Field Format Extract from the example above
1 process_name '[' pid ']:' nova[14389]:
2 client_ip ':' client_port 10.0.1.2:33317
3 '[' request_date ']' [06/Feb/2009:12:14:14.655]
4 frontend_name http-in
5 backend_name '/' server_name static/srv1
6 TR '/' Tw '/' Tc '/' Tr '/' Ta* 10/0/30/69/109
7 status_code 200
8 bytes_read* 2750
9 captured_request_cookie -
10 captured_response_cookie -
11 termination_state ----
12 actconn '/' feconn '/' beconn '/' srv_conn '/' retries* 1/1/1/1/0
13 srv_queue '/' backend_queue 0/0
14 '{' captured_request_headers* '}' {nova-adc.com}
15 '{' captured_response_headers* '}' {}
16 '"' http_request '"' "GET /index.html HTTP/1.1"
Nova ADCs support customizing what is logged (vs the defaults above). We do this using variables that you can use in your log format string.
% precedes log format variables. Variables can take arguments using braces ('{}'), and multiple arguments are separated by commas within the braces. Flags may be added or removed by prefixing them with a '+' or '-' sign.
Special variable "%o" may be used to propagate its flags to all other variables on the same format string. This is particularly handy with quoted ("Q") and escaped ("E") string formats.
Nova Default TCP Format
%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq
Nova Default HTTP Format
%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r Below are the available variables.
+---+------+-----------------------------------------------+-------------+
| R | var | field name (8.2.2 and 8.2.3 for description) | type |
+---+------+-----------------------------------------------+-------------+
| | %o | special variable, apply flags on all next var | |
+---+------+-----------------------------------------------+-------------+
| | %B | bytes_read (from server to client) | numeric |
| H | %CC | captured_request_cookie | string |
| H | %CS | captured_response_cookie | string |
| | %H | hostname | string |
| H | %HM | HTTP method (ex: POST) | string |
| H | %HP | HTTP request URI without query string (path) | string |
| H | %HQ | HTTP request URI query string (ex: ?bar=baz) | string |
| H | %HU | HTTP request URI (ex: /foo?bar=baz) | string |
| H | %HV | HTTP version (ex: HTTP/1.0) | string |
| | %ID | unique-id | string |
| | %ST | status_code | numeric |
| | %T | gmt_date_time | date |
| | %Ta | Active time of the request (from TR to end) | numeric |
| | %Tc | Tc | numeric |
| | %Td | Td = Tt - (Tq + Tw + Tc + Tr) | numeric |
| | %Tl | local_date_time | date |
| | %Th | connection handshake time (SSL, PROXY proto) | numeric |
| H | %Ti | idle time before the HTTP request | numeric |
| H | %Tq | Th + Ti + TR | numeric |
| H | %TR | time to receive the full request from 1st byte| numeric |
| H | %Tr | Tr (response time) | numeric |
| | %Ts | timestamp | numeric |
| | %Tt | Tt | numeric |
| | %Tw | Tw | numeric |
| | %U | bytes_uploaded (from client to server) | numeric |
| | %ac | actconn | numeric |
| | %b | backend_name | string |
| | %bc | beconn (backend concurrent connections) | numeric |
| | %bi | backend_source_ip (connecting address) | IP |
| | %bp | backend_source_port (connecting address) | numeric |
| | %bq | backend_queue | numeric |
| | %ci | client_ip (accepted address) | IP |
| | %cp | client_port (accepted address) | numeric |
| | %f | frontend_name | string |
| | %fc | feconn (frontend concurrent connections) | numeric |
| | %fi | frontend_ip (accepting address) | IP |
| | %fp | frontend_port (accepting address) | numeric |
| | %ft | frontend_name_transport ('~' suffix for SSL) | string |
| | %lc | frontend_log_counter | numeric |
| | %hr | captured_request_headers default style | string |
| | %hrl | captured_request_headers CLF style | string list |
| | %hs | captured_response_headers default style | string |
| | %hsl | captured_response_headers CLF style | string list |
| | %ms | accept date milliseconds (left-padded with 0) | numeric |
| | %pid | PID | numeric |
| H | %r | http_request | string |
| | %rc | retries | numeric |
| | %rt | request_counter (HTTP req or TCP session) | numeric |
| | %s | server_name | string |
| | %sc | srv_conn (server concurrent connections) | numeric |
| | %si | server_IP (target address) | IP |
| | %sp | server_port (target address) | numeric |
| | %sq | srv_queue | numeric |
| S | %sslc| ssl_ciphers (ex: AES-SHA) | string |
| S | %sslv| ssl_version (ex: TLSv1) | string |
| | %t | date_time (with millisecond resolution) | date |
| H | %tr | date_time of HTTP request | date |
| H | %trg | gmt_date_time of start of HTTP request | date |
| H | %trl | local_date_time of start of HTTP request | date |
| | %ts | termination_state | string |
| H | %tsc | termination_state with cookie status | string |
+---+------+-----------------------------------------------+-------------+
R = Restrictions : H = mode http only ; S = SSL only