Contents

1. Introduction
2. DNS Logging
3. Using the dns-logger
   1. System Changes
   2. Disaster Recovery & Redundancy
   3. Message Formats
   4. Standard Output Logging

Introduction

Welcome to the NoSpaceships Ltd dns-logger!

This document details features provided by the NoSpaceships Ltd dns-logger and information related to its operation.

This document is aimed at all users of the dns-logger.

DNS Logging

Almost all network transactions begin with a DNS query. This gives the DNS infrastructure insight into what services devices are wishing to connect with. This data has great value from multiple perspectives.

Operations teams can understand what services devices are accessing and how often. The data can be used to better schedule change requests, or to understand what devices are accessing a service which is to be decommissioned.

Security teams can know when devices wish to connect with domains linked to malicious activity. The same is also true for IP addresses, since DNS query responses will contain IP addresses. Response IP addresses can also give security teams’ data which can be used to pivot, e.g. a domain resolves to an IP address linked to malicious activity, the IP address can be used to identify other domains resolving the same IP address, and therefore possibly identifying other linked domains. The same is true for IP addresses, that is using the IP addresses to identify domains which resolve to them.

The dns-logger uses existing operating system packet capture features to obtain a real-time stream of DNS query request and response messages, both UDP and TCP. Messages include client and server information, detail about the question being asked, and specific response detail such as any IP addresses returned to a client.

The dns-logger maintains a memory based queue of the DNS messages and does not store data on disk, allowing it to be used on fairly busy DNS servers. Messages are streamed to one of a several configured servers in real-time, with messages being formatted using Syslog or JSON. The dns-logger automatically fails-over to another available target when one becomes unavailable.

The dns-logger has no third party dependencies, it does not require WinPCAP or npcap for example, and simply uses existing operating system provided features to capture network data.

The dns-logger aims to be the light-weight, vendor-agnostic, fast and effecient method administrators use to collect DNS activity in their environment, with fast and simple installation, and easy integraton with existing SIEM and Syslog infrastructures.

Using the dns-logger

System Changes

During installation the dns-logger will make a number of changes to the system.

On Linux platforms the following changes will be made:

• All files are copied to the installation directory, no file will be created outside of this directory
• A system service is created, if systemd is employed a dns-logger.service file is placed under the /usr/lib/systemd/system directory, otherwise a dns-logger file is placed under the /etc/init.d directory, in both cases the service is installed, enabled, and started

On Windows platforms the following changes will be made:

• All files are copied to the installation directory, no file will be created outside of this directory
• A Windows service named dns-logger will be created, this will be set to start automatically, and started
• A Windows firewall rule is added to allow the Windows service to capture network activity

For both platforms above, the steps taken during installation are reversed.

Disaster Recovery & Redundancy

Since the dns-logger will run on a DNS server it requires no disaster recovery or redundancy configuration, if a DNS server is unavailable so is the dns-logger instance for that server.

The dns-logger forwards DNS activity to one of several configured targets. It uses a failover strategy to continue sending messages to the next target in the case a target is unavailable.

For example, if the dns-logger had the targets 1.1.1.1 and 2.2.2.2 configured, and both were available, then the dns-logger on start will connect and send messages to 1.1.1.1. If that target then becomes unavailable, the connection to it will timeout and the dns-logger will then connect to 2.2.2.2. Even if 1.1.1.1 then becomes available, the dns-logger will continue to send messages to 2.2.2.2 until the dns-logger is restarted, or until 2.2.2.2 becomes unavailable. To force the dns-logger to fail-back to 1.1.1.1 the 2.2.2.2 would have to be made unavailable.

Message Formats

The dns-logger captures DNS activity from and to clients, and other DNS servers. Activity is formatted as messages, and then forwarded on to one of several configured targets. The dns-logger supports several formats for these messages. The first is a new-line delimited JSON format, and the second is a RFC 5424 Syslog format.

For all formats, currently a message is logged for each DNS query request (where the DNS header opcode is QUERY and the DNS header qr field is 0), and a message is logged for each DNS query response (where the DNS header opcode is QUERY and the DNS header qr field is 1).

JSON

The JSON message format is utilised by specifying the value json for the message-format attribute in the <install>/config/dns-logger.config file. Any changes to this file require a restart of the dns-logger service.

When this format is specified, a stringified JSON string is sent to a target. Each JSON object is terminated with a new-line character.

For DNS query request messages, the following fields are logged:

• timestamp - A string containing the date and time in ISO format of when the message was seen by the dns-logger, e.g. 2018-03-13T16:59:49.477080Z
• host - A string containing the hostname of the host which generated the message, e.g. dns1
• program - A string containing the name of the program which generated the message, this will always be dns-logger
• pid - The process ID of the dns-logger instance which generated the message, e.g. 4296
• client-ip - A string containing the dotted quad formatted IP address of the client which initiated the DNS query request, e.g. 192.168.1.16
• client-port - An unsigned integer containing the UDP or TCP port for the client which initiated the DNS query request, e.g. 55777
• server-ip - A string containing the dotted quad formatted IP address of the server to which the DNS query request was sent, e.g. 192.168.1.200
• server-port - An unsigned integer containing the UDP or TCP port for the server to which the DNS query request was sent, for most users this will always be 53
• protocol - Either the string udp or the string tcp indicating the transport protocol used for the message
• message-type - Always the string query-request to indicate the message is a DNS query request message
• message-size - An unsigned integer specifying the total size of the DNS message from the wire minus all other headers such as Ethernet, IP and TCP, i.e. it covers only the DNS message header and the DNS message body e.g. 205
• flags - An object containing the values for specific flag fields from the DNS message header, the key is the flag name as two characters and its value will be either 1 or 0, currently only the rd (Recursion Desired) flag is included for DNS query request messages, e.g. {"rd":1}
• question-name - A string containing the DNS question the DNS query message relates too, this field will encode specific characters using so called percent encoding, whereby a character is replaced with the % character followed by two hexadecimal characters specifying the value of the replaced character in hexadecimal, the characters replaced are mostly non-printable characters and characters that would interfere with the message format, e.g. the following, while not realistic, begins with the / character and demonstrates the encoding: %2fwww.nospaceships.com
• question-type - A string containing the DNS query type in its string form, e.g. the string A, when a DNS query type is unknown the string TYPEX will be used where X is the DNS query type code as an integer, e.g. for A this would be TYPE1
• question-class - A string containing the DNS query class in its string form, e.g. the string IN, when a DNS query class is unknown the string CLASSX will be used where X is the DNS query class code as an integer, e.g. for IN this would be CLASS1
• section-counts - An array containing three unsigned integers, the first specifies the total answer DNS resource records in the DNS query request, the second specifies the total number of authority DNS resource records in the DNS query request, and the third specifies the total number of additional DNS resource records in the DNS query request, e.g. [0,0,1]

The following is an example DNS query request (new-lines have been added for clarity, normally this message would be formatted on a single line followed by a single new-line character):

{
	"timestamp": "2018-03-13T18:04:10.4907087Z",
	"host": "t20",
	"program": "dns-logger",
	"pid": 9676,
	"client-ip": "192.168.1.199",
	"client-port": 55298,
	"server-ip": "192.168.1.200",
	"server-port": 53,
	"protocol": "udp",
	"message-type": "query-request",
	"message-size": 27,
	"flags": {
		"rd": 1
	},
	"question-name": "bbc.co.uk",
	"question-type": "A",
	"question-class": "IN",
	"section-counts": [
		0,
		0,
		0
	]
}

For DNS query response messages, the following fields are logged:

• timestamp - A string containing the date and time in ISO format of when the message was seen by the dns-logger, e.g. 2018-03-13T16:59:49.477080Z
• host - A string containing the hostname of the host which generated the message, e.g. dns1
• program - A string containing the name of the program which generated the message, this will always be dns-logger
• pid - The process ID of the dns-logger instance which generated the message, e.g. 4296
• client-ip - A string containing the dotted quad formatted IP address of the client the DNS query response was sent to, e.g. 192.168.1.16
• client-port - An unsigned integer containing the UDP or TCP port for the client the DNS query response was sent to, e.g. 55777
• server-ip - A string containing the dotted quad formatted IP address of the server from which the DNS query response was sent, e.g. 192.168.1.200
• server-port - An unsigned integer containing the UDP or TCP port for the server from which the DNS query response was sent, for most users this will always be 53
• protocol - Either the string udp or the string tcp indicating the transport protocol used for the message
• message-type - Always the string query-response to indicate the message is a DNS query response message
• message-size - An unsigned integer specifying the total size of the DNS message from the wire minus all other headers such as Ethernet, IP and TCP, i.e. it covers only the DNS message header and the DNS message body e.g. 451
• flags - An object containing the values for specific flag fields from the DNS message header, the key is the flag name as two characters and its value will be either 1 or 0, currently the aa (Authoritative Answer), tc (Truncated Response), rd (Recursion Desired) and ra (Recursion Available) flags are included for DNS query response messages, e.g. {"aa":1,"tc":0,"rd":1,"rd":1}
• question-name - A string containing the DNS question the DNS query response relates too, this field will encode specific characters using so called percent encoding, whereby a character is replaced with the % character followed by two hexadecimal characters specifying the value of the replaced character in hexadecimal, the characters replaced are mostly non-printable characters and characters that would interfere with the message format, e.g. the following, while not realistic, begins with the / character and demonstrates the encoding - %2fwww.nospaceships.com
• question-type - A string containing the DNS query type in its string form, e.g. the string A, when a DNS query type is unknown the string TYPEX will be used where X is the DNS query type code as an integer, e.g. for A this would be TYPE1
• question-class - A string containing the DNS query class in its string form, e.g. the string IN, when a DNS query class is unknown the string CLASSX will be used where X is the DNS query class code as an integer, e.g. for IN this would be CLASS1
• section-counts - An array containing three unsigned integers, the first specifies the total answer DNS resource records in the DNS query response, the second specifies the total number of authority DNS resource records in the DNS query response, and the third specifies the total number of additional DNS resource records in the DNS query response, e.g. [3,2,2]
• response-code - A string containing the DNS query response code in its string form, e.g. the string NXDOMAIN, when a DNS query response code is unknown the string RCODEX will be used where X is the DNS query response code as an integer, e.g. for NXDOMAIN this would be RCODE3
• answer-ips - An array of strings, each string is the dotted quad formatted IP address from DNS resource records in the answer section in the DNS query response if any are found, e.g. ["1.2.3.4","5.6.7.7"], this can be useful from a security perspective since it exposes IP addresses DNS clients may then go on and connect to

The following is an example DNS query response (new-lines have been added for clarity, normally this message would be formatted on a single line followed by a single new-line character):

{
	"timestamp": "2018-03-13T18:04:10.5555385Z",
	"host": "t20",
	"program": "dns-logger",
	"pid": 9676,
	"client-ip": "192.168.1.199",
	"client-port": 55298,
	"server-ip": "192.168.1.200",
	"server-port": 53,
	"protocol": "udp",
	"message-type": "query-response",
	"message-size": 259,
	"response-code": "NOERROR",
	"flags": {
		"aa": 0,
		"tc": 0,
		"rd": 1,
		"ra": 1
	},
	"question-name": "bbc.co.uk",
	"question-type": "A",
	"question-class": "IN",
	"section-counts": [
		4,
		4,
		4
	],
	"answer-ips": [
		"151.101.64.81",
		"151.101.0.81",
		"151.101.192.81",
		"151.101.128.81"
	]
}

Syslog

The Syslog message format is utilised by specifying the value syslog for the message-format attribute in the <install>/config/dns-logger.config file. Any changes to this file require a restart of the dns-logger service.

When this format is specified, Syslog messages similar to the ISC BIND querylog format will be generated. Each Syslog message is terminated with a new-line character. There are a few differences between the ISC BIND querylog format and the format generated by the dns-logger:

• Only the recursion desired flag (the + or - that can appear after the DNS query type in the message) and the TCP flag (the T which can also appear after the DNS query type in the message) are logged, no other flags are logged
• The total size of the DNS message header and the DNS message body is logged following the server IP address appearing in brackets at the end of the message
• The number of DNS resource records in each of the answer, authority and additional sections is logged after the message size
• DNS query responses are also logged, these are similar messages to the DNS query request messages with some extra fields

DNS query request messages will look like the following:

<135>1 2018-03-13T18:19:40.6749336Z t20 dns-logger 14940 - - client 192.168.1.199#58105: query: bbc.co.uk IN A + (192.168.1.200) 27bytes 0 0 0

This message is broken down as follows:

<priority>1 <timestamp> <host> <program> <pid> - - client <client-ip>#<client-port>: query: <question-name> <question-class> <question-type> [+-][T] (<server-ip>) <message-size>bytes <section-counts>

These fields are defined as follows:

• priority - The syslog priority, the syslog-facility and syslog-severity attributes defined in the <install>/config/dns-logger.config file are used to construct the value for this field
• timestamp - A string containing the date and time in ISO format when the message was seen by the dns-logger, e.g. 2018-03-13T16:59:49.477080Z
• host - A string containing the hostname of the host which generated the message, e.g. dns1
• program - A string containing the name of the program which generated the message, this will always be dns-logger
• pid - The process ID of the dns-logger instance which generated the message, e.g. 4296
• client-ip - A string containing the dotted quad formatted IP address of the client which initiated the DNS query request, e.g. 192.168.1.16
• client-port - An unsigned integer containing the UDP or TCP port for the client which initiated the DNS query request, e.g. 55777
• question-name - A string containing the DNS question the DNS query request relates too, this field will encode specific characters using so called percent encoding, whereby a character is replaced with the % character followed by two hexadecimal characters specifying the value of the replaced character in hexadecimal, the characters replaced are mostly non-printable characters and characters that would interfere with the message format, e.g. the following, while not realistic, begins with the / character and demonstrates the encoding - %2fwww.nospaceships.com
• question-class - A string containing the DNS query class in its string form, e.g. the string IN, when a DNS query class is unknown the string CLASSX will be used where X is the DNS query class code as an integer, e.g. for IN this would be CLASS1
• question-type - A string containing the DNS query type in its string form, e.g. the string A, when a DNS query type is unknown the string TYPEX will be used where X is the DNS query type code as an integer, e.g. for A this would be TYPE1
• [+-] - Either the character - or the character + indicating whether the DNS client requested recursion or not
• protocol - If TCP was utilised to perform the query the character T will appear directory following the previous field, i.e. there will be no white-space between them
• server-ip - A string containing the dotted quad formatted IP address of the server to which the DNS query request was sent, e.g. 192.168.1.200
• message-size - An unsigned integer specifying the total size of the DNS message from the wire minus all other headers such as Ethernet, IP and TCP, i.e. it covers on the DNS message header and the DNS message body e.g. 205
• section-counts - Three unsigned integers which are space separated, the first specifies the total answer DNS resource records in the DNS query request, the second specifies the total number of authority DNS resource records in the DNS query request, and the third specifies the total number of additional DNS resource records in the DNS query request, e.g. 0 0 1

DNS query response messages will look like the following:

<135>1 2018-03-13T18:19:40.7193243Z t20 dns-logger 14940 - - client 192.168.1.199#58105: response: bbc.co.uk IN A + (192.168.1.200) 259bytes 4 4 4 NOERROR 151.101.192.81 151.101.128.81 151.101.64.81 151.101.0.81

This message is broken down as follows:

<priority>1 <timestamp> <host> <program> <pid> - - client <client-ip>#<client-port>: response: <question-name> <question-class> <question-type> [+-][T] (<server-ip>) <message-size>bytes <section-counts> <response-code> <answer-ips>

These fields are defined as follows:

• priority - The syslog priority, the syslog-facility and syslog-severity attributes defined in the <install>/config/dns-logger.config file are used to construct the value for this field
• timestamp - A string containing the date and time in ISO format when the message was seen by the dns-logger, e.g. 2018-03-13T16:59:49.477080Z
• host - A string containing the hostname of the host which generated the message, e.g. dns1
• program - A string containing the name of the program which generated the message, this will always be dns-logger
• pid - The process ID of the dns-logger instance which generated the message, e.g. 4296
• client-ip - A string containing the dotted quad formatted IP address of the client to which the DNS query response was sent, e.g. 192.168.1.16
• client-port - An unsigned integer containing the UDP or TCP port for the client to which the DNS query response was sent, e.g. 55777
• question-name - A string containing the DNS question the DNS query response relates too, this field will encode specific characters using so called percent encoding, whereby a character is replaced with the % character followed by two hexadecimal characters specifying the value of the replaced character in hexadecimal, the characters replaced are mostly non-printable characters and characters that would interfere with the message format, e.g. the following, while not realistic, begins with the / character and demonstrates the encoding - %2fwww.nospaceships.com
• question-class - A string containing the DNS query class in its string form, e.g. the string IN, when a DNS query class is unknown the string CLASSX will be used where X is the DNS query class code as an integer, e.g. for IN this would be CLASS1
• question-type - A string containing the DNS query type in its string form, e.g. the string A, when a DNS query type is unknown the string TYPEX will be used where X is the DNS query type code as an integer, e.g. for A this would be TYPE1
• [+-] - Either the character - or the character + indicating whether the DNS client requested recursion or not
• protocol - If TCP was utilised to perform the query the character T will appear directory following the previous field, i.e. there will be no white-space between them
• server-ip - A string containing the dotted quad formatted IP address of the server from which the DNS query response was sent, e.g. 192.168.1.200
• message-size - An unsigned integer specifying the total size of the DNS message from the wire minus all other headers such as Ethernet, IP and TCP, i.e. it covers on the DNS message header and the DNS message body e.g. 205
• section-counts - Three unsigned integers which are space separated, the first specifies the total answer DNS resource records in the DNS query response, the second specifies the total number of authority DNS resource records in the DNS query response, and the third specifies the total number of additional DNS resource records in the DNS query response, e.g. 0 0 1
• response-code - A string containing the DNS query response code in its string form, e.g. the string NXDOMAIN, when a DNS query response code is unknown the string RCODEX will be used where X is the DNS query response code as an integer, e.g. for NXDOMAIN this would be RCODE3
• answer-ips - A space separated list of dotted quad formatted IP address from each DNS resource records in the answer section in the DNS query response if any are found, e.g. 1.2.3.4 5.6.7.7, this can be useful from a security perspective since it exposes IP addresses DNS clients may then go on and connect to

Standard Output Logging

The dns-logger program that provides the dns-logger background service can be passed a parameter to instruct it to print the messages it would normally send to a configured target to standard output instead. This feature can be used to verify an installation is working.

To have the dns-logger print messages to standard output specify the --run-stdout command line parameter:

Linux

cd <install>
./dns-logger.bin --run-stdout config/dns-logger.config

Windows

cd <install>
dns-logger.exe --run-stdout config\dns-logger.config

NOTE The dns-logger service does NOT need to be stopped to utilise this feature, and if started can be left running.

Simply issue a CTRL+C to stop the dns-logger.

The above example uses the same configuration file used by the dns-logger background service, therefore messages will be formatted using the same configuration. An alternative configuration can be specified, which can contain different configuration.

NOTE This method could also be used for ad-hoc DNS activity capture when troubleshooting specific issues.