You can use the dnstap log format to log DNS queries and responses at high rates to well-known destinations. NIOS logs all valid DNS queries and responses that are not dropped by Advanced DNS Protection. For information about dnstap, see https://dnstap.info/.

For information about capturing DNS queries and responses without using dnstap, see Capturing DNS Queries and Responses.

About dnstap Implementation

dnstap is implemented using the Google protocol buffer (protobuf) for packaging logged data, and Frame Streams as a lightweight streaming protocol to transmit the data to a receiver.

A bi-directional communication starts with a 3-way handshake sequence of Frame Streams control packets over a TCP (or other reliable byte-streaming). The following figure illustrates the connection between the transmitter and receiver pair.

Packet Flow for Frame Streams 3-way handshake initialization and data frames over a TCP connection

The transmitter initiates the communications to the receiver by first sending a READY control frame carrying the protobuf:dnstap.Dnstap string. The receiver responds with an ACCEPT control frame carrying the same protobuf:dnstap.Dnstap string. The handshake is completed when the transmitter sends a START control frame carrying the same protobuf:dnstap.Dnstap string.

Frame Streams Control Frame Format

Data packets are carried over the connection with only a 4-byte framing overhead as illustrated in the following figure.

Frame Streams Data Frame Format

To end the bi-directional communication that is shown in the following figure, the sequence starts with a STOP control frame from one of the communicating pairs, followed by a FINISH acknowledgment control frame from the other half of the pair.

Packet flow for Frame Streams termination over a TCP connection

Protobuf Template for Infoblox dnstap

The following is a sample template that you can create for flexible and efficient use of your logged data. You can specify the way in which information must be logged by defining protobuf message types.

// dnstap: flexible, structured event replication format for DNS software,
// This file contains the protobuf schemas for the "dnstap" structured event
// replication format for DNS software.
// Written in 2013-2014 by Farsight Security, Inc.
// Updated in 2019-2020 by Infoblox, Inc.
// To the extent possible under law, the author(s) have dedicated all copyright
// and related and neighboring rights to this file to the public domain worldwide.
// This file is distributed without any warranty.
// You should have received a copy of the CC0 Public Domain Dedication along with this file.
// If not, see:
// <http://creativecommons.org/publicdomain/zero/1.0/>. package dnstap;
// "Dnstap": this is the top-level dnstap type, which is a "union" type that
// contains other kinds of dnstap payloads, although currently only one type
// of dnstap payload is defined.
// See: https://developers.google.com/protocol-buffers/docs/techniques#union
message Dnstap {
// DNS server identity.
optional bytes identity = 1;
// DNS server version.
optional bytes version = 2;
// Extra data for this payload.
optional bytes extra = 3;
// Identifies which field below is filled in.
enum Type {
MESSAGE = 1;
}
required Type type = 15;
// One of the following will be filled in.
optional Message message = 14;
}
// SocketFamily: the network protocol family of a socket. This specifies how
// to interpret "network address" fields.
enum SocketFamily {
INET = 1; // IPv4 (RFC 791)
INET6 = 2; // IPv6 (RFC 2460)
}
// SocketProtocol: the transport protocol of a socket. This specifies how to
// interpret "transport port" fields.
enum SocketProtocol {
UDP = 1; // User Datagram Protocol (RFC 768)
TCP = 2; // Transmission Control Protocol (RFC 793)
}
// Message: a wire-format (RFC 1035 section 4) DNS message and associated
// metadata. Applications generating "Message" payloads should follow
// certain requirements based on the MessageType, see below.
message Message {
// We are supporting the following types of messages
// CQ: CLIENT_QUERY
// CR: CLIENT_RESPONSE
enum Type {
// CLIENT_QUERY is a DNS query message sent from a client to a DNS
// server which is expected to perform further recursion, from the
// perspective of the DNS server. The client may be a stub resolver or
// forwarder or some other type of software which typically sets the RD
// (recursion desired) bit when querying the DNS server. The DNS server
// may be a simple forwarding proxy or it may be a full recursive
// resolver.
CLIENT_QUERY = 5;
// CLIENT_RESPONSE is a DNS response message sent from a DNS server to
// a client, from the perspective of the DNS server. The DNS server
// typically sets the RA (recursion available) bit when responding.
CLIENT_RESPONSE = 6;
}
// One of the Type values described above.
required Type type = 1;
// One of the SocketFamily values described above.
optional SocketFamily socket_family = 2;
// One of the SocketProtocol values described above.
optional SocketProtocol socket_protocol = 3;
// The network address of the message initiator.
// For SocketFamily INET, this field is 4 octets (IPv4 address).
// For SocketFamily INET6, this field is 16 octets (IPv6 address).
optional bytes query_address = 4;
// The network address of the message responder.
// For SocketFamily INET, this field is 4 octets (IPv4 address).
// For SocketFamily INET6, this field is 16 octets (IPv6 address).
optional bytes response_address = 5;
// The transport port of the message initiator.
// This is a 16-bit UDP or TCP port number, depending on SocketProtocol.
optional uint32 query_port = 6;
// The transport port of the message responder.
// This is a 16-bit UDP or TCP port number, depending on SocketProtocol.
optional uint32 response_port = 7;
// The time at which the DNS query message was sent or received, depending
// on whether this is an AUTH_QUERY, RESOLVER_QUERY, or CLIENT_QUERY.
// This is the number of seconds since the UNIX epoch.
optional uint64 query_time_sec = 8;
// The time at which the DNS query message was sent or received.
// This is the seconds fraction, expressed as a count of nanoseconds.
optional fixed32 query_time_nsec = 9;
// The initiator's original wire-format DNS query message, verbatim.
optional bytes query_message = 10;
// This is a wire-format DNS domain name.
// Currently, we are not supporting this.
optional bytes query_zone = 11;
// The time at which the DNS response message was sent or received,
// depending on whether this is an CLIENT_RESPONSE.
// This is the number of seconds since the UNIX epoch.
optional uint64 response_time_sec = 12;
// The time at which the DNS response message was sent or received.
// This is the seconds fraction, expressed as a count of nanoseconds.
optional fixed32 response_time_nsec = 13;
// The responder's original wire-format DNS response message, verbatim.
optional bytes response_message = 14;
// Start of Infoblox specific log messages
optional bytes subscriber_id = 15; // 32 bytes
optional uint64 local_id = 16; // 8 bytes
optional bytes pcp_ssp = 17; // 16 bytes
optional bytes proxy_all = 18; // Proxy all
optional bytes fqdn = 19; // MAX 255 bytes
required uint32 txid = 20; //16-bit transaction id
}
// All fields except for 'type' in the Message schema are optional.
// It is recommended that at least the following fields be filled in for
// particular types of Messages.
// CLIENT_QUERY:
// socket_family, socket_protocol
// query_message
// query_time_sec, query_time_nsec
// CLIENT_RESPONSE:
// socket_family, socket_protocol
// query_time_sec, query_time_nsec
// response_message
// response_time_sec, response_time_nsec

Configuring dnstap to Log DNS Queries and Responses

You can use the dnstap log format to log DNS queries and responses at high rates to well-known destinations and achieve high performance query  and response logging. NIOS logs all valid DNS queries and responses that are not dropped by Advanced DNS Protection. For information about dnstap, see https://dnstap.info/.

To use the dnstap log format, you need to enable dnstap by running the set enable_dnstap command. To view whether the dnstap log format is enabled or disabled, run the show dnstap-status command. To view the number of queries and responses sent to the destination when the dnstap log format is enabled for high-performance logging of queries and responses, run the show dnstap-stats command.

If you choose to enable the dnstap log format, you will not be able to capture queries and responses using the Data connector for all DNS Queries/Responses to a Domain fields. And if you use the Data connector for all DNS Queries/Responses to a Domain fields for query capture, the DNSTAP settings for DNS Queries/Responses fields are disabled.

Limitations of Using dnstap to Log Queries and Responses

Ensure that you understand the following limitations before you use dnstap to log queries and responses:

• dnstap supports UDP, TCP, and EDNS protocols that require additional processing thus leading to a decrease in performance.
• NIOS does not support BIND9 dnstap.
• If the remote logging server is not accessible, then the logs are dropped and not buffered.
• The dnstap server cannot truncate EDNS0 queries.
• If you run a query that contains +edns=1, a dnstap server that uses the Golang DNS library to process the captured data displays it as a bad signature (TSIG signature failure).
• Capturing the queries and responses also depends on other factors such as the size of the flavor deployed and features enabled over it.
• dnstap does not support query and response logging on the MGMT interface.
• For long-running queries, using dnstap may cause some response packets to be dropped.

Configuring dnstap to Log DNS Queries and Response Captures

To configure dnstap to log DNS queries and to capture responses, complete the following:

1. Grid: From the Data Management tab, select the DNS tab, expand the Toolbar, and then click Grid DNS Properties.
   Member: From the Data Management tab, select the DNS tab, and then click the Members tab -> member checkbox -> Edit icon.
2. In the Grid DNS Properties or Member DNS Properties editor, click Toggle Advanced Mode and then select the Logging tab.
3. Select the Queries checkbox to start capturing DNS queries. When you enable this option at the member level, NIOS captures DNS queries for only the selected member.
4. Select the Responses checkbox to start capturing DNS responses. When you enable this option at the member level, NIOS captures DNS responses for only the selected member.
5. In the DNSTAP Receiver Address field, enter the IP address from which you want to capture queries or responses. It supports both IPv4 and IPv6 addresses.
6. In the DNSTAP Receiver Port field, enter the port number on which you want to configure the dnstap client system. The default port number is 6000. 
7. Click Save and Close.

Infoblox recommends the configurations in the following table to meet high-performance query logging using the dnstap log format: