Log File Format

Infoblox Platform logs are delivered in parquet format. You can use the Apache parquet tools that Infoblox provides (click here to access the tools) to convert the parquet files to JSON format if necessary. For more information, see Converting Parquet to JSON.

Infoblox Platform supports three log file types: DNS queries and responses, RPZ hits, and IPAM metadata.

When synchronizing data with your S3 bucket the first time, Infoblox Platform automatically creates a directory structure in the following format for each day.

For DNS Response data, the following directory structure is required:

Amazon S3 > <bucketname> / dns_enriched / year=xxxx / month=xx / day=xx

Example:

Amazon S3 > testbucket / dns_enriched/ year=2022 / month=10 / day=31

Each folder in the directory structure contains all the log files that you have selected for export. For example, if you have selected DNS Response Logs and RPZ logs, you will see all the dns_ and rpz_ files in the same folder for a specific date.

The following sections describe the file format and data schema for each log type.

Response Logs

File format: part-00000-0228fa58-6334-464c-8502-37f04dd40528.c000.snappy.parquet

Schema:

Resource Record

DNS

Answers count

Sample DNS response log:

java -jar parquet-tools-1.8.2-SNAPSHOT.jar cat -j dns_00000001525858200000_017.parquet

{“opcode”:0,”timestamp”:1525857674,”qname”:”eicar.co.”,”qtype”:1,”qclass”:1,”source”:
“3b9eea03015cee5cca1bcb22b02c837c”,”qip”:”54.152.30.60″,”qport”:47″3b9eea03015cee5cca1bcb22b02c837c”,”qip”:”54.152.30.60″,”qport”:47697,”rip”:”
“,”rport”:-1,”protocol”:17,”delay”:1.0,”rcode”:3,”type”:1,”qqr”:false,”qaa”:false,”qtc”:false,”qrd”:false,”qra”:false,
“qad”:false,”qcd”:false,”qdo”:false,”rqr”:true,”raa”:false,”rtc”:false,”rrd”:true,”rra”:false,”rad”:true,”rcd”:false,
“rdo”:false,”rrr1″:{},”rrr2″:{},”rrr3″:{},”view”:””,”anonymized”:false,”nanosec”:220087857,”pid”:”120873″,
“cid”:”:c4f0717dbd1150904aab042e1843a91f”,”tid”:””}

RPZ Logs

For RPZ  data, the following directory structure is required:

Amazon S3 > <bucketname> /archivers/ rpz_enriched / year=xxxx / month=xx / day=xx /hour=xx

Example:

Amazon S3 > testbucket/rpz_enriched /year=2024/month=3/day=3/hour=15/ 

File format: part-00000-0228fa58-6334-464c-8502-37f04dd40528.c000.snappy.parquet

RPZ

Sample RPZ log:

java -jar parquet-tools-1.10.1-SNAPSHOT.jar cat -j rpz_0000000000000000061_030.parquet

{“opcode”:-1,”timestamp”:1521522768,”nanosec”:0,”tcode”:0,”tname”:”eicar.co.base.rpz.infoblox.local”,”acode”:3,”arrtype”:-1,”arrdata”:””,”qname”:”eicar.co”,”qtype”:1,”qclass”:-1,”source”:””,”qip”:”10.120.20.247″,”qport”:39826,”rip”:”10.35.205.4″,”rport”:-1,”view”:”_default”,”pvendor”:”Infoblox”,”pname”:”NIOS”,”pversion”:”8.2.0-357775″,”loglevel”:7,”disabled”:false,”tid”:””,”pid”:””,”cid”:””,”anonymized”:false,”cmac”:””,”csite”:””,”qcat”:””,”tinfo”:””}

IPAM Metadata Logs

File format: ipmeta__Update_1521240816_0000000000_01965.parquet

Schema:

Session Record

IP Meta

Sample IPAM Metadata log:

java -jar parquet-tools-1.10.1-SNAPSHOT.jar cat -j ipmeta_Update_1521240816_0000000000_01965.parquet

{“opcode”:2,”source”:”10.35.205.4″,”timestamp”:1521535044,”nanosec”:0,”cip”:”10.35.205.4″,”hostnames”:{},”usernames”:{“array”:[{“name”:”frtest”,”actfrom”:1521238758,”actto”:1521534479}]},”mac”:””,”view”:”default”,”fingerprint”:””,”os”:””,”firstts”:-1,”lastts”:-1,”extattrs”:{},”anonymized”:false}