TIDE Data Submission Overview

TIDE data can be uploaded to a profile associated with an account. Data policies are used to control access to your organization's data and can be specified when the data is submitted. Data policies allow your organization to control how its submitted data is shared with other organizations or groups.

Data profiles are used to identify data in the Infoblox Portal from one or many data submissions and control who can access the data. A data profile must be specified when data is submitted and is associated with a policy, which controls who can access the data.

Users can submit threat indicators by using the Infoblox Portal or the TIDE Data API. To submit data, the following is required:

1. A data profile: Data profiles are associated with policies, which control who has access to your organization’s data, and they must be specified when data is submitted. When a data profile is created, it must be associated with a policy. Subscribers of Infoblox Threat Defense Advanced can create custom profiles that allow the use of custom-generated TIDE RPZ feeds that are based on an organization's requirements. When creating your own TIDE BYOF RPZ feeds for use with NIOS, keep in mind the practical limits of your NIOS system and its ability to ingest large amounts of data contained within only one RPZ feed. For details, see Creating Data Profiles.
2. Choose a file: Supported file formats include XML, JSON and comma/tab/pipe-separated values. Data files must follow the BloxOne Threat Defense API Guidelines for file and record-level fields. Records must contain a recognized threat class or property. For information, see Submitting TIDE Data.

Users can submit data as JSON, CSV, XML, and TSV (tab-separated values). For all data formats, the submitted data must identify the data/record type in addition to the list of data records. For CSV and TSV, the record type must be provided as one of the columns. For JSON and XML, the record type is defined in a separate top-level field. The record type field can have one of the following values: “host”, “ip”, or “url”. It is not possible to upload data using different profiles or different record types in the same file. Threat data consists of file-level fields and record-level fields. The following table contains descriptions of all available fields:

Data Submission Formats

Note

Any unknown fields in a record will automatically go under an “extended” field for that record. This will occur after the submission is done.

Threat Data Fields
File-level fields
profile	The data profile ID.
record_type	host, ip, url, email, or hash.
external_id	A string indicating an external ID to assign to the batch (optional).
record	Surrounds the individual record(s) in the XML and JSON formats.
Record-level fields
Field Name	Description
host	The threat’s hostname.
ip	The threat’s IP address.
url	The threat’s URL.
hash	The hash threat.
email	The email threat.
detected	The date/time the threat was detected, in the ISO 8601 format. For example, “20231114T124742Z” and “2024-01-12T00:00:00.000Z” are valid ISO 8601 formats.
class	The threat's class. Example: Sinkhole. Note: Either "class" or "property" is required, but not both.
property	The threat's property. Example: Sinkhole_SinkholedHost. Either "class" or "property" is required, but not both.
confidence	The threat's confidence score, from 0 to 100 (optional).
domain	The domain string (optional).
duration	The duration of the threat, in the Xd or XyXmXwXdXh format. The expiration date will be set to the detected date + this duration (optional).
expiration	The expiration is the datetime the threat will be expired. The expiration datetime is in the same format as that of the detected field.
threat_level	The threat's level. This is an integer from 0 to 100 (optional).
target	The threat’s target (optional). For example: “fakeamazon.com” is a threat targeting “amazon.com”.
tld	The top-level domain; a string (optional).

The following is an example of data submitted in the XML format:

<feed>
   <profile>SampleProfile</profile>
   <record_type>ip</record_type>
   <record>
     <ip>127.1.0.1</ip>
    <property>Phishing_Phish</property>
    <detected>20170602T154742Z</detected>
   </record>
   <record>
  <ip>8.8.8.8</ip>
   <property>Scanner_Generic</property>
   <detected>19980927T154242Z</detected>
   <duration>42y0m0w0d42h</duration>
   </record>
  </feed>

sample_xml_file.xml (XML format)

The following is an example of data submitted in the CSV format:

record_type,url,profile,detected,property
url,https://example.com/page1.html,SampleProfile,20170602T154742Z,UnwantedContent_Parasite
url,http://example.com/gift.html,SampleProfile,20170602T154742Z,Scam_FakeGiftCard

sample_csv_file.csv (CSV format)

The following is an example of data submitted in the JSON format:

  {
    "feed": {
      "profile": "SampleProfile",
      "record_type": "host",
      "record": [
        {"host": "www.google.com", "property": "Scanner_Generic",
          "detected": "19980927T154242Z", "duration":"42y0m0w0d42h"},                  
        {"host": "www.example.com", "property": "Phishing_Phish",
          "detected": "20170602T154742Z"}
        ]
      }
  }

samp le_json_file.json (JSON format)

The recommended limit for the number of records in a data submission is 50,000. The maximum number of records should not exceed 60,000 at this point in time.

The following is an example of data submitted in the JSON format and with additional parameter fields:

{ "feed": {
    "profile": "SampleProfile", 
    "record_type": "host",
    "record": [ 
        { 
            "host": "test-domain.org", 
                        "domain": "test-domain.org", 
                        "class": "CompromisedHost", 
                         "property" : "CompromisedHost_Generic", 
                         "detected": "2024-01-12T00:00:00.000Z",
                         "duration": "90d", 
                         "confidence": 70, 
                         "threat_level": 80 
                }, 
                { 
                         "host": "test-domain.net", 
                         "class": "Sinkhole", 
                         "property" : "Sinkhole_SinkholedHost", 
                         "detected": "2024-01-12T00:00:00.000Z", 
                         "expiration": "2024-03-01T00:00:00.000Z", 
                         "confidence": 30, 
                         "threat_level": 50 
                 }] 
     } 
}

Tide-Data.json (JSON format)

For additional information, see the following: