TIDE Data Submission Overview

TIDE data can be uploaded to a profile associated with an account. Policies are used to control access to your organization's data and can be specified when the data is submitted. Data Policies allow organizations to control how their submitted data is shared with other organizations or groups. Infoblox can enable accessing and data sharing between organizations upon request. Policies can be used for multiple data submissions and are only visible within your organization. Data profiles are used to identify data in the platform from one or many data submissions. A data profile must be specified when data is submitted. Data profiles are associated with policies, which control who can access the data. When a data profile is created it must be associated with a policy.

A dedicated service key for each data output is the recommended best practice.

Users can submit threat indicators using the Cloud Services Portal or via the TIDE Data API. In order to submit data, the following is required:

1. A policy: Defines how data is shared. For information on security policies, see Configuring Security Policies.
2. A data profile: Data profiles are associated with policies, which control who has access to your organization’s data, and must be specified when data is submitted. When a data profile is created it must be associated with a policy. Subscribers of BloxOne Threat Defense Advanced can create custom profiles that allow the use of custom-generated TIDE RPZ feeds that are based on an organization's requirements. When creating your own TIDE BYOF RPZ feeds for use with NIOS, keep in mind the practical limits of your NIOS system and its ability to ingest large amounts of data contained within only one RPZ feed. For details, see Creating Data Profiles.

Users can submit data using the following formats: JSON, CSV, XML, and TSV (tab separated values). For all data formats, the submitted data must identify the data/record type in addition to the list of data records. For CSV and TSV, the record type must be provided as one of the columns. For JSON and XML, the record type is defined in a separate top-level field. The record type field can be one of the following values: host, ip, or url. It is not possible to upload data using different profiles or different record types in the same file. Threat data consists of file-level fields and record-level fields. The following table contains descriptions of all available fields:

Data Submission Formats

Threat Data Fields
Field Name	Description
host	threat hostname
ip	threat IP address
url	threat URL
hash	hash threat
email	email threat
detected	date/time threat was detected, in ISO 8601 format
class	the threat's class, for example: Sinkhole
property	the threat's property, for example: Sinkhole_SinkholedHost
confidence	the threat's confidence score ranging from 0 - 100 (optional)
domain	domain string (optional)
duration	duration of the threat in XyXmXwXdXh format - the expiration date will be set to the detected date + this duration (optional)
expiration	expiration date, in ISO 8601 format (optional)
threat_level	the threat's level, ranging from 0 - 100 (optional)
target	target of threat (optional)
tld	top-level domain, string (optional)

The following listing contains a sample data submission in XML format:

<feed>
   <profile>SampleProfile</profile>
   <record_type>ip</record_type>
   <record>
     <ip>127.1.0.1</ip>
    <property>Phishing_Phish</property>
    <detected>20170602T154742Z</detected>
   </record>
   <record>
  <ip>8.8.8.8</ip>
   <property>Scanner_Generic</property>
   <detected>19980927T154242Z</detected>
   <duration>42y0m0w0d42h</duration>
   </record>
  </feed>

sample_xml_file.xml (XML format)

The following listing contains a sample data submission in CSV format:

record_type,url,profile,detected,property
url,"https://example.com/page1.html","SampleProfile","20170602T154742Z",
"UnwantedContent_Parasite"
url,"http://example.com/gift.html","SampleProfile","20170602T154742Z", "Scam_FakeGiftCard"

sample_csv_file.csv (CSV format)

The following listing contains a sample data submission in JSON format:

  {
    "feed": {
      "profile": "SampleProfile",
      "record_type": "host",
      "record": [
        {"host": "www.google.com", "property": "Scanner_Generic",
          "detected": "19980927T154242Z", "duration":"42y0m0w0d42h"},                  
        {"host": "www.example.com", "property": "Phishing_Phish",
          "detected": "20170602T154742Z"}
        ]
      }
  }

samp le_json_file.json (JSON format)

The recommended limit for the number of records in a given data submission is 50,000. The maximum number of records should be no more than 60,000 at this point in time.

The following listing contains a sample data submission in JSON format (with additional parameter fields):

{ "feed": {
    "profile": "abc_test", 
    "record_type": "host",
    "record": [ 
        { 
            "host": "test-domain.org", 
                        "domain": "test-domain.org", 
                        "class": "CompromisedHost", 
                         "property" : "CompromisedHost_Generic", 
                         "detected": "2024-01-12T00:00:00.000Z",
                         "duration": "90d", 
                         "confidence": 70, 
                         "threat_level": 80 
                }, 
                { 
                         "host": "test-domain.net", 
                         "class": "Sinkhole", 
                         "property" : "Sinkhole_SinkholedHost", 
                         "detected": "2024-01-12T00:00:00.000Z", 
                         "expiration": "2024-03-01T00:00:00.000Z", 
                         "confidence": 30, 
                         "threat_level": 50 
                 }] 
     } 
}

Tide-Data.json (JSON format)

For additional information, see the following: