Adding ETL Configurations

To create data filters for your source data, do the following:

1. Log in to the Cloud Services Portal.

2. Click Manage > Data Connector.

3. Select the ETL Configuration tab, and click Create.

4. From the Create drop-down list, select one of the following filtering criteria for the ETL configuration: Regex, IP/Network, FQDN, NIOS HOST, IP/Network, FQDN DNS Record Type, OPHID, and ON-PREM HOST.

5. Depending on the criterion you selected, specify the following information in the Create ETL Filter wizard, and then click Save & Close:

• Name: Enter a name for the ETL configuration. Create a name that best describes the filtering function.
• Description: Enter a description for the ETL configuration. The field’s length is 256 characters.
• State: Use the slider to enable or disable the ETL configuration. The ETL configuration is in effect only after you enable it; if you disable it, the ETL filter will not be in effect even if you have applied the ETL configuration to a traffic flow configuration.

6. Expand the Regex, IP/Network, FQDN, NIOS HOST, IP/Network, FQDN DNS Record Type, OPHID, or ON-PREM HOST section, and click Add to add the applicable parameters:

• Regex: The regex filter applies to DNS query/response events and RPZ events. You can specify any regular expressions for the member name. You can also specify the name of the Grid member that processed the query. 

  The regex filter for the RPZ flow works with IP addresses, not with hostnames. For all other workflows, the filter works with hostnames.

• IP/Network: This filter applies to DNS query/response events, IP metadata, and RPZ events. If the event is a query, specify the query source’s IP address; if the event is a response, specify the destination’s IP address. Specify the client_ip filter in the following format:

CIDR block: Example: 10.10.0.1/15, 2001:cdba:9abc:5678::/64, etc.

• FQDN: The FQDN filter applies to DNS query/response events and RPZ events. A query filter is a combination of valid FQDNs and wildcards. Note the following about wildcards:
• You can specify a wildcard either on the left or right side of the FQDN.
• A rule can have zero, one, or two wildcards.
• If a rule has two wildcards, they have to be on the opposite ends of the FQDN.
• With the exception of the “?” wildcard, a wildcard on the left side of the FQDN must be followed by a dot.
• With the exception of the “?” wildcard, a wildcard on the right side of the FQDN must be preceded by a dot.

The following wildcards are supported:

• DNS Record Type: This filter can be applied on DNS query/response events and RPZ events. These records provide important details about domains and hostnames. The following are some of the DNS Record Type filters:
  • A Record
  • AAAA Record
  • CAA Record
  • CNAME Record
  • MX Record
  • NAPTR Record
  • NS Record
  • PTR Record
  • SRV Record
  • TXT Record
• OPHID: This is a unique identifier for the on-prem host. The user can use this value or provide a custom-defined OPHID. The following are some of the OPHID filters:
  • e7d97bd6548y8bbasd766e3f8f3789jrob6
  • 4c168ec9ca885fa5d9ccca0d8dfe793f
  • cdc-filter-test
• ON-PREM HOST: This is a display name of the on-prem host. The following are some of the ON-PREM HOST filters:
  • iccrvr01.indu.test-example.com
  • ZTP_atlasautomation_8722411532980096350
  • APIKEY1
  • Inblox Test OnPrem

For the complete list of supported filters, see Data Connector ETL Data Filter Types.

For more information on ETL configurations, see the following:

• Configuring ETL Filters
• Viewing ETL Configurations
• Removing ETL Configurations