TIDE bulk export service endpoints
The TIDE bulk data export API allows for the accessing of active threats using the TIDE API and running the TIDE bulk data export API.
- To access active threats, use tide/api/data/threats/state/ and specify a provider organization using the "profile" query string parameter.
- The TIDE bulk data export API requires an endpoint to fetch the bulk threat data and allows specifying the “rlimit” query string parameter for limiting returned records. Note: The rlimit is set to a maximum of 100 responses.
- The authorization for this process is via the gateway, and the expected response is 200 OK with a file location provided.
Request:
GET /tide/bulk-export/threats?type=<type>&rlimit=<limit> |
Headers:
AuthContext
Note: This authorization is via the gateway.
Expected response:
200 OK
{ |
The file will be internally uploaded to an S3 bucket specifically dedicated to the client, using the file-id as the object name.
Sample request:
GET 'https://csp.infoblox.com/tide/api/data/bulk-export/threats?rlimit=10&type=host' |
Sample response:
"data_stream_location": "https://csp.infoblox.com/tide/api/data/bulk-export/data-stream?file_id=ba06742e-8006-4171-89b0-29a641dc04f0" |
Note: Run the call again, this time using only the file id (ba06742e-8006-4171-89b0-29a641dc04f0)
""file_uploaded": true, |
Note: When the generated url is clicked, the file can be directly updated
Expected Response Errors
- 401 Unauthorized: This error occurs when the tide-ng-atk-gateway is unable to validate the user's token, indicating an authentication issue.
- 400 Bad Request: Encountered if the request lacks a valid type or presents incorrect parameters, suggesting a client-side input error.
- 500 Internal Server Error: Triggered by issues such as data filtering errors or problems with S3 IO operations, pointing to server-side complications.
Exception Handling
- When attempting to retrieve materialized files, an IOException occurs if the specified key is not found on S3. This error is communicated to the user as a 500 internal server error.
- The process of uploading generated files containing threat data to S3 involves multipart upload sessions. These sessions either conclude successfully or fail entirely; there are no partial successes.
- Should the client's S3 bucket be nonexistent, one will be created accordingly.
Endpoint to access the threat data from the uploaded file
Sample request:
GET 'https://csp.infoblox.com/tide/bulk-export/file-access/<file-id≥ |
Response from tide
{ { |
{ } |
Note: Implementing Endpoint Handlers in Tide-ng-atk-gateway and Managing Redirection for File Access.
Expected Response Errors
401 Unauthorized: Occurs when the tide-ng-atk-gateway is unable to verify user authentication.
400 Bad Request: Triggered if the request lacks a valid fileID.
404 Not Found: Indicates that the file does not exist.
- Either the file has not been uploaded yet, retry after 20 seconds. The message will specify this.
- Or, re initiate the bulk upload.
403 Forbidden: This error is returned if the file does not belong to the client (note: this may also qualify as a 400 Bad Request).
500 Internal Server Error: Arises in cases where there's an error generating the pre-signed URL.