...
Full synchronization synchronizes the entire set of objects irrespective of updates. You can perform a full synchronization only on the Grid Master and Grid Master Candidate. Note that the tasks created on the Grid Master and Grid Master Candidates will be executed on them respectively and you can download the synchronized data on to the member where the task was created. The Grid Master Candidate can execute only read-only tasks. To save the output of a full synchronization, you must specify the output location. The output of a full synchronization is in the following formats: JSON, XML, ROWJSON, or ROWXML.
You can specify one of the following output locations: "FILE_DISTRIBUTION" and "LOCAL".
- When
_output_location = "FILE_DISTRIBUTION"
When you set the output location as mentioned above, you must specify a file name or a prefix for the file. NIOS saves the output file in the file distribution area and displays an error message if you do not specify a file name or a prefix. For more information about file distribution area, see Managing Directories.
When you select the above output location for a Grid Master Candidate, you must select Allow Upload to Grid Members to upload files to the file distribution area. Infoblox recommends that you use the Grid Master Candidate to offload the Grid Master. You must enable a full synchronization during off-peak hours if protocol services are running on the Grid Master Candidate. For more information, see Enabling Upload to Grid Members.
Example:
curl -k1 -u admin:infoblox -X POST 'https://..../wapi/v2.5/fileop?_function=read' -H 'Content-Type: application/json; charset=UTF-8' -d '{"_filename": "test.json", "_output_location":"FILE_DISTRIBUTION","_object": "db_objects", "all_object_types_supported_in_version": "2.5","_encoding": "ROWJSON"}'
You can specify either _object = db_objects or all_object_types_supported_in_version=2.5 to retrieve all objects. When you specify _object = db_objects, NIOS returns only standard Restful API fields as response. To retrieve all fields of all objects, you can specify all_object_types_supported_in_version=2.5. To fetch the output file, start file distribution service and use the following URL:
curl -k1 -u admin:infoblox -X GET http://<ipaddress>/wapi_output/test.json -H 'Content-Type:application/json'
- When
_output_location = "LOCAL"- If you schedule a full synchronization and set the output location as mentioned above, a file name is generated based on the task ID and the output file is saved at another location. You do not have to specify a file name or a prefix.
- If you do not schedule a synchronization, then NIOS returns an URL and a token, which is used for the download complete function, in the response.
- If you do not specify an output location, NIOS saves the file in the file distribution area.
- Use the above output location for saving full synchronization files when file distribution area does not have enough space for storage.
Example:
curl -k1 -u admin:infoblox -X POST 'https://127.0.0.1/wapi/v2.5/fileop?_function=read&_schedinfo.schedule_now=true' -H 'Content-Type: application/json; charset=UTF-8' -d '{"_output_location": "LOCAL", "_object": "db_objects", "all_object_types_supported_in_version": "2.5", "_encoding": "JSON"}'
NIOS saves the output file in the file distribution area if you do not specify an output location as shown in the code below:
curl -k1 -u admin:infoblox -X POST 'https://..../wapi/v2.5/fileop?_function=read' -H 'Content-Type: application/json; charset=UTF-8' -d '{"_filename": "test.json", "_output_location":"FILE_DISTRIBUTION","_object": "db_objects", "all_object_types_supported_in_version": "2.5","_encoding": "ROWJSON"}'
Note the following about full synchronization:
- A full synchronization is required initially for a complete snapshot of the database.
- Depending on the size of requested data, a full synchronization may take a longer time to complete. For example, when 1 million objects are requested with all fields on an IB-4010 appliance, full synchronization takes a couple of hours. When 1 million objects are requested with standard RESTful API fields on an IB-4010 appliance, full synchronization takes less than an hour.
- Infoblox recommends that you request only object types with standard RESTful API fields and specify any additional required fields during a full synchronization.
- To dump full synchronization updates into a file using
fileop->readoperation when the updated objects are large in number, you can specify_object=db_objectsand all the necessary parameters. - Infoblox recommends a full synchronization in the following cases:
- After an upgrade, restore or master promotion, which resets the sequence ID.
...
Following are a few samples of API requests for full synchronization:
- If you specify
_output_location = "local"and schedule the task, then you must usefileop ->
...
>get_file_urlto retrieve the URL:
curl -k1 -u admin:infoblox -X POST 'https://127.0.0.1/wapi/v2.5/fileop?_function=get_file_url' -H "Content-Type: application/json" -d '{"task_id":1}'
The response is as follows:
"url": "https://10.32.2.202/http_direct_file_io/req_id-OBJECTS-1/nios-db-objects-1.json"
You can fetch this file using the following API request:
curl -k1 -u admin:infoblox -X GET https://10.32.2.202/http_direct_file_io/req_id-OBJECTS-1/nios-db-objects-1.JSON -H 'Content-Type: application/json; charset=UTF-8'
- If you specify
_output_location = "local"and do not schedule the task, then NIOS returns an URL and a token:
curl -k1 -u admin:infoblox -X POST 'https://127.0.0.1/wapi/v2.5/fileop?_function=read' -H 'Content-Type: application/json; charset=UTF-8' -d '{ "_output_location":"LOCAL","_object": "db_objects", "all_object_types_supported_in_version": "2.5","_encoding": "JSON"}'
The response is as follows:
{ "token":
"eJy1kU1vwyAMhu/8kfaSD/JB0t5aZZU2Ta3UTtrRSoB2nlJggUztv5+ZtJ123QFk/L7mMUZK6+4w\n6QujTVr jwzTLYCfmOFtKNGc7jPaWWqPjCnenPev60MNRn5krmAQYZhwDGgCmUAbmSrZUrmKnhb45\nnO4Q8KoXzNVsxyt RtKVYFU0qqqpuBWf+tJinkWRBBW8hOL/OMp6nZZ2KtG2ymAKF1FuAM44a0GaT\n/gBUSXd43T8fNl3C85xnBq1 P1AB2eCevT5J8leR1UuRcQFGsy3pN1KfTYU+sJmJRUdQS9a/rSFpF\nk6KnUsxz8mWe5tJfdBau7n/64vyHCdp Iq9BcYrYg+Pbx21D+Gq5WxanyOOhu87KB48Munmvmw9Fx\nET+BNyTi0DtA4+YAn3ryaE20tWzvh/QLT4Gbhw= =\n", "url": "https://10.35.6.87/http_direct_file_io/req_id-DOWNLOAD-1001/nios-db_objects--09-05-20
16_22:35:27.JSON" }
- To save the file in a local area instead of the file distribution area:
curl -k1 -u admin:infoblox -X POST 'https://127.0.0.1/wapi/v2.5/fileop?_function=read&_schedinfo.schedule_now=true' -H 'Content-Type: application/json; charset=UTF-8' -d '{"_output_location": "LOCAL", "_object": "db_objects", "all_object_types_supported_in_version": "2.5", "_encoding": "JSON"}'
The response is as follows:
scheduledtask/b25lLnF1ZXVlZF90YXNrJDE:1/WAITING_EXECUTION
- You can also schedule a full synchronization task as follows:
curl -k1 -u admin:infoblox -X POST 'https://127.0.0.1/wapi/v2.5/fileop?_function=read&_schedinfo.schedule_now=true' -H 'Content-Type: application/json; charset=UTF-8' -d '{"_filename": "test2.txt", "_object": "db_objects", "all_object_types_supported_in_version": "2.5", "_encoding": "JSON"}' scheduledtask/b25lLnF1ZXVlZF90YXNrJDE:1/WAITING_EXECUTION
NIOS returns the scheduled task reference so that you can query and find out when will the scheduled task be complete:
...
...
curl -k1 -u admin:infoblox -X GET
https://127.0.0.1/wapi/v2.5/scheduledtask/b25lLnF1ZXVlZF90YXNrJDE:1/WAITING_EXECUTION
The response is as follows:
{ "_ref": "scheduledtask/b25lLnF1ZXVlZF90YXNrJDE:1/COMPLETED", "approval_status": "NONE", "execution_status": "COMPLETED", "task_id": 1 }
Note that the result is in either JSON or XML format. You can fetch the output file from the wapi-output
...
directory of the file distribution area when the scheduled task is complete.
| Anchor | ||||
|---|---|---|---|---|
|
| Anchor | ||||
|---|---|---|---|---|
|
Incremental synchronization synchronizes only those objects that are updated since the previous synchronization. With incremental synchronization, you can query NIOS for any updates and when updates are found, NIOS returns the changed objects since the previous incremental synchronization. Incremental synchronization synchronizes the NIOS database for object changes such as addition, updates, or deletion, mostly via scripts. The incremental synchronization query consists of object types, a cookie of the form
<id of db:sequence_id>, a sequence ID and an optional exclude_deleted object to indicate if the deleted objects must be excluded from the results. NIOS includes the deleted records in the results by default.
...
Note: Infoblox recommends that you use fileop->read for incremental synchronization when the updated objects are large in number.
...
Note the following about incremental synchronization:
- You cannot request an incremental synchronization to get updated (created, updated or deleted) objects after a full synchronization.
- For performance reasons, NIOS returns the objects that changed and not the object data that changed. Infoblox recommends that you figure out what changed by comparing the object with the previous version in the client database.
- Incremental synchronization may take a longer time to complete depending on the number of changes. For example, if 50K objects are requested with all fields on an IB-4010 appliance, an incremental synchronization approximately takes 2 minutes. When 50K objects are requested with standard RESTful API fields on an IB-4010, the synchronization process completes in approximately 1 minute.
- The size of the memory increases when the number of objects increases. Infoblox recommends that you use
_max_resultsto limit the number of results or use the asynchronousfileop->readoperation. - Infoblox recommends that for a busy Grid with a high Rate of Change that is greater than 500/sec, you must run an incremental synchronization more frequently. Example: If the expected Rate of Change for objects in the Grid is 50/sec, you can schedule an incremental synchronization every 5-10 minutes. If the Rate of Change for objects is 200/sec, schedule an incremental synchronization every minute or every 2 minutes.
- Infoblox recommends that you include all object types when you query using an incremental synchronization, so that it returns the highest sequence ID for all objects that can be used for subsequent incremental updates. If you query only selected object types such as A record and zone, NIOS returns the highest sequence ID for those object types only.
Consider an example where A record, Network, and A record are the object types with sequence ID 4, 5 and 6 respectively. When you query for an A record object type with a previous sequence ID using incremental synchronization, NIOS returns A record with the highest sequence ID, which is 6 and does not return the network object. Meanwhile, if you add or update a network object, then in the subsequent incremental synchronization it does not return the network object and you will lose updates made to the network object. Hence, Infoblox recommends that you include all object types in the query to ensure that you do not lose updates made to other object types. - You cannot perform a full or an incremental synchronization during Grid Master Candidate promotions, restores, and upgrades.
- The output of an incremental synchronization is either in JSON or XML format.
The following are a few samples of API requests for incremental synchronization:
...
1426NIOS Administrator Guide (Rev. A)NIOS 8.1 Tracking Object Changes in the Database
- To get updates with all_objects_supported_in_version=2.5:
curl -k1 -u admin:infoblox -H content-type:application/json -X GET
https://10.35.6.87/wapi/v2.5/db_objects?start_sequence_id=992684967:0\&all_object_type
s_supported_in_version=2.5\;return_fields=last_sequence_id,object,object_type,unique_id\;_return_type=json
The response for the above API request contains all fields of all object types.
- To get an A record and an auth zone object update:
...
curl -k1 -u admin:infoblox
...
-H
...
content-type:application/jason
...
-X
...
GET
...
https://10.32.2.202/wapi/v2.5/db_objects?start_sequence_id=183566281:0\&object_types=r
...
ecord:a,zone_auth\;_return_fields=last_sequence_id,object,object_type,unique_id\;_
...
return_type=json
...
The
...
response
...
is
...
as
...
follows:
...
[
...
{"_ref":
...
"db_objects/Li5hbGxfY2hhbmdlZF9vYmplY3RzJDA:19",
...
"object":
...
{"_ref":
...
"zone_auth/ZG5zLnpvbmUkLl9kZWZhdWx0LmNvbS5mb28:foo.com/default",
...
"fqdn":
...
"foo.com",
...
"view":
...
"default"
...
},
...
"object_type":
...
"zone_auth",
...
"unique_id":
...
"087af628fa03418faa0577b953807efc"
...
},
...
{"_ref":
...
"db_objects/Li5hbGxfY2hhbmdlZF9vYmplY3RzJDE:21",
...
"object":
...
{"_ref":
...
"record:a/ZG5zLmJpbmRfYSQuX2RlZmF1bHQuY29tLmZvbyxhcmVjMSwxLjIuMy4y:arec1.foo.com/
...
default",
...
"ipv4addr":
...
"1.2.3.2",
...
"name":
...
"arec1.foo.com",
...
"view":
...
"default"
...
},
...
"object_type":
...
"record:a",
...
"unique_id":
...
"89314c3fae2841f49600e1b686b0c7b7"
...
},
...
{"_ref":
...
"db_objects/Li5hbGxfY2hhbmdlZF9vYmplY3RzJDI:183566281%3A21",
...
"last_sequence_id":
...
"183566281:21"
...
},
- NIOS returns deleted objects as synthetic deleted objects. To get updates on deleted host objects:
...
curl -s -k1
...
-u
...
admin:infoblox
...
-H
...
content-type:application/json
...
-X
...
GET
...
https://10.34.19.220/wapi/v2.5/db_objects?start_sequence_id=1207501969:0\&object_types
...
=record:host,bulkhost,record:host_ipv4addr,record:host_ipv6addr\&exclude_deleted=false
...
The
...
response
...
is
...
as
...
follows:
...
[
...
{
...
"_ref":
...
"db_objects/Li5hbGxfY2hhbmdlZF9vYmplY3RzJDM2YQ:130",
...
"object":
...
{
...
"_ref":
...
"deleted_objects/Li5kZWxldGVkX29iamVjdHMkMA",
...
"object_type":
...
"record:host_ipv6addr"
...
},
...
"object_type":
...
"deleted_objects",
...
"unique_id":
...
"f7d87ec4b9204e16a3d14bfb340c402d"
...
},
...
{
...
"_ref":
...
"db_objects/Li5hbGxfY2hhbmdlZF9vYmplY3RzJDM2Yg:130",
...
"object":
...
{
...
"_ref":
...
"deleted_objects/Li5kZWxldGVkX29iamVjdHMkMQ",
...
"object_type":
...
"record:host_ipv4addr"
...
},
...
"object_type":
...
"deleted_objects",
...
"unique_id":
...
"68316853600f41f083e2be134628ce62"
...
},
...
{
...
"_ref":
...
"db_objects/Li5hbGxfY2hhbmdlZF9vYmplY3RzJDM2Yw:130",
...
"object":
...
{ "_ref":
...
"deleted_objects/Li5kZWxldGVkX29iamVjdHMkMg",
...
"object_type":
...
"record:host"
...
},
...
"object_type":
...
"deleted_objects",
...
"unique_id":
...
"7a5e0622e9db441caa47c12ca30700d5"
...
},
...
{
...
"_ref":
...
"db_objects/Li5hbGxfY2hhbmdlZF9vYmplY3RzJDM2ZA:1207501969%3A130",
...
"last_sequence_id":
...
"1207501969:130"
...
}
...
]
...
- To dump incremental synchronization updates into a file using
fileop->readoperation when the updated objects are large in number, you can specify_object=db_objectsand all the necessary parameters for incremental synchronization:
curl -k1 -u admin:infoblox -X POST 'https://10.34.19.100/wapi/v2.5/fileop?_function=read' -H 'Content-Type: application/json; charset=UTF-8' -d '{"_output_location": "LOCAL", "_object": "db_objects", "start_sequence_id" : "1973553522:0","all_object_types_supported_in_version": "2.5", "_max_results": 1000000,"_encoding": "JSON"}'
For more information about supported objects in the Infoblox API and Restful API, refer to the Infoblox API Documentation and the Infoblox WAPI Documentation.
1428NIOS Administrator Guide (Rev. A)NIOS 8.1