Consume Data REST API
Use the Data Integration Hub Consume Data REST API to consume, reconsume or reprocess data directly from a topic on the Data Integration Hub publication repository for data-driven subscriptions.
You can use the Consume Data API to consume data with data-driven subscriptions that consume data directly from a topic.
If the data-driven subscription process fails, you can attempt to consume the published data by reprocessing the subscription Error event.
You can reconsume data that had previously been processed by triggering the subscription Complete event with the API.
Consume Data REST API Request
To consume data through the API, copy the URL of the API from the Subscription page in Data Integration Hub.
- Request Headers
- Include the following headers in the request:
Accept - application/json
Content-Type - application/json
- To support UTF-8 character encoding, for example, to use Japanese characters in table and column names, include the following headers in the request:
Accept: application/json;charset=utf-8
Accept-Charset: charset=utf-8
Content-Type: application/json;charset=utf-8
- Request Body
- Use the following request syntax to consume data:
{
"batchSize": <value>
"requestType": <value>
"eventId": <value>
"aggregated": <value>
}
- Where <value> takes one of the following values:
- - The batchSize value can range between 1 to 5000. Default is 500.
- - The requestType option can use the following values:
- ▪ RECONSUME
- ▪ REPROCESS
- ▪ PROCESS
Default is PROCESS.
- - The eventId is mandatory only when the requestType is set to REPROCESS or RECONSUME.
- - The aggregated option can use the following values:
- ▪ true. The subscription consumes all the available publications in each API call. A separate aggregated event is created and the child event is processed.
- ▪ false. The subscription consumes only the oldest publication in each API call. A separate aggregated event is not created and the child event is processed.
Default is true. This option is not considered when the requestType is set to REPROCESS or RECONSUME .
For example:
- To reprocess a failed subscription, use the following request syntax:
{
"batchSize": <value>,
"requestType": "REPROCESS",
"eventId": <eventId>
}
For example:
{
"batchSize": 2,
"requestType" : "REPROCESS",
"eventId": 318007
}
- To reconsume data that had previously been processed, use the following request syntax:
{
"batchSize": <value>,
"requestType": "RECONSUME",
"eventId": <eventId>
}
For example:
{
"batchSize": 4,
"requestType" : "REPROCESS",
"eventId": 40559,
}
Consume Data REST API Action Response
When you use the Data Integration Hub Consume Data REST API to consume data directly from a topic, Data Integration Hub returns the response code of the action that you perform in the REST API response.
Consuming data through the REST API returns one of the following response codes:
- SUCCESS
- Data Integration Hub consumed the data successfully.
- The response includes the consumed data in the following syntax:
For example:
{
"Sales":
[
{"Opportunity_Name":"string”,"Opportunity_Owner_Id":"string"}
],
"Orders":
[
{"Account_Name":"string","Account_Id":"string","OrderId": "string"}
]
"responseStatus": "SUCCESS",
"responseExtraMessage": "",
"responseType": "application/json",
"aggregatedEventId": "1566010",
"statsSuccessRows": "4",
}
- A SUCCESS response also includes the aggregated event ID of the subscription event that Data Integration Hub generates, the number of rows successfully processed, and the number of total rows processed.
- FAILURE
- Data Integration Hub could not consume the data. For example, if is there is no pending data for the subscription to consume. The response includes a description of the error that caused the failure.
Topic Swagger Structure_sub
The Consume Data REST API returns the Swagger structure for the topic from which the subscription consumes data.
To view the Swagger structure, copy the URL of the structure from the Subscription page in Data Integration Hub.