Bulk Collect
API changes: Limitations in Bulk Collect request Impacted endpoints: All Bulk Collect endpoints Impact: A number of retrieved profiles cannot exceed 10k Release date: April 2nd, 2025
We will start limiting the number of profiles that can be retrieved with a single Bulk Collect request.
The request will be rejected, and a specific error will appear when any of the following scenarios are triggered:
Response code | Scenario | Error message |
|---|---|---|
400 | ID list (depending on the endpoint) exceeds 10k IDs (profiles) | File contains too many IDs. Maximum is 10,000 IDs |
422 | Results that match Bulk Collect filter request exceed 10k IDs (profiles) | Request matched too many IDs. Maximum is 10,000 IDs |
422 | Results that match Bulk Collect ES DSL request exceed 10k IDs (profiles) | Request matched too many IDs. Maximum is 10,000 IDs |
For the specific scenario listed above, the behaviour of getting the error message differs.
- If your ID list request to data_request endpoint exceeds 10k IDs, the request will be rejected. You will not be able to create a request that exceeds the limit.
- If the results that match Bulk Collect filter or ES DSL exceed 10k IDs, the request will be rejected in another way. Here, you will be able to create a request, and when the results exceed the limit while collecting the data, the request will be rejected. Several ways to know about this situation:
- While trying to collect the results
- A webhook about the failure will be sent if a webhook_url is given