Bulk Collect

discover the bulk collect (bulk api) capabilities and explore potential uses for efficiently retrieving company data in batches bulk collect requests' retrieved profiles limit is 10k requests that exceed the limit will be rejected overview endpoints https //docs coresignal com/api/bulk collect companies#yeo3c learn more about bulk collect functionality and endpoints credits and tokens https //docs coresignal com/api/bulk collect companies#hpdei information about api credits and tokens authorization docid\ tktw cgpqdgjronl09vnj find details about authorization in bulk collect rate limits https //docs coresignal com/api/bulk collect companies#b b2u information about the enabled api rate limits webhooks https //docs coresignal com/api/bulk collect companies#bc6zg find out how to use webhooks in bulk collect post endpoints response codes docid\ sp7phdapzbmyiyggsihfw find information about bulk collect error messages endpoints bulk collect is an extension of base company api and includes three post and two get endpoints collect company data in bulk using an id list, search filters, or elasticsearch dsl schema that is already used in the company api request type endpoint post /v2/data requests/company base/id file post /v2/data requests/company base/filter post /v2/data requests/company base/es dsl get /v2/data requests/{data request id}/files get /v2/data requests/{data request id}/files/{file name} credits and tokens use the same token you have received for your company api usage your credits for company api will also apply to bulk collect data collection requests the credits are deducted after making the post request the number of credits deducted from your account depends on the data records you download for instance, collecting 100 profiles will consume 100 credits test post requests using base company api to see how many credits the request will consume the returned record count will show how many credits will be deducted for the bulk collect query where to find the record count? https //docs coresignal com/api/company api search filters pagination#zx0x9 what happens if i don't have enough credits? if you don't have enough credits, the request in the /v2/data requests/{data request id}/files endpoint will return the error 402 and the following response insufficient credits { "detail" "insufficient credits" } what happens if i submit duplicate requests? we validate your search and elasticsearch post requests in bulk collect to prevent the submission of duplicate requests if you happen to submit duplicate post requests using search or elasticsearch filters, you will see the following error message 409 { "detail" "identical data request is already in progress " } limiting returned record count include the parameter "limit" int to control the number of records returned by your queries in /v2/data requests/company base/filter and /v2/data requests/company base/es dsl endpoints this parameter is entirely optional; however, use it to make sure you don't use too many credits with your bulk collect queries input the exact number of records you want returned, or delete the parameter if you don't need a set limit for your request general request template with the included parameter filters curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/company base/filter' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" {optional integer}, "filters" {} }' es dsl curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/company base/es dsl' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" {optional integer}, "es dsl query" {} }' the following request will return exactly five records filters curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/company base/filter' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" 5, "filters" { "industry" "information technology", "created at gte" "2018 01 01 00 00 01", "last updated gte" "2022 05 26 00 00 01" } }' es dsl curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/company base/es dsl' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" 5, "es dsl query" { "query" { "bool" { "must" \[ { "query string" { "query" "(3d printing) or (3d printing service) or (lead generation)", "default field" "description", "default operator" "and" } } ] } } } }' rate limits bulk collect endpoints have limited requests allowed per client's api token per second function endpoint rate limit post /v2/data requests/company base/id file 27 requests per second post /v2/data requests/company base/filter 27 requests per second post /v2/data requests/company base/es dsl 27 requests per second get /v2/data requests/{data request id}/files 27 requests per second get /v2/data requests/{data request id}/files/{file name} 27 requests per second webhooks post endpoints allow you to add webhooks and get notified when your data request is ready request body template filter template { "data format" "json", "webhook url" "{optional webhook url}", "filters" {} } es dsl template { "data format" "json", "webhook url" "{optional webhook url}", "es dsl query" {} } id file curl template curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/company base/id file' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type multipart/form data' \\ f 'ids file=@id list example csv;type=text/csv' \\ f 'data format=json' \\ f 'webhook url={optional webhook url}' keep in mind that the webhook url parameter is optional bulk collect requests check the endpoint usage tutorials in the following articles post requests https //docs coresignal com/api/bulk collect companies post find examples and instructions for the bulk collect post requests get requests https //docs coresignal com/api/bulk collect companies get find examples and instructions for the bulk collect get requests