Bulk Collect

discover the bulk collect (bulk api) capabilities and explore potential uses for efficiently retrieving job posting data in batches bulk collect requests' retrieved profiles limit is 10k requests that exceed the limit will be rejected overview endpoints https //docs coresignal com/api/bulk collect jobs#ml5ml information about bulk collect functionality and endpoints credits and tokens https //docs coresignal com/api/bulk collect jobs#ub5ti information about api credits and tokens authorization docid\ tktw cgpqdgjronl09vnj find details about authorization in bulk collect rate limits https //docs coresignal com/api/bulk collect jobs#jhy l information about the enabled api rate limits webhooks https //docs coresignal com/api/bulk collect jobs#rt24l find out how to use webhooks in bulk collect post endpoints response codes https //docs coresignal com/api/jobs data api response codes#y1adg find information about bulk collect error messages endpoints bulk collect is an extension of jobs data api and consists of two post and two get endpoints collect job posting data in bulk using a job posting id list or search filters request type endpoint post /v2/data requests/job base/id file post /v2/data requests/job base/filter get /v2/data requests/{data request id}/files get /v2/data requests/{data request id}/files/{file name} credits and tokens use the same token you have received for your jobs data api usage your credits for jobs data api will also apply to bulk collect data collection requests the credits are deducted after making the post request the number of credits deducted from your account depends on the data records you download for instance, collecting 100 profiles will consume 100 credits test post requests using jobs data api to see how many credits the request will consume the returned record count will show how many credits will be deducted for the bulk collect query where to find the record count? https //docs coresignal com/api/jobs data api esdsl endpoint pagination#fgjd8 what happens if i don't have enough credits? if you don't have enough credits, the request in the /v1/bulk collect/{data request id}/files endpoint will return the error 402 and the following response insufficient credits { "detail" "insufficient credits" } what happens if i submit duplicate requests? we validate your search post requests in bulk api to prevent the submission of duplicate requests if you happen to submit duplicate post requests using search filters, you will see the following error message 409 { "detail" "identical data request is already in progress " } limiting returned record count include the parameter "limit" int to control the number of records returned by your queries in /v2/data requests/job base/filter endpoint this parameter is entirely optional; however, use it to make sure you don't use too many credits with your bulk collect queries input the exact number of records you want to be returned, or delete the parameter if you don't need a set limit for your request general request template with the parameter included filters curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/job base/filter' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" {optional integer}, "filters" {} }' request example to retrieve five records all personal/company information mentioned within this context is entirely fictional and is solely intended for illustrative purposes filters curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/job base/filter' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" 5, "filters" { "created at gte" "2023 05 01 00 00 01", "created at lte" "2023 05 07 00 00 01", "company professional network url" "https //www professional network com/company/scientific company" } }' where can i see my credit balance? every search and collect response contains an x credits remaining header that shows the number of credits left in your account you can find the credit balance by clicking headers in the postman application example of remaining credits header x credits remaining 9973 rate limits bulk collect endpoints have limited requests allowed per client's api token per second request type endpoint rate limit post /v2/data requests/job base/id file 27 requests per second post /v2/data requests/job base/filter 27 requests per second get /v2/data requests/{data request id}/files 27 requests per second get /v2/data requests/{data request id}/files/{file name} 27 requests per second webhooks post endpoints allow you to add webhooks and get notified when your data request is ready request body template filter template { "data format" "json", "webhook url" "{optional webhook url}", "filters" {} } id file curl template curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/{entity}/id file' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type multipart/form data' \\ f 'ids file=@id list example csv;type=text/csv' \\ f 'data format=json' \\ f 'webhook url={optional webhook url}' how to use the api? check the endpoint usage tutorials in the following articles post requests https //docs coresignal com/api/bulk collect jobs post find examples and instructions on bulk collect post requests get requests https //docs coresignal com/api/bulk collect jobs get find examples and instructions on the bulk collect get requests