Bulk Collect

employee api will be deprecated on august 6th until then, you must transfer to a base employee api docid\ gobtmn jebqrtgr8akb1w read all about the benefits and changes base employee api docid\ zhyxkcw5ssxx1p94ejuwl discover the bulk api (bulk collect) capabilities and explore potential uses for efficiently retrieving employee data in bulk bulk collect requests' retrieved profiles limit is 10k requests that exceed the limit will be rejected overview endpoints https //docs coresignal com/api/bulk collect employees#b p48 learn more about bulk collect functionality and endpoints credits and tokens https //docs coresignal com/api/bulk collect employees#ei743 information about bulk collect functionality and endpoints authorization docid\ tktw cgpqdgjronl09vnj find details about authorization in bulk collect rate limits https //docs coresignal com/api/bulk collect employees#xap7x information about the enabled api rate limits response codes https //docs coresignal com/api/employee api response codes#vts6i find information about bulk collect error messages webhooks https //docs coresignal com/api/bulk collect employees#gkkok find out how to use webhooks in bulk collect post endpoints endpoints bulk api (bulk collect) is an extension of employee api and includes three post and two get endpoints collect employee data in bulk using an id list, search filters, or elasticsearch dsl schema that is already used in the employee api request type endpoint post /v2/data requests/member/id file post /v2/data requests/member/filter post /v2/data requests/member/es dsl get /v2/data requests/{data request id}/files get /v2/data requests/{data request id}/files/{file name} credits and tokens use the same token you have received for your employee api usage your credits for employee api will also apply to bulk collect data collection requests the credits are deducted after making the post request the number of credits deducted from your account depends on the data records you download for instance, collecting 100 profiles will consume 100 credits test post requests using employee api to see how many credits the request will consume the returned record count will show how many credits will be deducted for the bulk collect query where to find the record count? https //docs coresignal com/api/employee api search filters pagination#ywe u what happens if i don't have enough credits? if you don't have enough credits, the request in the /v2/data requests/{data request id}/files endpoint will return the error 402 and the following response insufficient credits { "detail" "insufficient credits" } what happens if i submit duplicate requests? we validate your search and elasticsearch post requests in bulk collect to prevent the submission of duplicate requests if you happen to submit duplicate post requests using search or elasticsearch filters, you will see the following error message 409 { "detail" "identical data request is already in progress " } limiting returned record count include the parameter "limit" int to control the number of records returned by your queries in /v2/data requests/member/filter and /v2/data requests/member/es dsl endpoints this parameter is entirely optional; however, use it to make sure you don't use too many credits with your bulk collect queries input the exact number of records you want to be returned or delete the parameter if you don't need a set limit for your request general request template with the parameter included filters curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/member/filter' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" {optional integer}, "filters" {} }' es dsl curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/member/es dsl' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" {optional integer}, "es dsl query" {} }' request example to retrieve five records filters curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/member/filter' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" 5, "filters" { "title" "cso", "location" "united states", "industry" "information technology" } }' es dsl curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/member/es dsl' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" 5, "es dsl query" { "query" { "bool" { "must" \[ { "match" { "title" { "query" "developer", "operator" "and" } } }, { "match" { "location" { "query" "united states", "operator" "and" } } }, { "nested" { "path" "member experience collection", "query" { "bool" { "must" \[ { "query string" { "query" "it company", "default field" "member experience collection company name", "default operator" "and" } } ] } } } } ] } } } }' rate limits bulk collect endpoints have limited requests allowed per client's api token per second function endpoint rate limit post /v2/data requests/member/id file 27 requests per second post /v2/data requests/member/filter 27 requests per second post /v2/data requests/member/es dsl 27 requests per second get /v2/data requests/{data request id}/files 27 requests per second get /v2/data requests/{data request id}/files/{file name} 27 requests per second webhooks post endpoints allow you to add webhooks and get notified when your data request is ready request body template filter template { "data format" "json", "webhook url" "{optional webhook url}", "filters" {} } es dsl template { "data format" "json", "webhook url" "{optional webhook url}", "es dsl query" {} } id file curl template curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/member/id file' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type multipart/form data' \\ f 'ids file=@id list example csv;type=text/csv' \\ f 'data format=json' \\ f 'webhook url={optional webhook url}' keep in mind that webhook url is optional bulk collect requests check the endpoint usage tutorials in the following articles post requests https //docs coresignal com/api/bulk collect employees post find examples and instructions for the bulk collect post requests get requests https //docs coresignal com/api/bulk collect employees get find examples and instructions for the bulk collect get requests