Bulk Collect

when creating a bulk collect data request based on es dsl query to the new base employee endpoints, we strongly advise including the following clause to your query "term" { "is parent" 1 } this ensures that you get the most out of our new systematic approach to exclude duplicate records, which we can resolve with the same profile bulk collect requests' retrieved profiles limit is 10k requests that exceed the limit will be rejected overview bulk collect /#endpoints information about bulk collect functionality and endpoints bulk collect /#credits and tokens information about bulk collect functionality and endpoints authorization docid\ tktw cgpqdgjronl09vnj find details about authorization in bulk collect bulk collect /#rate limits information about the enabled api rate limits response codes docid\ eegbe5xqblfigvmcnwsfb find information about bulk collect error messages webhook subscriptions docid\ ut xggm ibmtusznmsqsc find out how to use webhooks in bulk collect post endpoints endpoints bulk api (bulk collect) is an extension of base employee api and includes three post and two get endpoints collect employee data in bulk using an employee id list, search filters, or elasticsearch dsl schema that is already used in the base employee api request type endpoint post /v2/data requests/employee base/ids post /v2/data requests/employee base/es dsl post /v2/data requests/employee base/filter post /v2/data requests/employee base/shorthand names post /v2/data requests/employee base/urls get /v2/data requests/{data request id}/files get /v2/data requests/{data request id}/files/{file name} credits and tokens use the same token you have received for your base employee api usage your credits for base employee api will also apply to bulk collect data collection requests the credits are deducted after making the post request the number of credits deducted from your account depends on the data records you download for instance, collecting 100 profiles will consume 100 credits test post requests using base employee api to see how many credits the request will consume the returned record count will show how many credits will be deducted for the bulk collect query limiting returned record count include the parameter "limit" int to control the number of records returned by your queries in /v1/ professional network /employee base/bulk collect/filter and /v1/ professional network /employee base/bulk collect/es dsl endpoints this parameter is entirely optional; however, use it to make sure you don't use too many credits with your bulk collect queries input the exact number of records you want returned, or delete the parameter if you don't need a set limit for your request general request template with the parameter included filters curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/employee base/filter' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" {optional integer}, "filters" {} }' es dsl curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/employee base/es dsl' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "https //server com/api/webhook", "limit" {optional integer}, "es dsl query" {} }' request example to retrieve five records filters curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/employee base/filter' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" 5, "filters" { "title" "cso", "location" "united states", "industry" "information technology" } }' es dsl curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/employee base/es dsl' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "data format" "json", "webhook url" "{optional webhook url}", "limit" 5, "es dsl query" { "query" { "bool" { "must" \[ { "match" { "title" { "query" "developer", "operator" "and" } } }, { "match" { "location" { "query" "united states", "operator" "and" } } }, { "nested" { "path" "member experience collection", "query" { "bool" { "must" \[ { "query string" { "query" "it company", "default field" "member experience collection company name", "default operator" "and" } } ] } } } } ] } } } }' rate limits bulk collect endpoints have limited requests allowed per client's api token per second request type endpoint rate limit post /v2/data requests/employee base/ids 27 requests per second post /v2/data requests/employee base/filter 27 requests per second post /v2/data requests/employee base/es dsl 27 requests per second post /v2/data requests/employee base/shorthand names 27 requests per second post /v2/data requests/employee base/urls 27 requests per second get /v2/data requests/{data request id}/files 27 requests per second get /v2/data requests/{data request id}/files/{file name} 27 requests per second webhooks post endpoints allow you to add webhooks and get notified when your data request is ready request body template filter template { "data format" "json", "webhook url" "{optional webhook url}", "filters" {} } es dsl template { "data format" "json", "webhook url" "{optional webhook url}", "es dsl query" {} } id file curl template curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/employee base/ids' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type multipart/form data' \\ f 'ids file=@id list example csv;type=text/csv' \\ f 'data format=json' \\ f 'webhook url={optional webhook url}' keep in mind that webhook url is optional bulk collect requests check the endpoint usage tutorials in the following articles post requests docid\ hvajncbflry6xoff 548c find examples and instructions for the bulk collect post requests get requests docid\ sxlwwtymx9hieydap5wso find examples and instructions for the bulk collect get requests