Bulk Collect

bulk collect requests' retrieved profiles limit is 10k requests that exceed the limit will be rejected overview discover the bulk collect (bulk api) capabilities and explore potential uses for efficiently retrieving company data in batches bulk collect /#endpoints information about bulk collect functionality and endpoints bulk collect /#credits and tokens information about api credits and tokens bulk collect /#authorization find details about authorization in bulk collect bulk collect /#rate limits information about the enabled api rate limits endpoints bulk collect is an extension of clean company api and includes four post and two get endpoints collect company data in bulk using ids, elasticsearch dsl schema or shorthand names and urls function endpoint post /v2/data requests/company clean/ids post /v2/data requests/company clean/es dsl post /v2/data requests/company clean/shorthand names post /v2/data requests/company clean/urls get /v2/data requests/{data request id}/files get /v2/data requests/{data request id}/files/{file name} credits and tokens y our credits for clean company api will also apply to bulk collect data collection requests the credits are deducted after making the post request the number of credits deducted from your account depends on the data records you download for instance, collecting 100 profiles will consume 100 credits test post requests using clean company api to see how many credits the request will consume the returned record count will show how many credits will be deducted for the bulk collect query what happens if i don't have enough credits? if you don't have enough credits, the request in the /v2/data requests/{data request id}/files endpoint will return the error 402 and the following response insufficient credits { "detail" "insufficient credits" } what happens if i submit duplicate requests? we validate your search and elasticsearch post requests in bulk collect to prevent the submission of duplicate requests if you happen to submit duplicate post requests using search or elasticsearch filters, you will see the following error message 409 { "detail" "identical data request is already in progress " } limiting returned record count include the parameter "limit" int to control the number of records returned by your queries in /v2/data requests/company clean/es dsl endpoints this parameter is entirely optional; however, use it to make sure you don't use too many credits with your bulk collect queries input the exact number of records you want to be returned or delete the parameter if you don't need a set limit for your request general request template with the parameter included es dsl curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/company clean/es dsl' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "webhook url" "{optional webhook url}", "limit" {optional integer}, "es dsl query" {} }' request example to retrieve five records es dsl curl x 'post' \\ 'https //api coresignal com/cdapi/v2/data requests/company clean/es dsl' \\ h 'accept application/json' \\ h 'apikey {api key}' \\ h 'content type application/json' \\ d '{ "query" { "bool" { "must" \[ { "query string" { "query" "2023", "default field" "founded", "default operator" "and" } } ] } } }' where can i see my credit balance? every search and collect response contains an x credits remaining header that shows the number of credits left in your account you can find the credit balance by clicking headers in the postman application example of remaining credits header x credits remaining 9973 authorization to start using our api, you need an api key your account manager or sales representative will generate the key for you upon your request for the api plan all bulk collect requests must contain an authorization header that consists of an apikey authentication string and the api key general authorization examples import requests api endpoint = "api endpoint" api key = {api key} headers = { "apikey" api key } response = requests get(api endpoint, headers=headers) \# print the response content or handle it as needed print(response text)require 'net/http' require 'uri' uri = uri parse("api endpoint") http = net http new(uri host, uri port) request = net http get new(uri request uri) api key = {api key} request\['apikey'] = api key response = http request(request) puts response bodyconst https = require('https'); const apiendpoint = 'api endpoint'; const apikey = {api key}; const options = { headers { 'apikey' apikey } }; https get(apiendpoint, options, (response) => { let data = ''; response on('data', (chunk) => { data += chunk; }); response on('end', () => { console log(data); }); });\<?php $apiendpoint = "api endpoint"; $apikey = {api key}; $options = \[ 'http' => \[ 'header' => "apikey $apikey" ] ]; $context = stream context create($options); $response = file get contents($apiendpoint, false, $context); ?> a ll api requests must be made over https requests made over http will fail use any api compatible tool to authorize and start querying the api if you're unsure where to begin, check out the authorization tutorials below rate limits bulk collect endpoints have limited requests allowed per second function endpoint rate limit post /v2/data requests/company clean/ids 27 requests per second post /v2/data requests/company clean/es dsl 27 requests per second post /v2/data requests/company clean/shorthand names 27 requests per second post /v2/data requests/company clean/urls 27 requests per second get /v2/data requests/{data request id}/files 27 requests per second get /v2/data requests/{data request id}/files/{file name} 27 requests per second bulk collect requests check the endpoint usage tutorials in the following articles post requests docid\ nvdjmp68ifh7xsw oqrly find examples and instructions for the bulk collect post requests get requests docid 5lulh8qjly3volzrftrtu find examples and instructions for the bulk collect get requests