Bulk Collect

Overview

Discover the Bulk Collect (Bulk API) capabilities and explore potential uses for efficiently retrieving employee data in batches. Find all Bulk Collect related information in the following topic:

General information about Bulk Collect

Endpoints

Request type	Endpoint
POST	/v2/data_requests/employee_clean/id_file
POST	/v2/data_requests/employee_clean/es_dsl
POST	/v2/data_requests/employee_clean/shorthand_names
POST	/v2/data_requests/employee_clean/urls
GET	/v2/data_requests/{data_request_id}/files
GET	/v2/data_requests/{data_request_id}/files/{file_name}

Limiting returned record count

Include the parameter "limit": int to control the number of records returned by your queries in /v2/data_requests/employee_clean/es_dsl endpoint.

Request example to retrieve five records

Elasticsearch DSL example

curl -X 'POST' \
'https://api.coresignal.com/cdapi/v2/data_requests/employee_clean/es_dsl' \
  -H 'accept: application/json' \
  -H 'apikey: {API Key}' \
  -H 'Content-Type: application/json' \
  -d '{
   "webhook_url": "{optional_webhook_url}",
   "limit": 5,
   "query": {
      "bool": {
         "should": [
            {
               "query_string": {
                  "query": "Python",
                  "default_field": "description",
                  "default_operator": "and"
               }
            },
            {
               "query_string": {
                  "query": "Python",
                  "default_field": "job_title",
                  "default_operator": "and"
               }
            },
            {
               "nested": {
                  "path": "experience",
                  "query": {
                     "bool": {
                        "should": [
                           {
                              "query_string": {
                                 "query": "Python",
                                 "default_field": "experience.description",
                                 "default_operator": "and"
                              }
                           },
                           {
                              "query_string": {
                                 "query": "Python",
                                 "default_field": "experience.title",
                                 "default_operator": "and"
                              }
                           }
                        ]
                     }
                  }
               }
            }
         ]
      }
   }
}'

Credits

Your credits for Clean Employee API will also apply to Bulk Collect data collection requests.

Learn about the credits in Bulk Collect usage in the general Bulk Collect topic.

Rate limits

Bulk Collect endpoints have limited requests allowed per second. Learn about credits in Bulk Collect usage in the main Clean Employee API topic.

Webhooks

POST endpoints allow you to add webhooks and get notified when your data request is ready.

Keep in mind that webhook_url is optional.

Elasticsearch DSL template

Elasticsearch DSL template

{
  "webhook_url": "{optional_webhook_url}",
  "es_dsl_query": {}
}

ID file cURL template

curl -X 'POST' \
  'https://api.coresignal.com/cdapi/v2/data_requests/employee_clean/id_file' \
  -H 'accept: application/json' \
  -H 'apikey: {API Key}' \
  -H 'Content-Type: multipart/form-data' \
  -F 'ids_file=@id_list_example.csv;type=text/csv' \
  -F 'webhook_url={optional_webhook_url}'