FAQ

refresh rate available formats delivery frequency ongoing jsonl, csv, and parquet daily, monthly, and quarterly 📌 employee profiles are constantly being scraped from a queue of profiles, prioritizing the ones with recent changes some profiles are no longer accessible as such, a complete data refresh is not feasible how do we send data? we send the professional network data using the following methods method description links we provide you with the link and login credentials for you to retrieve the data amazon s3 provide your storage credentials, and we will send the data to you google cloud provide your storage credentials, and we will send the data to you microsoft azure provide your storage credentials, and we will send the data to you what does the data look like? we deliver data in locational datasets global (all countries), english speaking countries, europe, and the united states however, you can always submit a custom request the following example illustrates downloading a dataset using a download link and credentials provided by us json download the gzipped json file using the provided link and credentials click on the file you want to download unzip the file by clicking on it a json file will appear at the unzip location each file will have up to 10,000 employee profile records csv the following example illustrates downloading a dataset using a download link and credentials provided by us click on the link and download the csv gz file unzip the file by clicking each gzipped csv file contains a table with specific data collection (e g , a skills table) from employee profile records the gzipped file might contain several files, but they all belong to the same table (e g , skills ) what tools would you suggest using? we can only offer general solutions since it depends on the tech stack you use or what you prefer using ingesting a large dataset like a professional network employees can be efficiently managed using a combination of tools and technologies tailored to handle big data workloads tool category tool example database systems mongo db https //www mongodb com/docs/manual/ couchbase https //docs couchbase com/home/index html postgresql https //www postgresql org/docs/ apache cassandra https //cassandra apache org/ /index html amazon redshift https //docs aws amazon com/redshift/?icmpid=docs homepage analytics amazon s3 https //docs aws amazon com/s3/?icmpid=docs homepage featuredsvcs + athena https //docs aws amazon com/athena/?icmpid=docs homepage analytics elasticsearch https //www elastic co/elasticsearch data processing frameworks apache spark https //spark apache org/docs/latest/ apache hadoop https //hadoop apache org/docs/current/ data ingestion tools apache nifi https //nifi apache org/documentation/ google bigquery https //cloud google com/bigquery/?utm source=google\&utm medium=cpc\&utm campaign=emea emea all en dr bkws all all trial e gcp 1707574\&utm content=text ad none any dev c cre 683760970761 adgp hybrid+%7c+bkws+ +exa+%7c+txt+ +data+analytics+ +bigquery+ +v1 kwid 43700078882901453 kwd 63326440124 userloc 9062284\&utm term=kw google%20bigquery net g plac &\&gad source=1\&gclid=cjwkcajwjqwzbhaqeiwaqmtgt ydxbhoa9hu9m1p8vqzqtyo1esrm4j0f dmdnxirswc4levn5adtxocyioqavd bwe\&gclsrc=aw\ ds#how it works data etl (extract, transform, load) tools aws glue https //docs aws amazon com/prescriptive guidance/latest/serverless etl aws glue/aws glue etl html talend https //www talend com/knowledge center/ data transformation dbt https //docs getdbt com/ pandas https //pandas pydata org/docs/