Salesforce Bulk API: Query

Looking for Salesforce Training & HandsOn Projects?

Salesforce Bulk API Query is a powerful asynchronous tool designed to efficiently retrieve large datasets, exceeding the limitations of standard API requests. But its magic lies beneath the surface, in the intricate technical details that drive its performance and flexibility.

To refresh your memory on fundamental Bulk API concepts, I encourage you to revisit my prior articles on Bulk API:

Bulk API Query streamlines large-scale data retrieval by employing a queuing system that decouples query execution from the API call, allowing you to submit jobs, monitor progress in the background, and free up resources for other tasks. It intelligently divides queries into optimized batches for parallel processing and offers flexible output in CSV or XML formats, ensuring seamless integration with external tools and systems.

Bulk API Query efficiently handles large-scale data retrieval with a maximum capacity of 15 GB per job, divided into 15 files. It supports both query and queryAll operations for comprehensive data access, including deleted records and archived events. For optimal performance, aim for batch sizes of 10,000-20,000 records, adjusting as needed based on data complexity and Salesforce instance limits. Leverage either REST or SOAP endpoints for interaction, ensuring API version 38.0 or later.

The Bulk API Query offers flexibility and control for efficient operations, allowing you to choose between sequential or parallel batch processing based on your needs. It supports compression using GZIP to optimize network traffic and storage and incorporates robust error-handling mechanisms to safeguard data integrity and enable job reruns in case of failures. Additionally, you can seamlessly track job progress, including completion percentage, batch times, and any errors, through API calls or the user interface, ensuring transparency and visibility throughout the process.

SOQL optimization minimizes data retrieval and query execution time, relationship queries unlock insights across connected objects with elegant joins, and Bulk API 2.0 enhancements like query locators and custom field filters empower you to grab specific data chunks with laser precision, boosting performance and flexibility for your Salesforce data explorations.

Demo

In this article we look into a step-by-step process of executing large-scale queries using the Salesforce Bulk API, ensuring optimal performance and resource utilization-

1. Initiating a Query Job

Endpoint: /services/data/vXX.X/job (replace XX.X with your API version)
HTTP Method: POST

2. Adding a Batch to the Job

Endpoint: /services/data/vXX.X/job/<jobid>/batch (replace XX.X and <jobid>)
HTTP Method: POST
Request Body:

query: A valid SOQL query to retrieve the desired invoice data

3. Retrieving the Results ID

Endpoint: /services/data/vXX.X/job/<jobid>/batch/<batchId>/result
HTTP Method: GET
Response: Returns a resultId that uniquely identifies the query results

4. Downloading the Results

Endpoint: /services/data/vXX.X/job/<jobid>/batch/<batchId>/result/<resultId>
HTTP Method: GET
Response: Returns a CSV file containing the query results

Conclusion

Salesforce Bulk API Query is a game-changer for retrieving massive data without hitting API walls. It works its magic behind the scenes by queuing jobs, dividing queries into efficient batches, and offering flexible output formats like CSV and XML.

With support for 15 GB per job, deleted records, and optimized batch sizes, you can grab even the largest datasets with ease. Beyond sheer volume, Bulk API Query gives you control through parallel processing, compression, error handling, and progress tracking.

Whether you're optimizing queries, navigating relationships, or utilizing the latest 2.0 features, Bulk API Query unlocks the full potential of your Salesforce data, letting you explore with precision and efficiency.

Search This Blog

Salesforce Geek