FireDAC Components for Databricks

Build 21.0.7930

Batch Processing

Executing Batch Operations to Databricks

FireDAC batch commands drastically improve performance for large numbers of updates. When you execute a FireDAC batch command, the CData FireDAC Components for Databricks utilize the Databricks bulk APIs to post the entire batch to the server in a single request. The server executes all commands in the batch simultaneously.

Batch Processing Flow

Batch processing in FireDAC is accomplished by binding arrays to the columns of a parameterized statement. To execute the batch, you can call the Execute method available in the TFDQuery, TFDCustomCommand, and TFDStoredProc classes. The size of the batch to execute must be specified.

Controlling Batch Size

Instead executing every statement in the batch in bulk, you can use the following overloaded method to execute sub-batches:

procedure Execute(ATimes: Integer, AOffset: Integer);

ATimes defines the number of items to execute in the Params array. ATimes must be equal or less than the ArraySize property of the TFDQuery object.

AOffset defines the index of the row you want to start from. For example, the following calls execute the first two statements:

FDQuery1.Execute(1, 0);
FDQuery1.Execute(2, 1);

Retrieving Affected Rows

You can use the RowsAffected property of the TFDQuery class to retrieve the total number of successful changes.



The examples below demonstrate the batch operation process:

Bulk Insert

The following example prepares a single batch that inserts records in bulk.

FDConnection1.ResourceOptions.ServerOutput := True;
FDQuery1.SQL.Text := 'insert into [CData].[Sample].Customers (City, CompanyName) values (:City, :CompanyName )';
FDQuery1.Params.ArraySize := 100;
FDQuery1.Params[0].AsStrings[0]:= 'MyCity1';
FDQuery1.Params[1].AsStrings[0]:= 'MyCompanyName1';

//next statement
FDQuery1.Params[0].AsStrings[1]:= 'MyCity2';
FDQuery1.Params[1].AsStrings[1]:= 'MyCompanyName2';


Copyright (c) 2021 CData Software, Inc. - All rights reserved.
Build 21.0.7930