Class PslScanBuilder (1.0.0)

Version latest

• 1.0.0 (latest)
• 0.4.5
• 0.3.4

public class PslScanBuilder implements ScanBuilder, Scan, SupportsReportStatistics

Both Scan and ScanBuilder implementation, otherwise estimateStatistics() is not called due to bug in DataSourceV2Relation.

Information from https://github.com/GoogleCloudDataproc/spark-bigquery-connector/blob/b0b2a8add37bd044e5a9976ce635bb4230957f29/spark-bigquery-dsv2/spark-3.1-bigquery/src/main/java/com/google/cloud/spark/bigquery/v2/BigQueryScanBuilder.java#L30

Inheritance

java.lang.Object > PslScanBuilder

Implements

org.apache.spark.sql.connector.read.ScanBuilder, org.apache.spark.sql.connector.read.Scan, org.apache.spark.sql.connector.read.SupportsReportStatistics

Inherited Members

Object.clone()

Object.equals(Object)

Object.finalize()

Object.getClass()

Object.hashCode()

Object.notify()

Object.notifyAll()

Object.toString()

Object.wait()

Object.wait(long)

Object.wait(long,int)

Constructors

PslScanBuilder(PslReadDataSourceOptions options)

public PslScanBuilder(PslReadDataSourceOptions options)

Parameter
Name	Description
options	PslReadDataSourceOptions

Methods

build()

public Scan build()

Returns
Type	Description
org.apache.spark.sql.connector.read.Scan

description()

public String description()

Returns
Type	Description
String

estimateStatistics()

public Statistics estimateStatistics()

Returns
Type	Description
org.apache.spark.sql.connector.read.Statistics

readSchema()

public StructType readSchema()

Returns
Type	Description
org.apache.spark.sql.types.StructType

toContinuousStream(String checkpointLocation)

public ContinuousStream toContinuousStream(String checkpointLocation)

Parameter
Name	Description
checkpointLocation	String

Returns
Type	Description
org.apache.spark.sql.connector.read.streaming.ContinuousStream

toMicroBatchStream(String checkpointLocation)

public MicroBatchStream toMicroBatchStream(String checkpointLocation)

Parameter
Name	Description
checkpointLocation	String

Returns
Type	Description
org.apache.spark.sql.connector.read.streaming.MicroBatchStream