With the Conversational Analytics API, you can connect a data agent to BigQuery to let users ask questions about your data in natural language and receive answers as text, tables, or charts.
To configure your data sources, define references to BigQuery tables, graphs, or both by using either the Python SDK or HTTP. You then provide these references when you create a persistent data agent or send a stateless chat query with inline context, as described in Build a data agent using HTTP and Build a data agent using the Python SDK.
For stateless chat requests, you can instead specify a search scope of datasets, projects, or both, and let the API choose which tables to query.
Before you begin
Before you connect a data agent to BigQuery, complete the following tasks:
- In your Google Cloud project, enable the required APIs for the Conversational Analytics API and BigQuery.
- Verify that you have the required Identity and Access Management (IAM) roles.
- Ensure that you have access to the BigQuery datasets, tables, or graphs that you want to query.
- If you plan to use a search scope, enable the Dataplex API in your Google Cloud project.
Required roles
To get the permissions that you need to connect a data agent to BigQuery and run queries, ask your administrator to grant you the following IAM roles:
-
Create or manage data agents:
- Gemini Data Analytics Data Agent Creator (
roles/geminidataanalytics.dataAgentCreator) on your project - Gemini Data Analytics Data Agent Editor (
roles/geminidataanalytics.dataAgentEditor) on your project
- Gemini Data Analytics Data Agent Creator (
-
Chat with data agents:
- Gemini Data Analytics Data Agent User (
roles/geminidataanalytics.dataAgentUser) on your project - Gemini for Google Cloud User (
roles/cloudaicompanion.user) on your project
- Gemini Data Analytics Data Agent User (
-
Send stateless chat queries with inline context:
Gemini Data Analytics Stateless Chat User (
roles/geminidataanalytics.dataAgentStatelessUser) on your project -
Run BigQuery queries:
BigQuery Job User (
roles/bigquery.jobUser) on your billing project -
Read BigQuery data:
BigQuery Data Viewer (
roles/bigquery.dataViewer) on your dataset, table, or graph -
Search for tables in a search scope:
Dataplex Catalog Viewer (
roles/dataplex.catalogViewer) on your project
For more information about granting roles, see Manage access to projects, folders, and organizations.
You might also be able to get the required permissions through custom roles or other predefined roles.
Connect to BigQuery tables
To connect to BigQuery tables, you specify table references in your data source configuration and attach them to a published_context object. You can connect to multiple tables across different projects and datasets. If you're sending stateless chat queries without creating a persistent data agent, you can provide this data source configuration in the inline_context field of your request instead.
The following sample code connects to two tables and configures a published_context object with the data sources. The first table (TABLE_ID_1) defines optional custom descriptions for the table and a column to provide additional semantic context, while the second table (TABLE_ID_2) relies on automatic schema discovery:
Python SDK
# Table 1: Define a table reference with custom table and column descriptions
table_reference_1 = geminidataanalytics.BigQueryTableReference()
table_reference_1.project_id = "PROJECT_ID"
table_reference_1.dataset_id = "DATASET_ID"
table_reference_1.table_id = "TABLE_ID_1"
# Optional: Provide semantic descriptions for the table and specific columns
table_reference_1.schema = geminidataanalytics.Schema()
table_reference_1.schema.description = "TABLE_DESCRIPTION"
table_reference_1.schema.fields = [
geminidataanalytics.Field(
name="COLUMN_NAME",
description="COLUMN_DESCRIPTION",
)
]
# Table 2: Minimal table reference that relies on automatic schema discovery
table_reference_2 = geminidataanalytics.BigQueryTableReference()
table_reference_2.project_id = "PROJECT_ID"
table_reference_2.dataset_id = "DATASET_ID"
table_reference_2.table_id = "TABLE_ID_2"
# Combine table references into a data source configuration
datasource_references = geminidataanalytics.DatasourceReferences()
datasource_references.bq.table_references = [
table_reference_1,
table_reference_2,
]
# Attach the data source to published context
published_context = geminidataanalytics.Context()
published_context.datasource_references = datasource_references
Replace the sample values as follows:
- PROJECT_ID: The ID of the Google Cloud project that contains your BigQuery dataset and tables. To connect to a public dataset, specify
bigquery-public-data. - DATASET_ID: The ID of the BigQuery dataset that contains your tables. For example,
san_francisco. - TABLE_ID_1 and TABLE_ID_2: The IDs of the BigQuery tables to connect. For example,
street_trees. - TABLE_DESCRIPTION: An optional description of the table's content and purpose.
- COLUMN_NAME: The name of a column in the table for which you're providing an optional description.
- COLUMN_DESCRIPTION: An optional description of the column's content and purpose.
HTTP
bigquery_data_sources = {
"bq": {
"tableReferences": [
# Table 1: Define a table reference with custom table and column descriptions
{
"projectId": "PROJECT_ID",
"datasetId": "DATASET_ID",
"tableId": "TABLE_ID_1",
# Optional: Provide semantic descriptions for the table and specific columns
"schema": {
"description": "TABLE_DESCRIPTION",
"fields": [
{
"name": "COLUMN_NAME",
"description": "COLUMN_DESCRIPTION",
}
],
},
},
# Table 2: Minimal table reference that relies on automatic schema discovery
{
"projectId": "PROJECT_ID",
"datasetId": "DATASET_ID",
"tableId": "TABLE_ID_2",
},
],
},
}
# Attach the data source to published context
published_context = {
"datasourceReferences": bigquery_data_sources,
}
Replace the sample values as follows:
- PROJECT_ID: The ID of the Google Cloud project that contains your BigQuery dataset and tables. To connect to a public dataset, specify
bigquery-public-data. - DATASET_ID: The ID of the BigQuery dataset that contains your tables. For example,
san_francisco. - TABLE_ID_1 and TABLE_ID_2: The IDs of the BigQuery tables to connect. For example,
street_trees. - TABLE_DESCRIPTION: An optional description of the table's content and purpose.
- COLUMN_NAME: The name of a column in the table for which you're providing an optional description.
- COLUMN_DESCRIPTION: An optional description of the column's content and purpose.
While there are no hard limits on the number of tables that you can connect, connecting a large number of tables or tables across unrelated domains can reduce query accuracy—particularly for queries that require complex joins—or exceed the model's input token limit. Configure each data agent with related tables for a specific use case.
To provide additional context for your tables, such as synonyms, tags, example queries, or user-defined functions, see Define data agent context for BigQuery data sources.
Connect to BigQuery Graph
You can connect to graphs by using BigQuery Graph. You can configure a data agent to query graphs alone or to combine graph and table references in the same data source configuration. To connect, provide one or more graph references in the property_graph_references field (for the Python SDK) or the propertyGraphReferences field (for HTTP).
The following sample code defines a connection to a graph and configures a published_context object:
Python SDK
# Define a BigQuery Graph reference
graph_reference = geminidataanalytics.BigQueryPropertyGraphReference()
graph_reference.project_id = "GRAPH_PROJECT_ID"
graph_reference.dataset_id = "GRAPH_DATASET_ID"
graph_reference.property_graph_id = "GRAPH_ID"
# Combine references into a data source configuration
datasource_references = geminidataanalytics.DatasourceReferences()
datasource_references.bq.property_graph_references = [graph_reference]
# Attach the data source to published context
published_context = geminidataanalytics.Context()
published_context.datasource_references = datasource_references
Replace the sample values as follows:
- GRAPH_PROJECT_ID: The ID of the Google Cloud project that contains your BigQuery graph.
- GRAPH_DATASET_ID: The ID of the BigQuery dataset that contains your graph. For example,
financial_data. - GRAPH_ID: The ID of the BigQuery graph. For example,
FinGraph.
HTTP
bigquery_data_sources = {
"bq": {
"propertyGraphReferences": [
# Define a BigQuery Graph reference
{
"projectId": "GRAPH_PROJECT_ID",
"datasetId": "GRAPH_DATASET_ID",
"propertyGraphId": "GRAPH_ID",
},
],
},
}
# Attach the data source to published context
published_context = {
"datasourceReferences": bigquery_data_sources,
}
Replace the sample values as follows:
- GRAPH_PROJECT_ID: The ID of the Google Cloud project that contains your BigQuery graph.
- GRAPH_DATASET_ID: The ID of the BigQuery dataset that contains your graph. For example,
financial_data. - GRAPH_ID: The ID of the BigQuery graph. For example,
FinGraph.
To query both tables and graphs with the same data agent, include both table_references and property_graph_references (Python SDK) or tableReferences and propertyGraphReferences (HTTP) in your bq data source configuration.
Connect to BigQuery datasets and projects
Instead of specifying individual tables, you can let the Conversational Analytics API choose
which tables to query for each question. To let the API choose, specify a
search scope, which is the list of datasets, projects, or both that the API can
search. Add the search scope to your bq data source by using the
search_scope field (for the Python SDK) or the searchScope field (for
HTTP). Then, include the data source configuration in the inline_context
field of a stateless chat request.
You must specify at least one dataset or project. You can specify up to 100 datasets and up to 100 projects.
The following sample code defines a search scope and adds it to an
inline_context object:
Python SDK
# Define a search scope with one or more datasets
search_scope = geminidataanalytics.SearchScope()
search_scope.include_dataset_references = [
geminidataanalytics.BigQueryDatasetReference(
project_id="PROJECT_ID",
dataset_id="DATASET_ID",
)
]
# Optional: Search all tables in one or more projects
# search_scope.include_project_ids = ["SCOPE_PROJECT_ID"]
# Add the search scope to a data source configuration
datasource_references = geminidataanalytics.DatasourceReferences()
datasource_references.bq.search_scope = search_scope
# Attach the data source to inline context
inline_context = geminidataanalytics.Context()
inline_context.datasource_references = datasource_references
Replace the sample values as follows:
- PROJECT_ID: The ID of the Google Cloud project that contains the dataset to search. To search a public dataset, specify
bigquery-public-data. - DATASET_ID: The ID of the BigQuery dataset to search. For example,
faa. - SCOPE_PROJECT_ID: Optional. The ID of a Google Cloud project to search. If you specify both datasets and projects, the API searches the tables in all of the listed datasets and projects.
HTTP
bigquery_data_sources = {
"bq": {
"searchScope": {
# Define a search scope with one or more datasets
"includeDatasetReferences": [
{
"projectId": "PROJECT_ID",
"datasetId": "DATASET_ID",
},
],
# Optional: Search all tables in one or more projects
# "includeProjectIds": ["SCOPE_PROJECT_ID"],
},
},
}
# Attach the data source to inline context
inline_context = {
"datasourceReferences": bigquery_data_sources,
}
Replace the sample values as follows:
- PROJECT_ID: The ID of the Google Cloud project that contains the dataset to search. To search a public dataset, specify
bigquery-public-data. - DATASET_ID: The ID of the BigQuery dataset to search. For example,
faa. - SCOPE_PROJECT_ID: Optional. The ID of a Google Cloud project to search. If you specify both datasets and projects, the API searches the tables in all of the listed datasets and projects.
The API searches and queries by using your credentials, so BigQuery IAM permissions, row-level and column-level security, and VPC Service Controls perimeters still apply.
If the API doesn't find any relevant tables in your search scope, the response
states that no relevant tables were found. The response might also include a
list of the tables in the search scope. To see which tables the API searched
and queried, check the
THOUGHT messages
in the response stream.
For best results, scope your search to specific datasets or to a short list of projects. If you already know which tables contain the data that you need, use table references instead, because the API responds faster when it doesn't need to search.
What's next
- Define data agent context for BigQuery data sources to improve your agent's performance with descriptions, synonyms, and example queries.
- Create a data agent or send stateless chat queries with your data source configuration, as described in Build a data agent using HTTP or Build a data agent using the Python SDK.