- Resource: Dataset
- DatasetReference
- Replica
- ReplicaState
- ReplicaOptions
- LinkedDatasetSource
- LinkedDatasetMetadata
- LinkState
- ExternalDatasetReference
- ExternalCatalogDatasetOptions
- GcpTag
- StorageBillingModel
- Methods
Resource: Dataset
Represents a BigQuery dataset.
| JSON representation |
|---|
{ "kind": string, "etag": string, "id": string, "selfLink": string, "datasetReference": { object ( |
| Fields | |
|---|---|
kind |
Output only. The resource type. |
etag |
Output only. A hash of the resource. |
id |
Output only. The fully-qualified unique name of the dataset in the format projectId:datasetId. The dataset name without the project name is given in the datasetId field. When creating a new dataset, leave this field blank, and instead specify the datasetId field. |
selfLink |
Output only. A URL that can be used to access the resource again. You can use this URL in Get or Update requests to the resource. |
datasetReference |
Required. A reference that identifies the dataset. |
friendlyName |
Optional. A descriptive name for the dataset. |
description |
Optional. A user-friendly description of the dataset. |
defaultTableExpirationMs |
Optional. The default lifetime of all tables in the dataset, in milliseconds. The minimum lifetime value is 3600000 milliseconds (one hour). To clear an existing default expiration with a PATCH request, set to 0. Once this property is set, all newly-created tables in the dataset will have an expirationTime property set to the creation time plus the value in this property, and changing the value will only affect new tables, not existing ones. When the expirationTime for a given table is reached, that table will be deleted automatically. If a table's expirationTime is modified or removed before the table expires, or if you provide an explicit expirationTime when creating a table, that value takes precedence over the default expiration time indicated by this property. |
defaultPartitionExpirationMs |
This default partition expiration, expressed in milliseconds. When new time-partitioned tables are created in a dataset where this property is set, the table will inherit this value, propagated as the When creating a partitioned table, if |
labels |
The labels associated with this dataset. You can use these to organize and group your datasets. You can set this property when inserting or updating a dataset. See Creating and Updating Dataset Labels for more information. |
access[] |
Optional. An array of objects that define dataset access for one or more entities. You can set this property when inserting or updating a dataset in order to control who is allowed to access the data. If unspecified at dataset creation time, BigQuery adds default dataset access for the following entities: access.specialGroup: projectReaders; access.role: READER; access.specialGroup: projectWriters; access.role: WRITER; access.specialGroup: projectOwners; access.role: OWNER; access.userByEmail: [dataset creator email]; access.role: OWNER; If you patch a dataset, then this field is overwritten by the patched dataset's access field. To add entities, you must supply the entire existing access array in addition to any new entities that you want to add. |
access[].role |
An IAM role ID that should be granted to the user, group, or domain specified in this access entry. The following legacy mappings will be applied:
This field will accept any of the above formats, but will return only the legacy format. For example, if you set this field to "roles/bigquery.dataOwner", it will be returned back as "OWNER". |
access[].userByEmail |
[Pick one] An email address of a user to grant access to. For example: fred@example.com. Maps to IAM policy member "user:EMAIL" or "serviceAccount:EMAIL". |
access[].groupByEmail |
[Pick one] An email address of a Google Group to grant access to. Maps to IAM policy member "group:GROUP". |
access[].domain |
[Pick one] A domain to grant access to. Any users signed in with the domain specified will be granted the specified access. Example: "example.com". Maps to IAM policy member "domain:DOMAIN". |
access[].specialGroup |
[Pick one] A special group to grant access to. Possible values include:
Maps to similarly-named IAM members. |
access[].iamMember |
[Pick one] Some other type of member that appears in the IAM Policy but isn't a user, group, domain, or special group. |
access[].view |
[Pick one] A view from a different dataset to grant access to. Queries executed against that view will have read access to views/tables/routines in this dataset. The role field is not required when this field is set. If that view is updated by any user, access to the view needs to be granted again via an update operation. |
access[].routine |
[Pick one] A routine from a different dataset to grant access to. Queries executed against that routine will have read access to views/tables/routines in this dataset. Only UDF is supported for now. The role field is not required when this field is set. If that routine is updated by any user, access to the routine needs to be granted again via an update operation. |
access[].dataset |
[Pick one] A grant authorizing all resources of a particular type in a particular dataset access to this dataset. Only views are supported for now. The role field is not required when this field is set. If that dataset is deleted and re-created, its access needs to be granted again via an update operation. |
access[].condition |
Optional. condition for the binding. If CEL expression in this field is true, this access binding will be considered |
creationTime |
Output only. The time when this dataset was created, in milliseconds since the epoch. |
lastModifiedTime |
Output only. The date when this dataset was last modified, in milliseconds since the epoch. |
location |
The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations. |
defaultEncryptionConfiguration |
The default encryption key for all tables in the dataset. After this property is set, the encryption key of all newly-created tables in the dataset is set to this value unless the table creation request or query explicitly overrides the key. |
replicas[] |
Optional. The list of replicas in a cross region replicated or disaster recovery dataset. |
satisfiesPzs |
Output only. Reserved for future use. |
satisfiesPzi |
Output only. Reserved for future use. |
type |
Output only. Same as
|
linkedDatasetSource |
Optional. The source dataset reference when the dataset is of type LINKED. For all other dataset types it is not set. This field cannot be updated once it is set. Any attempt to update this field using Update and Patch API Operations will be ignored. |
linkedDatasetMetadata |
Output only. Metadata about the LinkedDataset. Filled out when the dataset type is LINKED. |
externalDatasetReference |
Optional. Reference to a read-only external dataset defined in data catalogs outside of BigQuery. Filled out when the dataset type is EXTERNAL. |
externalCatalogDatasetOptions |
Optional. Options defining open source compatible datasets living in the BigQuery catalog. Contains metadata of open source database, schema or namespace represented by the current dataset. |
isCaseInsensitive |
Optional. TRUE if the dataset and its table names are case-insensitive, otherwise FALSE. By default, this is FALSE, which means the dataset and its table names are case-sensitive. This field does not affect routine references. |
defaultCollation |
Optional. Defines the default collation specification of future tables created in the dataset. If a table is created in this dataset without table-level default collation, then the table inherits the dataset default collation, which is applied to the string fields that do not have explicit collation specified. A change to this field affects only tables created afterwards, and does not alter the existing tables. The following values are supported:
|
defaultRoundingMode |
Optional. Defines the default rounding mode specification of new tables created within this dataset. During table creation, if this field is specified, the table within this dataset will inherit the default rounding mode of the dataset. Setting the default rounding mode on a table overrides this option. Existing tables in the dataset are unaffected. If columns are defined during that table creation, they will immediately inherit the table's default rounding mode, unless otherwise specified. |
maxTimeTravelHours |
Optional. Defines the time travel window in hours. The value can be from 48 to 168 hours (2 to 7 days). The default value is 168 hours if this is not set. |
tags[] |
Output only. Tags for the dataset. To provide tags as inputs, use the |
storageBillingModel |
Optional. Updates storageBillingModel for the dataset. |
resourceTags |
Optional. The tags attached to this dataset. Tag keys are globally unique. Tag key is expected to be in the namespaced format, for example "123456789012/environment" where 123456789012 is the ID of the parent organization or project resource for this tag key. Tag value is expected to be the short name, for example "Production". See Tag definitions for more details. |
failoverReservationId |
Optional. The failover reservation associated with the dataset, in regionless format. For example, |
catalogSource |
Output only. The origin of the dataset, one of:
|
DatasetReference
Identifier for a dataset.
| JSON representation |
|---|
{ "datasetId": string, "projectId": string } |
| Fields | |
|---|---|
datasetId |
Required. A unique ID for this dataset, without the project name. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters. |
projectId |
Optional. The ID of the project containing this dataset. |
Replica
The replica metadata of the dataset. If the dataset has two or more replicas, it's considered a cross-region replicated (CRR) dataset.
| JSON representation |
|---|
{ "id": string, "location": string, "primaryAssignmentTime": string, "primaryAssignmentCompletionTime": string, "replicaState": enum ( |
| Fields | |
|---|---|
id |
Required. The user-visible string identifier for this dataset replica. For example, |
location |
Required. The logical BigQuery region where this replica is located. This is uppercase if "US" or "EU", otherwise a lowercase cloud region like "asia-northeast1". For more information, see Dataset locations. |
primaryAssignmentTime |
Output only. The time at which the replica was assigned primary. Only one replica has this field populated at any point. |
primaryAssignmentCompletionTime |
Output only. After a new primary assignment is made, this is the time that the dataset's data has been fully replicated to the replica, and after which the system uses current reads in the new primary replica. |
replicaState |
Output only. The current replication state of the replica. |
creationTime |
Output only. The timestamp when the replica was created. For any dataset's first replica, this matches the dataset's |
backfillCompletionTime |
Output only. The time when the backfill was completed and the replica is ready to use. |
failoverReservationUri |
Output only. The failover reservation resource name for the replica's region if this replica is used for disaster recovery (DR). Fully-qualified format, for example: "projects/{project}/locations/{location}/reservations/{reservation}". The location portion of the format is uppercase if "US" or "EU", otherwise a lowercase cloud region like "asia-northeast1". |
replicaOptions |
Optional. Contains modification options for the replica, such as setting a Cloud KMS key. |
ReplicaState
Specifies the replication state of the replica.
| Enums | |
|---|---|
REPLICA_STATE_UNSPECIFIED |
Should never be set. |
PRIMARY |
The replica is the primary. All routing by default goes to the primary, and only the primary is writable. |
SECONDARY |
The replica is the secondary and is read-only. |
ASSIGNED_PRIMARY |
The replica where the new primary is going to be. At most one replica can have this value, and it's only present when the dataset has a primary switch in progress. |
PREVIOUSLY_ASSIGNED_PRIMARY |
The replica that was previously the primary, but is no longer assigned as the primary. All routing still goes to this replica, but it is no longer writable. Only set on the old primary during a primary switch. At most one replica at any moment can have this value. |
BACKFILLING |
The replica is not queryable yet since it's still backfilling. |
ReplicaOptions
Options set by the user when creating the replica. These include region-specific dataset values, for example default KMS key name. When a resource is replicated, the destination region's replica contains values that should be overwritten in the destination region.
| JSON representation |
|---|
{ "defaultKmsKeyName": string } |
| Fields | |
|---|---|
defaultKmsKeyName |
Optional. The default Cloud KMS encryption key resource name for the replica. |
LinkedDatasetSource
A dataset source type which refers to another BigQuery dataset.
| JSON representation |
|---|
{
"sourceDataset": {
object ( |
| Fields | |
|---|---|
sourceDataset |
The source dataset reference contains project numbers and not project ids. |
LinkedDatasetMetadata
Metadata about the Linked Dataset.
| JSON representation |
|---|
{
"linkState": enum ( |
| Fields | |
|---|---|
linkState |
Output only. Specifies whether Linked Dataset is currently in a linked state or not. |
LinkState
Specifies whether Linked Dataset is currently in a linked state or not.
| Enums | |
|---|---|
LINK_STATE_UNSPECIFIED |
The default value. Default to the LINKED state. |
LINKED |
Normal Linked Dataset state. Data is queryable via the Linked Dataset. |
UNLINKED |
Data publisher or owner has unlinked this Linked Dataset. It means you can no longer query or see the data in the Linked Dataset. |
ExternalDatasetReference
Configures the access a dataset defined in an external metadata storage.
| JSON representation |
|---|
{ "externalSource": string, "connection": string } |
| Fields | |
|---|---|
externalSource |
Required. External source that backs this dataset. |
connection |
Required. The connection id that is used to access the externalSource. Format: projects/{projectId}/locations/{locationId}/connections/{connectionId} |
ExternalCatalogDatasetOptions
Options defining open source compatible datasets living in the BigQuery catalog. Contains metadata of open source database, schema, or namespace represented by the current dataset.
| JSON representation |
|---|
{ "parameters": { string: string, ... }, "defaultStorageLocationUri": string } |
| Fields | |
|---|---|
parameters |
Optional. A map of key value pairs defining the parameters and properties of the open source schema. Maximum size of 2MiB. |
defaultStorageLocationUri |
Optional. The storage location URI for all tables in the dataset. Equivalent to hive metastore's database locationUri. Maximum length of 1024 characters. |
GcpTag
A global tag managed by Resource Manager. https://cloud.google.com/iam/docs/tags-access-control#definitions
| JSON representation |
|---|
{ "tagKey": string, "tagValue": string } |
| Fields | |
|---|---|
tagKey |
Required. The namespaced friendly name of the tag key, e.g. "12345/environment" where 12345 is org id. |
tagValue |
Required. The friendly short name of the tag value, e.g. "production". |
StorageBillingModel
Indicates the billing model that will be applied to the dataset.
| Enums | |
|---|---|
STORAGE_BILLING_MODEL_UNSPECIFIED |
Value not set. |
LOGICAL |
Billing for logical bytes. |
PHYSICAL |
Billing for physical bytes. |
Methods |
|
|---|---|
|
Deletes the dataset specified by the datasetId value. |
|
Returns the dataset specified by datasetID. |
|
Creates a new empty dataset. |
|
Lists all datasets in the specified project to which the user has been granted the READER dataset role. |
|
Updates information in an existing dataset. |
|
Undeletes a dataset which is within time travel window based on datasetId. |
|
Updates information in an existing dataset. |