PubSubSubscription

Property Value
Google Cloud Service Name Cloud Pub/Sub
Google Cloud Service Documentation /pubsub/docs/
Google Cloud REST Resource Name v1.projects.subscriptions
Google Cloud REST Resource Documentation /pubsub/docs/reference/rest/v1/projects.subscriptions
Config Connector Resource Short Names gcppubsubsubscription
gcppubsubsubscriptions
pubsubsubscription
Config Connector Service Name pubsub.googleapis.com
Config Connector Resource Fully Qualified Name pubsubsubscriptions.pubsub.cnrm.cloud.google.com
Can Be Referenced by IAMPolicy/IAMPolicyMember Yes
Supports IAM Conditions No
Supports IAM Audit Configs No
IAM External Reference Format

projects/{{project}}/subscriptions/{{name}}
Config Connector Default Average Reconcile Interval In Seconds 600

Custom Resource Definition Properties

Annotations

Fields
cnrm.cloud.google.com/project-id

Spec

Schema

ackDeadlineSeconds: integer
bigqueryConfig:
  dropUnknownFields: boolean
  tableRef:
    external: string
    name: string
    namespace: string
  useTopicSchema: boolean
  writeMetadata: boolean
cloudStorageConfig:
  avroConfig:
    writeMetadata: boolean
  bucketRef:
    external: string
    name: string
    namespace: string
  filenamePrefix: string
  filenameSuffix: string
  maxBytes: integer
  maxDuration: string
  state: string
deadLetterPolicy:
  deadLetterTopicRef:
    external: string
    name: string
    namespace: string
  maxDeliveryAttempts: integer
enableExactlyOnceDelivery: boolean
enableMessageOrdering: boolean
expirationPolicy:
  ttl: string
filter: string
messageRetentionDuration: string
pushConfig:
  attributes:
    string: string
  noWrapper:
    writeMetadata: boolean
  oidcToken:
    audience: string
    serviceAccountEmail: string
  pushEndpoint: string
resourceID: string
retainAckedMessages: boolean
retryPolicy:
  maximumBackoff: string
  minimumBackoff: string
topicRef:
  external: string
  name: string
  namespace: string
Fields

ackDeadlineSeconds

Optional

integer

This value is the maximum time after a subscriber receives a message before the subscriber should acknowledge the message.

bigqueryConfig

Optional

object

If delivery to BigQuery is used with this subscription, this field is used to configure it. Either pushConfig, bigQueryConfig or cloudStorageConfig can be set, but not combined. If all three are empty, then the subscriber will pull and ack messages using API methods.

bigqueryConfig.dropUnknownFields

Optional

boolean

When true and useTopicSchema is true, any fields that are a part of the topic schema that are not part of the BigQuery table schema are dropped when writing to BigQuery. Otherwise, the schemas must be kept in sync and any messages with extra fields are not written and remain in the subscription's backlog.

bigqueryConfig.tableRef

Required*

object

The name of the table to which to write data.

bigqueryConfig.tableRef.external

Optional

string

A reference to an externally-managed BigQueryTable resource. Should be in the format "projects/{{projectID}}/datasets/{{datasetID}}/tables/{{tableID}}".

bigqueryConfig.tableRef.name

Optional

string

The name of a BigQueryTable resource.

bigqueryConfig.tableRef.namespace

Optional

string

The namespace of a BigQueryTable resource.

bigqueryConfig.useTopicSchema

Optional

boolean

When true, use the topic's schema as the columns to write to in BigQuery, if it exists.

bigqueryConfig.writeMetadata

Optional

boolean

When true, write the subscription name, messageId, publishTime, attributes, and orderingKey to additional columns in the table. The subscription name, messageId, and publishTime fields are put in their own columns while all other message properties (other than data) are written to a JSON object in the attributes column.

cloudStorageConfig

Optional

object

If delivery to Cloud Storage is used with this subscription, this field is used to configure it. Either pushConfig, bigQueryConfig or cloudStorageConfig can be set, but not combined. If all three are empty, then the subscriber will pull and ack messages using API methods.

cloudStorageConfig.avroConfig

Optional

object

If set, message data will be written to Cloud Storage in Avro format.

cloudStorageConfig.avroConfig.writeMetadata

Optional

boolean

When true, write the subscription name, messageId, publishTime, attributes, and orderingKey as additional fields in the output.

cloudStorageConfig.bucketRef

Required*

object

User-provided name for the Cloud Storage bucket. The bucket must be created by the user. The bucket name must be without any prefix like "gs://".

cloudStorageConfig.bucketRef.external

Optional

string

A reference to an externally managed StorageBucket resource. Should be in the format "projects/{{projectID}}/buckets/{{bucketID}}".

cloudStorageConfig.bucketRef.name

Optional

string

The name of a StorageBucket resource.

cloudStorageConfig.bucketRef.namespace

Optional

string

The namespace of a StorageBucket resource.

cloudStorageConfig.filenamePrefix

Optional

string

User-provided prefix for Cloud Storage filename.

cloudStorageConfig.filenameSuffix

Optional

string

User-provided suffix for Cloud Storage filename. Must not end in "/".

cloudStorageConfig.maxBytes

Optional

integer

The maximum bytes that can be written to a Cloud Storage file before a new file is created. Min 1 KB, max 10 GiB. The maxBytes limit may be exceeded in cases where messages are larger than the limit.

cloudStorageConfig.maxDuration

Optional

string

The maximum duration that can elapse before a new Cloud Storage file is created. Min 1 minute, max 10 minutes, default 5 minutes. May not exceed the subscription's acknowledgement deadline. A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".

cloudStorageConfig.state

Optional

string

An output-only field that indicates whether or not the subscription can receive messages.

deadLetterPolicy

Optional

object

A policy that specifies the conditions for dead lettering messages in this subscription. If dead_letter_policy is not set, dead lettering is disabled.

deadLetterPolicy.deadLetterTopicRef

Optional

object

The name of the topic to which dead letter messages should be published.

deadLetterPolicy.deadLetterTopicRef.external

Optional

string

A reference to an externally managed PubSubTopic resource. Should be in the format "projects/{{projectID}}/topics/{{topicID}}".

deadLetterPolicy.deadLetterTopicRef.name

Optional

string

The name of a PubSubTopic resource.

deadLetterPolicy.deadLetterTopicRef.namespace

Optional

string

The namespace of a PubSubTopic resource.

deadLetterPolicy.maxDeliveryAttempts

Optional

integer

The maximum number of delivery attempts for any message. The value must be between 5 and 100.

enableExactlyOnceDelivery

Optional

boolean

If 'true', Pub/Sub provides the guarantees for the delivery of a message with a given value of messageId on this Subscriptions'.

enableMessageOrdering

Optional

boolean

Immutable. If 'true', messages published with the same orderingKey in PubsubMessage will be delivered to the subscribers in the order in which they are received by the Pub/Sub system. Otherwise, they may be delivered in any order.

expirationPolicy

Optional

object

A policy that specifies the conditions for this subscription's expiration.

expirationPolicy.ttl

Required*

string

Specifies the "time-to-live" duration for an associated resource. The resource expires if it is not active for a period of ttl. If ttl is set to "", the associated resource never expires. A duration in seconds with up to nine fractional digits, terminated by 's'. Example - "3.5s".

filter

Optional

string

Immutable. The subscription only delivers the messages that match the filter.

messageRetentionDuration

Optional

string

How long to retain unacknowledged messages in the subscription's backlog, from the moment a message is published.

pushConfig

Optional

object

If push delivery is used with this subscription, this field is used to configure it. An empty pushConfig signifies that the subscriber will pull and ack messages using API methods.

pushConfig.attributes

Optional

map (key: string, value: string)

Endpoint configuration attributes.

pushConfig.noWrapper

Optional

object

When set, the payload to the push endpoint is not wrapped.Sets the 'data' field as the HTTP body for delivery.

pushConfig.noWrapper.writeMetadata

Required*

boolean

When true, writes the Pub/Sub message metadata to 'x-goog-pubsub-:' headers of the HTTP request. Writes the Pub/Sub message attributes to ':' headers of the HTTP request.

pushConfig.oidcToken

Optional

object

If specified, Pub/Sub will generate and attach an OIDC JWT token as an Authorization header in the HTTP request for every pushed message.

pushConfig.oidcToken.audience

Optional

string

Audience to be used when generating OIDC token. The audience claim identifies the recipients that the JWT is intended for. The audience value is a single case-sensitive string. Having multiple values (array) for the audience field is not supported. More info about the OIDC JWT token audience here: https://tools.ietf.org/html/rfc7519#section-4.1.3 Note: if not specified, the Push endpoint URL will be used.

pushConfig.oidcToken.serviceAccountEmail

Required*

string

Service account email to be used for generating the OIDC token. The caller (for subscriptions.create, subscriptions.patch, and subscriptions.modifyPushConfig RPCs) must have the iam.serviceAccounts.actAs permission for the service account.

pushConfig.pushEndpoint

Required*

string

A URL locating the endpoint to which messages should be pushed. For example, a Webhook endpoint might use "https://example.com/push".

resourceID

Optional

string

Immutable. Optional. The name of the resource. Used for creation and acquisition. When unset, the value of `metadata.name` is used as the default.

retainAckedMessages

Optional

boolean

Indicates whether to retain acknowledged messages. If 'true', then messages are not expunged from the subscription's backlog, even if they are acknowledged, until they fall out of the messageRetentionDuration window.

retryPolicy

Optional

object

A policy that specifies how Pub/Sub retries message delivery for this subscription.

retryPolicy.maximumBackoff

Optional

string

The maximum delay between consecutive deliveries of a given message. Value should be between 0 and 600 seconds. Defaults to 600 seconds. A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".

retryPolicy.minimumBackoff

Optional

string

The minimum delay between consecutive deliveries of a given message. Value should be between 0 and 600 seconds. Defaults to 10 seconds. A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".

topicRef

Required

object

Reference to a PubSubTopic.

topicRef.external

Optional

string

A reference to an externally managed PubSubTopic resource. Should be in the format "projects/{{projectID}}/topics/{{topicID}}".

topicRef.name

Optional

string

The name of a PubSubTopic resource.

topicRef.namespace

Optional

string

The namespace of a PubSubTopic resource.

* Field is required when parent field is specified

Status

Schema

conditions:
- lastTransitionTime: string
  message: string
  reason: string
  status: string
  type: string
observedGeneration: integer
Fields
conditions

list (object)

Conditions represent the latest available observations of the object's current state.
conditions[]

object
conditions[].lastTransitionTime

string

Last time the condition transitioned from one status to another.
conditions[].message

string

Human-readable message indicating details about last transition.
conditions[].reason

string

Unique, one-word, CamelCase reason for the condition's last transition.
conditions[].status

string

Status is the status of the condition. Can be True, False, Unknown.
conditions[].type

string

Type is the type of the condition.
observedGeneration

integer

ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource.

Sample YAML(s)

Basic PubSub Subscription

# Copyright 2020 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubSubscription
metadata:
  labels:
    label-one: "value-one"
  name: pubsubsubscription-sample-basic
spec:
  ackDeadlineSeconds: 15
  messageRetentionDuration: 86400s
  retainAckedMessages: false
  topicRef:
    name: pubsubsubscription-dep1-basic
  deadLetterPolicy:
    deadLetterTopicRef:
      name: pubsubsubscription-dep2-basic
---
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
  name: pubsubsubscription-dep1-basic
---
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
  name: pubsubsubscription-dep2-basic

BigQuery PubSub Subscription

# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Replace ${PROJECT_ID?} below with your desired project ID.
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubSubscription
metadata:
  name: pubsubsubscription-sample-bigquery
  annotations:
    # Replace ${PROJECT_ID?} with your project ID
    cnrm.cloud.google.com/project-id: ${PROJECT_ID?}
spec:
  bigqueryConfig:
    tableRef:
      name: pubsubsubscription-dep-bigquery
  topicRef:
    name: pubsubsubscription-dep-bigquery
---
# Replace ${PROJECT_ID?} below with your desired project ID.
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
kind: BigQueryDataset
metadata:
  name: pubsubsubscription-dep-bigquery
  annotations:
    # Replace ${PROJECT_ID?} with your project ID
    cnrm.cloud.google.com/project-id: ${PROJECT_ID?}
spec:
  resourceID: pubsubsubscriptiondepbigquery
  location: us-central1
---
# Replace ${PROJECT_ID?} below with your desired project ID.
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
kind: BigQueryTable
metadata:
  name: pubsubsubscription-dep-bigquery
  annotations:
    # Replace ${PROJECT_ID?} with your project ID
    cnrm.cloud.google.com/project-id: ${PROJECT_ID?}
spec:
  resourceID: pubsubsubscriptiondepbigquery
  friendlyName: pubsubsubscription-dep-bigquery
  datasetRef:
    name: pubsubsubscription-dep-bigquery
  schema: >
    [
      {
        "name": "data",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "The data"
      }
    ]
---
# Replace ${PROJECT_ID?} and ${PROJECT_NUMBER?} below with your desired project
# ID and project number.
apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMPolicyMember
metadata:
  name: pubsubsubscription-dep1-bigquery
spec:
  member: serviceAccount:service-${PROJECT_NUMBER?}@gcp-sa-pubsub.iam.gserviceaccount.com
  role: roles/bigquery.metadataViewer
  resourceRef:
    apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
    kind: Project
    external: projects/${PROJECT_ID?}
---
apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMPolicyMember
metadata:
  name: pubsubsubscription-dep2-bigquery
spec:
  member: serviceAccount:service-${PROJECT_NUMBER?}@gcp-sa-pubsub.iam.gserviceaccount.com
  role: roles/bigquery.dataEditor
  resourceRef:
    apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
    kind: Project
    external: projects/${PROJECT_ID?}
---
# Replace ${PROJECT_ID?} below with your desired project ID.
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
  name: pubsubsubscription-dep-bigquery
  annotations:
    # Replace ${PROJECT_ID?} with your project ID
    cnrm.cloud.google.com/project-id: ${PROJECT_ID?}