Managed Service for Apache Spark optional ZooKeeper component

The Apache ZooKeeper component is a centralized service for providing distributed synchronization of data. You can install ZooKeeper when you create a Managed Service for Apache Spark cluster by using the Optional components feature.

Install the component

Install the component when you create a Managed Service for Apache Spark cluster.

See Supported image versions for the component version included in each Managed Service for Apache Spark image release.

Google Cloud console

  1. In the Google Cloud console, open Create cluster page.
  2. Click Additional configuration to expand that section.
  3. Edit Optional components.
  4. In the panel that opens, select the checkbox for ZooKeeper, then click Save.

gcloud CLI

To create a Managed Service for Apache Spark cluster that includes the ZooKeeper component, use the gcloud dataproc clusters create cluster-name command with the --optional-components flag (using image version 1.0 or later).

gcloud dataproc clusters create cluster-name \
    --optional-components=ZOOKEEPER \
    --region=region \
    ... other flags

REST API

The ZooKeeper component can be specified through the Dataproc API using SoftwareConfig.Component as part of a clusters.create request.