Cluster Director overview

This document describes the key features and use cases of Cluster Director, which lets you deploy, manage, and monitor clusters for artificial intelligence (AI), machine learning (ML), or high performance computing (HPC) workloads. Cluster Director is a Google Cloud product that automates the complex setup and configuration of clusters, helping you configure compute, networking, and storage resources for your clusters to maximize performance and minimize downtime.

Cluster Director is designed for IT administrators and AI researchers who want to avoid the overhead of managing a cluster, and instead focus on running their workloads.

Cluster Director key features

Cluster Director integrates multiple Google Cloud services into a cohesive system that simplifies cluster management and provides the following features:

  • Automated lifecycle management for clusters: the Cluster Director platform is built around a unified control plane with its own API and UI in the Google Cloud console. This feature is a central place to manage the lifecycle of your clusters, letting you deploy, manage, and scale clusters with all their necessary compute, networking, and storage resources.

  • Managed cluster orchestration and scaling: Cluster Director uses a managed Slurm environment for fault-tolerant and scalable job scheduling. This managed environment lets you scale your cluster on demand by adding and editing machine configurations to an existing cluster to adjust resources based on your workload demands.

  • Resilient, performance-optimized infrastructure: Cluster Director builds clusters on performance-optimized hardware, including CPU-based, GPU-accelerated, and TPU-accelerated machines. The system uses topology-aware scheduling to colocate compute instances and minimize network latency. Cluster Director also provides built-in resiliency, with autohealing capabilities that detect and replace unhealthy compute instances to help minimize downtimes.

  • Integrated observability: Cluster Director provides built-in tools for monitoring the health, topology, and performance of your clusters through a dashboard in the Google Cloud console. These tools let you view the health of the components in your cluster.

Cluster Director use cases

Cluster Director is designed to meet the needs of the following use cases:

Use case Example workloads
Large-scale AI workloads
  • Generative AI and large language model (LLM) training
  • Fraud detection
  • Recommendation models
HPC workloads
  • Complex simulations
  • Drug discovery, protein folding, and genomic analysis
  • Risk analysis and quantitative trading

What's next