Cluster creation process overview

This document explains how to plan and configure the infrastructure for a cluster in Cluster Director. To meet the performance, capacity, and budget requirements of your artificial intelligence (AI), machine learning (ML), or high performance computing (HPC) workload, you must choose the compute, storage, network, and data protection resources that your cluster uses. You must also define how the Slurm orchestrator manages your cluster nodes.

Cluster creation process

Creating a cluster involves configuring compute, storage, and networking resources, as well as the Slurm orchestrator. The following process is to help you pick the configurations that best fits your workload needs:

1. Configure compute resources

2. Configure storage resources

3. Configure networking resources

4. Optional: Configure data protection

5. Configure the Slurm orchestrator

If you want an interactive planning session before you configure resources, then run the following prompt in Compute Advisor:

agent_mode_2 Compute Advisor Compute Advisor is an AI-powered assistant that you can prompt to get customized recommendations for your infrastructure based on information such as your quota limits, existing reservations, and resource location constraints.

This prompt starts an interactive session to evaluate your project quotas and reservations, recommends machine configurations for your cluster, and outputs a summary table with recommended Google Cloud console settings.

Act as a cloud architect. Help me design a cluster in Cluster Director based on my project quotas, existing reservations, and available regions.

Start by asking me about my workload type (such as foundation model training, fine-tuning, or inference) and my target scale. After I provide my workload details, walk me through the five setup steps in this guide:

1. Configure compute resources: Recommend machine series for compute nodes (A4X, A4, A3 Ultra, A3 Mega, or N2), controller nodes (N2), and login nodes (C4A or N2), a deployment region and zone, and a consumption option (future reservations for capacity blocks, future reservations in calendar mode, Flex-start, Spot, or On-demand).

2. Configure storage resources: Recommend a new or existing Filestore or Google Cloud Managed Lustre instance for the shared /home directory, plus any optional Cloud Storage buckets.

3. Configure networking resources: Recommend whether to let Cluster Director create a new VPC network or to use an existing VPC or Shared VPC network with the required subnet, Private Google Access, health check (TCP port 8319), and IAP (TCP port 22) firewall rules.

4. Configure data protection (optional): Identify whether I need a VPC Service Controls service perimeter and list the required restricted APIs.

5. Configure the Slurm orchestrator: Recommend boot disk options for the controller and login nodes, and define the partitions and nodesets with static and dynamic node counts (using multiples of 18 VMs for A4X nodesets).

Format the recommendations as a Markdown summary table for each step, followed by recommended Google Cloud console settings and next steps.

Act as a cloud architect. Help me design a cluster in Cluster Director based on my project quotas, existing reservations, and available regions.

Start by asking me about my workload type (such as foundation model training, fine-tuning, or inference) and my target scale. After I provide my workload details, walk me through the five setup steps in this guide:

1. Configure compute resources: Recommend machine series for compute nodes (A4X, A4, A3 Ultra, A3 Mega, or N2), controller nodes (N2), and login nodes (C4A or N2), a deployment region and zone, and a consumption option (future reservations for capacity blocks, future reservations in calendar mode, Flex-start, Spot, or On-demand).

2. Configure storage resources: Recommend a new or existing Filestore or Google Cloud Managed Lustre instance for the shared /home directory, plus any optional Cloud Storage buckets.

3. Configure networking resources: Recommend whether to let Cluster Director create a new VPC network or to use an existing VPC or Shared VPC network with the required subnet, Private Google Access, health check (TCP port 8319), and IAP (TCP port 22) firewall rules.

4. Configure data protection (optional): Identify whether I need a VPC Service Controls service perimeter and list the required restricted APIs.

5. Configure the Slurm orchestrator: Recommend boot disk options for the controller and login nodes, and define the partitions and nodesets with static and dynamic node counts (using multiples of 18 VMs for A4X nodesets).

Format the recommendations as a Markdown summary table for each step, followed by recommended Google Cloud console settings and next steps.

Edit and run prompt in Compute Advisor

Configure compute resources

You must choose the type and number of Compute Engine instances that you want to create in your cluster. You must consider the following:

• Machine series: you must choose the machines series and types that you want to create in your cluster. Each cluster partition can use a different machine series. Each compute instance in your cluster serves as a node. For each type of node, we recommend the following machine series:

  • Compute nodes: based on your workload, use one of the following machine series:

    • Foundational model training and inference: A4X machine series

    • Large model training, fine-tuning, and inference: A4 or A3 Ultra machine series

    • Mainstream model inference, fine tuning, and serving inference: A3 Mega machine series

    • General purpose high performance computing (HPC): N2 machine series

  • Controller nodes: N2 machine series

  • Login nodes: C4A or N2 machine series

• Location: based on the locations where your chosen machine series are available, you must select the region and zone where you want to deploy your cluster. For more information, review the regions and zones where machine series are available.

• Consumption option: you must choose the consumption option that you want to use to get and use compute resources. Each option affects the availability, obtainability, and pricing of a compute resource. For more information, see Choose a consumption option.

Configure storage resources

You must define the storage that your cluster uses:

• Shared file system for /home directory: a Filestore instance or Managed Lustre instance is required for the shared /home directory in a Slurm cluster. When you create a cluster, you can do one of the following:

  • Let Cluster Director create a new Filestore or Managed Lustre instance with pre-configured settings.

  • Use an existing Filestore or Managed Lustre instance on the same VPC network as your cluster.

  If you use the Dynamic tier for Managed Lustre, then your cluster must use an existing VPC network that has the dynamic_tier_capacity quota allocated.

• Additional storage: for large-scale AI workloads that require more storage, you can add other solutions—for example, Cloud Storage buckets.

Note: You can use both new and existing storage resources in the same cluster. If you use an existing Filestore or Managed Lustre instance, then the instance must be on the same VPC network as your cluster. Each storage resource that you define must specify either a new resource configuration or an existing resource configuration, not both.

Configure networking resources

You must define the networking resources that your cluster uses. To do so, choose one of the following options:

• Create a new VPC network

• Use an existing VPC or Shared VPC network

Create a new VPC network

If you want the cluster to use a new VPC network, then Cluster Director creates the network on your behalf. Cluster Director automatically configures the necessary firewall rules to let you connect to your cluster.

Use an existing VPC or Shared VPC network

To use an existing VPC or Shared VPC network in your cluster, you must configure the network and subnetwork as follows:

• You must enable Private Google Access in your network. For instructions, see Private Google Access configuration.

• You must configure firewall rules that allow the following:

  • SSH connections to VMs

  • Internal ingress connections among VMs

• You must configure the subnetwork as follows:

  • The subnetwork must allow health checks by using the TCP protocol and port 8319. For more information, see Use health checks.

  • The subnetwork must have a firewall rule that allows Identity-Aware Proxy (IAP) to access the VMs by using the TCP protocol on port 22. For more information, see Prepare your project for IAP TCP forwarding.

  • The subnetwork can't overlap with the following IPv4 address ranges:

    • 10.20.1.0/28

    • 10.0.0.0/17

    • 192.168.64.0/18

    • 192.168.128.0/18

• If you want to use a Shared VPC network, then you must also do the following:

  • Configure private services access.

  • Enable the Cloud DNS API in the network's host project.

  • Grant the following IAM roles either at the host project level or the subnetwork level:

    • Grant access to all subnets in the host project:

      • The Compute Network User (roles/compute.networkUser) role to the Google APIs Service Agent

      • The Hypercompute Shared VPC Service Agent (roles/hypercomputecluster.sharedVpcServiceAgent) role to the Cluster Director Service Agent (service-PROJECT_NUMBER@gcp-sa-hypercomputecluster.)

    • Grant access to specific subnets: use this option if your organization's policies require more granular control. Grant the following IAM roles:

      • The Compute Network User (roles/compute.networkUser) role on the subnetwork to the Google APIs Service Agent and Cluster Director Service Agent (service-PROJECT_NUMBER@gcp-sa-hypercomputecluster.)

      • The Compute Network Viewer (roles/compute.networkViewer) role on the network's host project to the Cluster Director Service Agent (service-PROJECT_NUMBER@gcp-sa-hypercomputecluster.)

      • A custom role that includes the dns.managedZones.get, dns.managedZones.list, dns.networks.bindPrivateDNSZone, and dns.networks.targetWithPeeringZone permissions on the network's host project to the Cluster Director Service Agent (service-PROJECT_NUMBER@gcp-sa-hypercomputecluster.)

  For more information about setting up a Shared VPC network, see Set up Shared VPC.

Optional: Configure data protection

If you want to help prevent sensitive data in your cluster from being copied to unauthorized sources, then use VPC Service Controls. Otherwise, skip this section. VPC Service Controls lets you control communication to and among services that Google manages. To use Cluster Director with VPC Service Controls, you must create a security boundary around your resources. Google then creates a service perimeter to manage this boundary.

To create a service perimeter, follow the instructions on Create a service perimeter and specify the following configurations:

• Perimeter resources (perimeterResources): include the project in which you want to create your cluster. If you want to use an existing Shared VPC network in your cluster, then you must also include the host project of that network.

• Restricted services (restrictedServices): include the following APIs:

  • Compute Engine API (compute.googleapis.com)

  • Filestore API (file.googleapis.com)

  • Hypercompute Cluster API (hypercomputecluster.googleapis.com)

  • Cloud Logging API (logging.googleapis.com)

  • Google Cloud Managed Lustre API (lustre.googleapis.com)

  • Cloud Monitoring API (monitoring.googleapis.com)

After you create the service perimeter, it can take up to 30 minutes for the changes to take effect. After that time, the service perimeter applies to any cluster that you create in your project.

Configure the Slurm orchestrator

For the final step, you must configure the Slurm orchestrator. To configure the orchestrator, you define the following components:

• Login and controller nodes: you must define the machine series and boot disk options for the login and controller nodes. These nodes manage the cluster and let you connect to it. Controller nodes require standalone boot disks. For login nodes, you can specify standalone boot disks or storage pools if your selected machine series supports Hyperdisk.

• Partitions and nodesets: you must define how you want to group your compute resources. Each partition can contain one or more nodesets, letting you create different pools of resources for different types of jobs. For each nodeset, you must specify the minimum and maximum number of compute instances. For node boot disks, you can specify standalone boot disks or storage pools.

  Important: If you use A4X VMs in a nodeset, then the total number of VMs in the nodeset must be a multiple of 18. This total is the sum of the static node count and the dynamic node count. Because Compute Engine provisions A4X capacity in indivisible sub-blocks for creating 18 VMs, any remaining capacity is unusable.

What's next

• Try out Cluster Director

• Choose a consumption option