Prerequisites

This document explains how to prepare your Google Cloud project before you deploy VMware Cloud Foundation (VCF) in VMware solution on Google Cloud.

Required project settings include access to the Layer 2 networking allowlist, Compute Engine quotas, networking, DNS, hostnames, passwords, and security policies. Completing these steps lets bare-metal ESXi instances boot with valid TLS certificates and communicate across VLAN-tagged subnets. For an overview of the service, see VMware solution on Google Cloud overview.

Request access to the Layer 2 networking allowlist

To support VLAN-based networking in VMware solution, subnets in your Virtual Private Cloud (VPC) network require custom MAC learning and Address Resolution Protocol (ARP) broadcasting. Specifically, each node subnet must have the Resolve Subnet Mask property set to Broadcast primary range only with learning by using the --resolve-subnet-mask=ARP_BROADCAST_PRIMARY_RANGE_WITH_LEARNING flag. In the Google Cloud console, VMware Engine > Self-managed displays a warning banner and disables deployment actions until Support adds your project to this allowlist. For more information about VLAN-based networking and custom MAC learning, see Layer 2 emulation: ARP broadcasting and custom MAC learning.

To request that Support add your project to the Layer 2 networking allowlist, complete the following steps:

  1. In the Google Cloud console, go to the Support page:

    Go to Support

  2. Create a support case with the following field values:

    • Product: select VMware solution on Google Cloud.
    • Priority: select P2.
    • Category: select Network Connectivity.
    • Business Impact: select General Technical question.
    • Provide more details: enter Allowlist L2 network connectivity for my project, along with your project ID, project number, and the target region and zone for your deployment.
  3. Submit the support case.

  4. After Support confirms that your project is on the allowlist, go to VMware Engine > Self-managed in the Google Cloud console. Verify that the allowlist warning banner no longer appears and that deployment actions are enabled.

Verify quotas and permissions

Before you deploy infrastructure, verify that your user account or provisioning service account has the required Identity and Access Management (IAM) roles and sufficient Compute Engine quota.

Required roles

To get the permissions that you need to provision VMware solution infrastructure, ask your administrator to grant you the following IAM roles on your target Google Cloud project:

  • Provision bare-metal instances, placement policies, VPC networks, Layer 2 broadcast subnets, zonal network endpoint groups (NEGs), and internal load balancers: Compute Admin (roles/compute.admin)
  • Manage private forward and reverse lookup zones, inbound server policies, and A and PTR records in Cloud DNS: DNS Administrator (roles/dns.admin)
  • Read ESXi and VCF passwords from Secret Manager when running the VMware solution toolkit: Secret Manager Secret Accessor (roles/secretmanager.secretAccessor)

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Verify regional quota

Before you provision bare-metal nodes or create a cluster, verify that your project has sufficient regional and global Compute Engine quota. Add the resources of all existing clusters in the region to the following incremental values before you compare them against your project quota:

Quota metric Resources required per bare-metal node Resources required for a 3-node cluster
compute.googleapis.com/cpus_per_vm_family for Z3 192 vCPUs 3 × 192 = 576 vCPUs
compute.googleapis.com/local_ssd_total_storage_per_vm_family for Z3 72 TB 3 × 72 TB = 216 TB
compute.googleapis.com/hyperdisk_balanced_total_storage 128 GB 3 × 128 GB = 384 GB
compute.googleapis.com/resource_policies 0 when adding nodes to an existing cluster 1 spread placement policy per cluster
compute.googleapis.com/regional_backend_services_protocol_forwarding_fast_ha 0 when adding nodes or clusters without new appliances 1 per new VCF management appliance VIP or NSX Edge endpoint

For the complete list of required quotas and instructions on how to view or request quota increases, see Quotas and limits.

Plan VPC networking

Before you provision bare-metal ESXi nodes, verify that your VPC network architecture meets the following requirements:

  • Maximum transmission unit (MTU): before you deploy bare-metal nodes, set the host VPC network MTU to 8896 bytes for jumbo frames. Configure your VMware vSphere Distributed Switches, VMkernel adapters, and NSX profiles to match 8896 bytes.
  • Layer 2 broadcast subnets: plan non-overlapping primary IPv4 CIDR ranges of /24 or smaller for your untagged ESXi management (MGMT) subnet and VLAN-tagged dynamic network interface subnets. These dynamic subnets carry vSAN, vMotion, and NSX Tunnel Endpoint (NSX-TEP) traffic. Configure each node subnet by using the --resolve-subnet-mask=ARP_BROADCAST_PRIMARY_RANGE_WITH_LEARNING flag and enable Private Google Access on the management subnet.
  • Dedicated standard subnet: provision or identify a standard subnet without Layer 2 ARP broadcasting in the same region and VPC network. Use this subnet for your jumpbox VM instance, Cloud DNS inbound forwarding DNS_RESOLVER IP address, and Private Service Connect regional offline depot endpoint (GCE_ENDPOINT). Don't place DNS_RESOLVER or GCE_ENDPOINT addresses in Layer 2 ARP broadcast subnets. For more information, see DNS resolution failure with Cloud DNS inbound forwarding on Layer 2 broadcast subnets.
  • Firewall rules: configure VPC firewall rules to allow internal traffic across all ports and protocols within the CIDR ranges of your VPC subnets. If you connect to the VCF environment from peered or hybrid networks, then include those CIDR ranges in your firewall rules.

For architectural details, see Underlay VPC networking, Layer 2 MAC learning, and DNS.

Plan DNS and mandatory node hostnames

VCF requires forward (A) and reverse (PTR) DNS resolution for all bare-metal ESXi hosts, VCF management appliances, and cluster endpoints. When you plan your hostnames and IP address assignments, follow these requirements:

  • Specify unique fully qualified domain names (FQDNs): provide a unique, valid FQDN of 64 characters or fewer for each bare-metal ESXi instance. During initial boot, ESXi generates self-signed X.509 TLS certificates whose Common Name (CN) attribute has a 64-character limit. FQDNs longer than 64 characters fall back to the short hostname and can cause VCF SSL thumbprint and certificate validation checks to fail. Specify each node FQDN across your chosen deployment interface:
    • In the Google Cloud console, enter the FQDN in the required Hostname field for each node.
    • In the Google Cloud CLI, pass the FQDN with the --hostname flag when you run the gcloud compute instances create command.
    • In Terraform, configure the domain suffix and node names so that each instance receives a valid FQDN in the hostname attribute.
  • Configure DNS beforehand for custom or static IP addresses: if you assign Ephemeral (Custom) or reserved static IPv4 addresses to ESXi nodes, then set up your forward and reverse DNS lookup zones first. Create the corresponding A and PTR records before you deploy the instances. Configuring these records beforehand lets each ESXi host resolve its FQDN through reverse DNS and generate matching TLS certificates on initial boot.
  • Synchronize DNS after provisioning automatic ephemeral IP addresses: if you select Ephemeral (Automatic) IPv4 addresses, then create or synchronize the forward (A) and reverse (PTR) DNS records. Update these records right after you provision the bare-metal instances and before you run the VMware solution toolkit or VCF validation.

Verify Broadcom password requirements

Make sure that the root and local user passwords for the VCF Installer and SDDC Manager, as well as the ESXi root password, meet Broadcom password complexity requirements. Weak or non-compliant passwords can cause toolkit initialization or VCF bring-up to fail. For more information, see Default password requirements for VCF components in the Broadcom documentation.

Configure organization policies and VPC Service Controls

Bare-metal ESXi instances in VMware solution boot from Google-provided ESXi images hosted in the smve-prod-public-images project, which has project number 999037279871. By default, you don't need any additional configuration. Apply the following settings only if your organization enforces trusted image constraints or VPC Service Controls service perimeters:

  • Trusted image policy: if your organization enforces the constraints/compute.trustedImageProjects organization policy constraint, then allow the projects/smve-prod-public-images value so that Compute Engine can create bare-metal boot disks. For more information, see Set image access constraints in the Compute Engine documentation.
  • VPC Service Controls egress rule: if your target project is in a service perimeter, then add an egress rule that allows compute.googleapis.com operations to projects/999037279871:

    - egressFrom:
        identityType: ANY_IDENTITY
      egressTo:
        operations:
        - serviceName: compute.googleapis.com
          methodSelectors:
          - method: "*"
        resources:
        - projects/999037279871
    

    For more information, see Egress rules reference in the VPC Service Controls documentation.

What's next