Smart sharding in Developer Device Platform Device Run distributes Android instrumentation tests and iOS XCTests across parallel shards using historical test execution timings stored in Cloud Storage. Unlike uniform sharding, which divides test cases by count regardless of how long each test runs, smart sharding balances shards by expected duration so that parallel shards finish around the same target time.
This document explains how the smart sharding algorithm calculates shard counts and assigns tests, describes the schema of the YAML timing record stored in your Cloud Storage bucket, and answers common questions about Android Test Orchestrator overhead and the 30-day rolling data window. For command-line instructions using the Google Cloud CLI, see Android instrumentation and iOS XCTest instructions.
How the smart sharding algorithm works
Sharding is not equivalent to how many tests can run concurrently. It's possible that available devices are less than the shard limit at the time of their request. The actual concurrent number of devices a user can be allocated may be lower than the number of shards they specified.
When you submit a session with --sharding-option=smart in the
gcloud CLI or configure smartSharding in the REST API,
Developer Device Platform runs a six-stage workflow across session creation, test
execution, and completion:
- Read historical timing data and match the device context. Before
scheduling shards, the Device Run service agent reads your
smart sharding YAML file from Cloud Storage. It searches the
recordslist for ashardingContextthat matches the target job'sdeviceModelandosVersion(case-insensitive). If no exact match exists for that device and OS combination, Developer Device Platform falls back to the first availablerecordsentry in the file. If the YAML file doesn't exist yet, the session proceeds as a cold start with empty timing history. - Filter samples to the 30-day rolling window and compute expected
durations. For the selected context record, Developer Device Platform retains
only timing samples whose
recordTimefalls within the past 30 days and computes the arithmetic mean duration for each test case. If Android Test Orchestrator is enabled with--orchestrator-version, Developer Device Platform also computes the 30-day mean of theANDROID_TEST_ORCHESTRATORoverhead samples and adds that mean overhead to every test case's expected duration. - Discover runnable tests and assign fallback estimates. On the worker
host, Developer Device Platform statically inspects your test package—reading
DEX files in the test APK on Android, or compiled
.xctestsymbols and.xctestrunfilters on iOS—applies any--test-targetsfilters, and excludes ignored or suppressed tests (such as@Ignoreor@Suppress). Each runnable test unit receives an expected duration based on three tiers:- Recorded test case: uses its 30-day historical mean duration (plus
mean orchestrator overhead when Android Test Orchestrator is enabled).
On Android, parameterized JUnit classes (such as classes using
@RunWith(Parameterized.class),JUnitParamsRunner, orTestParameterInjector) are scheduled as a single class-level unit whose expected duration is the sum of all recorded method durations for that class. - Unrecorded test case in an existing suite: uses the mean expected duration of all recorded non-zero test cases in the matched record.
- Cold start (no recorded history): uses a default estimate of 30 seconds per test unit.
- Recorded test case: uses its 30-day historical mean duration (plus
mean orchestrator overhead when Android Test Orchestrator is enabled).
On Android, parameterized JUnit classes (such as classes using
Calculate the shard count. Developer Device Platform sums the expected durations of all runnable test units and divides by your configured
--smart-sharding-target-duration(valid from2mto1h), rounding up to the nearest whole shard:optimal_shards = max(1, ceil(total_expected_duration / target_duration))Next, Developer Device Platform sets the final shard count between
1and the lowest of the following limits:- The total number of runnable test units (preventing empty shards).
- Your
--smart-sharding-max-shard-countvalue, if set greater than0. - The platform limit (
20shards for physical devices or200shards for Android virtual devices).
Assign tests using longest-processing-time-first bin packing. Developer Device Platform sorts all runnable test units in descending order of expected duration (slowest to fastest) and initializes a min-heap of empty shards ordered by accumulated duration. Taking one test unit at a time, it assigns the slowest remaining test to whichever shard has the smallest accumulated duration and updates that shard's total. Scheduling slow "long-pole" tests first isolates them onto separate shards, while shorter tests fill the remaining gaps to keep total shard durations balanced.
Record passing timings and update Cloud Storage. After all shards finish, Developer Device Platform collects the measured
startTimetoendTimedurations of passing test cases, prunes any samples older than 30 days across all device contexts in the YAML file, removes any context records that have no remaining valid samples, and writes the updated YAML file back to Cloud Storage using generation-matched optimistic concurrency retries.
Smart sharding YAML file structure
When you run a session with the gcloud CLI,
--smart-sharding-record-name specifies the base filename (excluding the YAML
extension) of the timing record stored in your Google Cloud project's
Cloud Storage bucket:
gs://BUCKET_NAME/automation/smart-sharding/RECORD_NAME.yaml
Replace the following placeholders:
BUCKET_NAME: the Cloud Storage bucket name specified with--bucket-name, or<var>PROJECT_ID</var>-devicerunby default (wherePROJECT_IDis your Google Cloud project ID).RECORD_NAME: the record identifier passed to--smart-sharding-record-namewithout the YAML file extension (for example,checkout-suitemaps toautomation/smart-sharding/checkout-suite.yaml).
Example YAML timing record
The following example shows a smart sharding YAML file (checkout-suite.yaml)
containing timing records for two device contexts—an Android device running with
Android Test Orchestrator and an iOS device:
records:
- shardingContext:
deviceModel: shiba
osVersion: '34'
testCaseTimingRecords:
- testCaseName: com.example.shop.CheckoutTest#testCompletePurchase
timingSamples:
- duration: 42.180s
recordTime: '2026-11-08T14:20:11Z'
- duration: 44.050s
recordTime: '2026-11-09T16:45:03Z'
- testCaseName: com.example.shop.CartTest#testApplyPromoCode
timingSamples:
- duration: 8.600s
recordTime: '2026-11-08T14:20:11Z'
- duration: 9.120s
recordTime: '2026-11-09T16:45:03Z'
conditionalOverheads:
- condition: ANDROID_TEST_ORCHESTRATOR
timingSamples:
- duration: 2.850s
recordTime: '2026-11-08T14:20:11Z'
- duration: 3.150s
recordTime: '2026-11-09T16:45:03Z'
- shardingContext:
deviceModel: iphonese3
osVersion: '18.4'
testCaseTimingRecords:
- testCaseName: ShopTests.CheckoutTests#testApplePaySheet
timingSamples:
- duration: 19.340s
recordTime: '2026-11-09T17:10:29Z'
YAML field reference
The smart sharding YAML file serializes the SuiteTiming schema using camelCase
field names. The following table describes each field in the file:
| Field | Type | Description |
|---|---|---|
metadata |
Map (string to string) |
Optional suite-level key-value metadata preserved across aggregation updates. |
records |
List of objects | Per-device timing entries (SuiteTimingWithContext), partitioned by hardware model and OS version. |
records[].shardingContext |
Object | Hardware and OS dimensions that scope a set of timing measurements. |
records[].shardingContext.deviceModel |
string |
Model identifier of the device where timings were recorded (for example, shiba on Android or iphonese3 on iOS). |
records[].shardingContext.osVersion |
string |
OS or SDK version of the device (for example, Android API level '34' or iOS version '18.4'). |
records[].testCaseTimingRecords |
List of objects | Historical execution durations for individual passing test cases in this context. |
records[].testCaseTimingRecords[].testCaseName |
string |
Fully qualified test identifier in package.ClassName#methodName format (or package.ClassName if method metadata is absent). |
records[].testCaseTimingRecords[].timingSamples |
List of objects | Rolling 30-day list of measured durations for the test case, appended after passing runs. |
timingSamples[].duration |
Duration string | Measured duration formatted in seconds with optional fractional precision (for example, 10s or 42.180s). |
timingSamples[].recordTime |
Timestamp string | RFC 3339 UTC timestamp indicating when Developer Device Platform recorded the sample (for example, '2026-11-09T16:45:03Z'). |
records[].conditionalOverheads |
List of objects | Optional execution overheads measured under specific runner modes, scoped to shardingContext. Omitted when no conditional overheads apply. |
records[].conditionalOverheads[].condition |
string |
Identifier for the execution condition. Supported value: ANDROID_TEST_ORCHESTRATOR. |
records[].conditionalOverheads[].timingSamples |
List of objects | Rolling 30-day list of per-transition overhead samples recorded from qualifying shards. |
Understand ANDROID_TEST_ORCHESTRATOR overhead timings
When you run Android instrumentation tests with --orchestrator-version, your
smart sharding YAML file includes a conditionalOverheads entry with
condition: ANDROID_TEST_ORCHESTRATOR. Unlike testCaseTimingRecords, these
overhead samples aren't attached to an individual test method or a numbered
shard.
What one overhead duration sample represents
When Android Test
Orchestrator
is enabled, it runs each test method in its own isolated Instrumentation
process. Individual test startTime and endTime timestamps capture only the
time spent inside each test method—they don't include the inter-test gap spent
stopping the previous app process, optionally clearing package data
(clearPackageData), and starting a fresh Instrumentation process for the
next test.
To measure that inter-test transition cost, Developer Device Platform inspects each
passing shard that runs two or more test cases (N > 1) and calculates:
- Total shard execution span: the elapsed wall-clock time from the
earliest test
startTimein the shard to the latest testendTimein the shard. - Total test case duration: the sum of
(endTime - startTime)across allNtest cases in that shard. Average per-transition overhead: because
Nsequential tests haveN - 1transitions between tests, Developer Device Platform subtracts the summed test durations from the shard span and divides byN - 1:overhead_sample = max(0, total_span - total_test_duration) / (N - 1)
Each passing multi-test shard in your job contributes one TimingSample to
ANDROID_TEST_ORCHESTRATOR. For example, if a session runs across three shards
and each shard passes with at least two test cases, that session appends three
overhead samples with the session's recordTime. Shards that run only a single
test case (N = 1) have zero inter-test transitions and don't emit an overhead
sample.
Why overhead isn't linked to a test case or shard
Developer Device Platform stores ANDROID_TEST_ORCHESTRATOR samples at the
shardingContext (deviceModel and osVersion) level rather than per test or
per shard for two reasons:
- Overhead depends on the device and OS, not test logic. Process startup, binder initialization, and package-data clearing take roughly the same time regardless of which JUnit method runs next, whereas shard groupings change dynamically whenever test timings or shard counts change.
- Separating overhead preserves clean baseline test timings. Keeping
process-isolation overhead out of
testCaseTimingRecordsmeans you can enable or turn off--orchestrator-versionacross runs that share the same YAML record without distorting the underlying per-test execution durations.
How Developer Device Platform uses overhead during shard assignment
When --orchestrator-version is set on a new session, Developer Device Platform
computes the 30-day average of valid ANDROID_TEST_ORCHESTRATOR samples
(avg_overhead) and adds it to every test case's 30-day average duration
before calculating shard counts or packing shards:
expected_test_duration = avg_test_duration + avg_overhead
If --orchestrator-version isn't set on the session, avg_overhead is 0s and
any existing ANDROID_TEST_ORCHESTRATOR entries in the YAML file are ignored
for that run.
Adding avg_overhead to every test case affects sharding in three ways:
- Accurate shard count for suites with many fast tests. Consider a suite
of 60 unit-style instrumentation tests that each run in
1s, with a3saverage orchestrator transition overhead and a2m(120s) target shard duration. Without overhead, the total expected duration looks like60s, so all 60 tests pack into1shard that actually takes nearly four minutes to finish (60sof test execution plus59 * 3s = 177sof orchestrator transitions). By adding the3soverhead to each test (4sexpected duration per test, or240stotal), Developer Device Platform allocates2shards (ceil(240s / 120s)) so each shard finishes near the two-minute target. - Balanced shard packing when test counts differ. During
longest-processing-time-first bin packing, a shard assigned one
40sUI test incurs43sof expected cost (40s + 3s), whereas a shard assigned 104stests incurs70sof expected cost (10 * (4s + 3s)). Including per-test overhead prevents shards with many short tests from silently accumulating minutes of uncounted process-restart latency. - Realistic estimates for newly added tests. Because unrecorded tests in
an existing suite inherit the mean expected duration of known tests—which
already includes
avg_overhead—new tests automatically account for orchestrator overhead on their very first run.
Understand the 30-day data window and initial run behavior
Smart sharding doesn't require 30 days of historical data before it begins optimizing your test runs. Even a single recorded sample activates duration-based shard calculation and bin packing on the very next session. Instead, 30 days is the rolling retention and lookback window that Developer Device Platform uses when reading and updating the YAML timing record.
Why smart sharding uses a 30-day rolling window
Every time Developer Device Platform reads or updates a smart sharding YAML file,
it filters out any TimingSample whose recordTime is more than 30 days old.
This 30-day window balances two engineering goals:
- Adapting to codebase changes and bounding file size. App features, UI flows, and test implementations evolve over time. Automatically evicting samples older than 30 days keeps obsolete timings—for example, timings from before a screen redesign or network-layer refactor—from skewing current shard assignments. Evicting expired entries (and dropping device contexts that no longer have valid samples) also keeps the YAML file compact and well under the 32 MB limit.
- Smoothing run-to-run variance while supporting weekly suites. A single test run can experience transient noise from garbage collection, background system services, or emulator scheduling. Averaging samples across a 30-day window smooths out one-off spikes while retaining enough history that nightly, weekly, or sprint-cadence continuous integration pipelines keep warm timing records between runs.
What to expect in weeks one and two
If you enable smart sharding on an existing test suite today, execution times follow a rapid warm-up curve rather than waiting weeks to improve:
- First run (cold start). Because the YAML file doesn't exist in
Cloud Storage yet, Developer Device Platform assigns every runnable
test unit the 30-second default estimate. It calculates the shard count
as
ceil(num_tests * 30s / target_duration)(capped by your maximum shard count and platform limits) and distributes tests across shards—similar to uniform sharding. On this first run:- If your tests average much less than 30 seconds, Run 1 might allocate
more shards than needed (up to
--smart-sharding-max-shard-count). - If your suite contains several multi-minute UI tests, two slow tests might land in the same shard by chance, making that shard the bottleneck for wall-clock completion time.
- When Run 1 finishes, Developer Device Platform creates the YAML file in Cloud Storage with the measured durations of your passing tests.
- If your tests average much less than 30 seconds, Run 1 might allocate
more shards than needed (up to
- Second run (immediately after Run 1 completes). Smart sharding is already active. Developer Device Platform reads the durations recorded by Run 1, recalculates the shard count from your suite's real execution time, and isolates your slowest tests onto separate shards first. You typically see the primary reduction in wall-clock session duration and shard-count right-sizing starting on Run 2—even on the same day you enable smart sharding.
- Remainder of week one and week two. Over the next several runs, shard
balance continues to tighten in three specific cases:
- Smoothing single-run variance. As each passing run appends another
TimingSample, transient fluctuations from Run 1 average out, reducing straggler shards that finish slightly past--smart-sharding-target-duration. - Filling in tests from initially failed runs. To prevent fast crashes
or test timeouts from corrupting historical averages,
Developer Device Platform records timings only for passing tests (and, on
Android, only from shards whose overall suite status is
PASSED). Any test that didn't record a timing on Run 1 uses the suite's mean duration on Run 2, and then switches to its own measured timing once it passes. - Adding new tests or device models. Any new test cases committed
during weeks one and two automatically use the suite's mean expected
duration instead of reverting to the 30-second default. If you add a
second
--devicemodel to the session using the same record name, its first run uses the existing device's timings as an initial fallback and then records a dedicatedshardingContextentry for subsequent runs.
- Smoothing single-run variance. As each passing run appends another
Frequently asked questions
The following questions cover common operational scenarios when running smart sharding in CI/CD pipelines.
Yes, when multiple sessions run the same test suite. When updating the YAML
file at job completion, Developer Device Platform uses Cloud Storage
object-generation preconditions (generationMatch) with automatic retry and
randomized exponential backoff. If two parallel sessions finish at the same
time, both sessions merge their passing timing samples without overwriting each
other's updates.
Use a distinct --smart-sharding-record-name for each separate test APK or
XCTest bundle (for example, smoke-suite versus regression-suite) so that
fallback suite averages reflect the tests in that suite.
Failed runs rarely reflect normal test execution time: an unhandled exception or
startup crash can fail in milliseconds, whereas a deadlocked UI wait runs until
it hits --instrumentation-timeout or --xctest-timeout. Recording those
outliers can skew 30-day averages and cause future runs to under-allocate or
over-allocate shards.
On Android instrumentation runs, Developer Device Platform records test and
orchestrator timings only from shards whose overall suite result is PASSED. On
iOS XCTest runs, Developer Device Platform records timings for individual PASSED
test cases even if another test case in the same shard failed.
Smart sharding manages shard partitioning automatically and rejects conflicting manual sharding or sub-method filter arguments at session submission:
- Manual
AndroidJUnitRunnersharding arguments. Don't passnumShardsorshardIndexin--additional-test-optionswhen--sharding-option=smartis set. - Bracketed parameterized method targets on Android. Smart sharding
schedules parameterized test classes at the class level and rejects
bracketed
--test-targetsentries such asclass com.example.MyTest#myMethod[param1, param2]. Target the enclosing parameterized class (class com.example.MyTest) instead.
What's next
- Learn about the TestParameterInjector test runner.
- Configure smart sharding flags in Run an Android instrumentation test.
- Configure iOS test runs in Run an iOS XCTest.
- Verify cross-project bucket permissions for the service agent in Share your Cloud Storage buckets with Developer Device Platform.
- Inspect per-shard
junit.xmland log artifacts in Find your Developer Device Platform test results.