Developer Device Platform Troubleshooting & FAQ

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This page provides troubleshooting help and answers to frequently asked questions about running tests with Developer Device Platform. If you can't find what you're looking for or need additional help, contact us.

Troubleshooting

Why is my test taking so long to run?

When you select a device with a high capacity level in the Developer Device Platform catalog, tests may start faster. When a device has low capacity, tests might take longer to run. If the number of tests invoked is much larger than the capacity of the selected devices, tests can take longer to finish.

Tests running on any device-capacity level may take longer due to the following factors:

• Traffic, which affects device availability and test speed.
• Device or infrastructure failures, which can happen at any time. To check if there is a reported infrastructure for Developer Device Platform, see the Google Cloud Personalized Service Health dashboard.

To learn more about device capacity in the Developer Device Platform, see the Device Catalog.

Why am I receiving inconclusive test results?

Inconclusive test outcomes commonly occur either because of canceled test runs or infrastructure errors. In addition to PASSED and FAILED, Developer Device Platform may return ERROR, TIMED_OUT and CANCELLED.

Infrastructure errors are caused by internal Developer Device Platform issues, like network errors or unexpected device behaviors. Developer Device Platform internally retries test runs that produce infrastructure errors multiple times before reporting an inconclusive outcome.

To determine the cause of the error, follow these steps:

1. Check for known outages in the Google Cloud Service Health dashboard.

2. Retry the test in Developer Device Platform to verify that it is reproducible.

   Note: Developer Device Platform does not charge you for infrastructure errors.

3. Try running the test on a different device or device type, if applicable. See the Device Catalog for more information.

Why did sharding make my tests run longer?

Sharding can cause your tests to run longer when the number of shards you specified exceeds the number of devices available for use in Developer Device Platform. To avoid this situation, limit your device count to your shard count. For more information about choosing a different device, see the Device Catalog.

Why is it taking a long time for my test to start?

When you submit a test request, your app is first validated, re-signed, etc. in preparation for running tests on a device. Normally, this process completes in less than a few seconds, but it can be affected by factors like the size of your app.

After your app is prepared, test executions are scheduled and remain in a queue until a device is ready to run it.

Note: The time your test spends waiting for an available device does not count toward your billing time.

Why is it taking a long time for my test to finish?

After the test execution is finished, test artifacts are downloaded from the device, processed, and uploaded to Cloud Storage. The duration of this step can be affected by the amount and size of the artifacts.

Android-specific troubleshooting

App not returning data and can't locate screenshots

Test execution artifacts (such as screenshots and log files) are stored in Cloud Storage and directly rendered into the Google Cloud console. Check that you have assigned project-level roles.

Also note, Developer Device Platform has a dedicated service agent that uses its own credentials rather than yours to:

• Read and write to Cloud Storage buckets and objects
• Download Cloud Storage input files to the internal system
• Upload files from the internal system to theCloud Storage output bucket

It is possible you have access to a Cloud Storage bucket and file, but that bucket is owned by a different Google Cloud project than the one used in DDP. Therefore, the Developer Device Platform service account does not have access to it.

You could also have additional access controls on individual buckets. See Device Run for how to include files with your tests.

Why am I receiving partial or missing instrumentation test case results?

When you run instrumentation tests, you might see total test cases are less than expected. This is often caused by Developer Device Platform being incapable of parsing the logcat for test case start or end markers that are usually generated by AndroidJUnitRunner.

The following are common causes of this issue:

Issue description	Possible resolution
Test case did not run because of a timeout. If the total duration of the tests is longer than a timeout you specified or longer than a max timeout, Developer Device Platform cancels the rest of the test cases.	Increase the instrumentation timeout for the test, using `--instrumentation-timeout`, to make sure all tests can complete. Shard the tests if you did not already do so, so that each shard runs a subset of the tests and completes in a shorter period of time. If you already enabled sharding, increase the number of shards.
The test case failed to complete because it exited prematurely or got stuck. The test case may exit prematurely because of an uncaught exception or assertion error. Test cases can get stuck in an infinite loop or might be unable to proceed, for example, if the app does not show the correct view and the test case can't perform the action on the UI.	Check the video and the logcat to investigate where the test stopped.
A custom test runner (including extending AndroidJUnitRunner) crashed unexpectedly or wrote unexpected test case start or end markers to logcat.	Check your test runner code.
Excessive logs were written to logcat, which overwhelmed the buffer or crashed the logcat process.	Reduce writes to logcat.
The app under test crashed.	Debug your app.

---

Frequently asked questions (FAQ)

Where can I find information on Developer Device Platform pricing?

See Pricing and billing questions for details.

Where can I find device details, like resolution, etc.?

Detailed device information is available through the API and can be accessed from the Developer Device Platform CLI with the device-run devices describe <device-id> command:

gcloud beta device-run devices describe DEVICE_ID

How do I find out if the traffic reaching my backend is coming from Developer Device Platform?

From your backend, you can determine if traffic is coming from Developer Device Platform-hosted test devices by checking the source IP address against our IP ranges.

Does Developer Device Platform work with VPC-SC?

Developer Device Platform does not work with VPC-SC, which blocks the copying of apps and other test artifacts between Developer Device Platform's internal storage and users' results buckets.

How do I reduce flaky tests in Developer Device Platform?

To detect flaky behavior in your tests, we recommend using the --flaky-test-attempts option. Deflake reruns are billed or counted toward your daily quota the same as normal test executions.

Keep the following in mind:

• DDP will by default run the retry in sequence to save cost. Users need to set --flaky-test-parallel-retry to run in parallel.
• The --flaky-test-retry-level flag defines whether to retry at the shard or individual test level and defaults to shard. Set it to test to reduce the retry test size and duration.

Note: Infrastructure errors are independent from the deflake feature and don't trigger deflake reruns.

iOS-specific FAQ

Does Developer Device Platform support Appium, Flutter/FlutterDriver, ReactNative/Jest, or Cucumber?

While some of these items are on our roadmap, we're unable to provide commitment to supporting these testing and app development platforms.

Why is my iOS test missing videos in the results?

Support for videos in the results is planned for iOS 18 or later.

Android-specific FAQ

Does Developer Device Platform support wearable devices?

Yes! Developer Device Platform supports Google Pixel Watch. You can now run tests on your standalone Wear OS app on Google Pixel Watches. To learn more about Developer Device Platform devices, see the Device Catalog.

Does Developer Device Platform support the latest Google devices?

Yes! Developer Device Platform supports the Google Pixel Tablet and Google Pixel Fold. You can run your tests on your standalone physical devices. To learn more about devices available in Developer Device Platform, see the Device Catalog.

Does Developer Device Platform support Appium, Flutter/FlutterDriver, ReactNative/Jest, or Cucumber?

While some of these items are on our roadmap, we're unable to provide commitment to supporting these testing and app development platforms. However, if you built your app with a framework that supports Espresso (for example, Flutter), you can write an instrumentation test using Espresso and then run the test in Developer Device Platform.

Does Developer Device Platform support testing obfuscated apps, for example, with ProGuard or R8)?

Developer Device Platform does not explicitly support obfuscation or deobfuscation. While the app will likely run, any obfuscated app data, like stack traces, will appear as obfuscated in the logs.

Can I use my foldable device in different foldable states and postures while testing on Developer Device Platform?

Yes! You can test your foldable device in foldable states and postures.

Foldable devices can be in various folded states, such as FLAT (fully open) or HALF_OPENED (between fully open and completely closed).

Postures, on the other hand, consist of specific device orientation and foldable state. For example, tabletop posture, which is a HALF_OPENED state in horizontal orientation, or book posture, which is a HALF_OPENED state in vertical orientation.

If you are running instrumentation tests, you can use the Jetpack WindowManager library and follow testing your app on foldables documentation to test on different states and postures.

Alternatively, available states are device-specific and can be interacted with using the adb shell command cmd device_state.

• To list the current state, run adb shell cmd device_state state.
• To set or override the current state, run adb shell cmd device_state state <IDENTIFIER>.
• To reset the state, run adb shell cmd device_state state reset.
• To check available states, run the adb shell cmd device_state print-states command on the foldable device.

Google Pixel Fold (model ID felix)

$ adb shell cmd device_state print-states
Supported states: [
      DeviceState{identifier=0, name='CLOSED', app_accessible=true},
      DeviceState{identifier=1, name='HALF_OPENED', app_accessible=true},
      DeviceState{identifier=2, name='OPENED', app_accessible=true},
      DeviceState{identifier=3, name='REAR_DISPLAY_STATE', app_accessible=true},
]

Samsung Galaxy Z Fold4 (model ID q4q)

$ adb shell cmd device_state print-states
Supported states: [
      DeviceState{identifier=0, name='CLOSE', app_accessible=true},
      DeviceState{identifier=1, name='TENT', app_accessible=true},
      DeviceState{identifier=2, name='HALF_FOLDED', app_accessible=true},
      DeviceState{identifier=3, name='OPEN', app_accessible=true},
]

Can I try out Developer Device Platform if I don't have an app?

Unlike other Developer Device Platform products, you don't need to add a Developer Device Platform SDK in order to use Developer Device Platform. If you don't already have an app, you can download an APK online or build an app and a test APK from one of the samples in the AndroidX GitHub repository. Note that an instrumentation test requires both an app and a test APK that are built from source code. For more information, read about Instrumented tests.

To learn more about Developer Device Platform features, see the DDP product overview.

What devices are best for screenshot-diff testing?

Screenshot-diff testing is where test assertions are based on comparing screen images obtained while running a test to golden images representing expected behavior. Such tests may be more brittle on some device types than others. We recommend targeting Arm (*.arm) emulator devices for these kinds of tests. Arm emulator devices use images that are very similar or identical to Android Studio generic emulators.

We also recommend that you investigate test libraries that can help make screenshot tests more robust in the presence of expected changes.

Does Developer Device Platform update virtual devices?

Yes! Virtual devices are updated when the following changes are made:

1. Updates to existing images
2. Deprecation of earlier API levels
3. New Android API levels are added

How do I enable coverage reports?

To enable coverage reports, add coverage=true to the additional-test-options field. If you're using Android Test Orchestrator, you must provide a directory path to store the coverage results:

--additional-test-options coverage=true,coverageFilePath=/sdcard/Download/

If you're not using Orchestrator, you can specify a file path:

--additional-test-options coverage=true,coverageFile=/sdcard/Download/coverage.ec

How do I sign in to a Wear app without a phone?

If your app normally requires a phone for login, you can create a build variant that skips login and uses a token embedded in your test build, or read from files on disk typically pushed using the --other-files-to-push flag.

---