- Test from the command line
- Running tests
- Running unit tests with Gradle
- Multi-module reports
- Running tests with ADB
- Running unit tests with Gradle
- Using the am instrument command
- am instrument options
- Usage examples
- Running the entire test package
- Running all tests in a test case class
- Selecting a subset of tests
- Running tests
Test from the command line
This document describes how to create and run tests directly from the command line. This document assumes that you already know how to create a Android application in your programming environment.
Running tests
You can run tests from the command-line, either with Gradle or with an Android Debug Bridge (adb) shell.
The Android plugin for Gradlelets you run unit tests from your Gradle project via the command-line. For more information onhow to build unit tests for your app, seeBuilding Effective Unit Tests.
Running unit tests with Gradle
The Android Plugin for Gradle lets you run unit tests from your Gradle project via the command-line. For more information on how to build unit tests for your app, see Building Effective Unit Tests.
The table below summarizes how to run your unit tests with Gradle:
Unit test type | Command to run | Test result location |
---|---|---|
Local unit test | Call the test task:
| HTML test result files: path_to_your_project/module_name/build/reports/tests/ directory.XML test result files: path_to_your_project/module_name/build/test-results/ directory. |
Instrumented unit test | Call the connectedAndroidTest task:
| HTML test result files: path_to_your_project/module_name/build/reports/androidTests/connected/ directory.XML test result files: path_to_your_project/module_name/build/outputs/androidTest-results/connected/ directory. |
Gradle supports task name abbreviations. This means, for example, you can initiate the connectedAndroidTest
task by simply entering the following command.
- ./gradlew cAT
The test
and connectedAndroidTest
tasks run tests on each module and build variant in your project. You can run tests for just a specific module in your project by prefixing the test
or connectedAndroidTest
task with the module name and a colon (:). For example, the following command runs instrumented unit tests for just the mylibrary
module.
- ./gradlew mylibrary:connectedAndroidTest
You can also target a specific build variant using the following syntax.
- For local unit tests:
- ./gradlew testVariantNameUnitTest
- For instrumented unit tests:
- ./gradlew connectedVariantNameAndroidTest
Note: If you don't specify a target module to test, Gradle looks through all your modules and runs tests for each variant that matches the configuration name you specify.
Gradle also allows you to target specific tests using the —tests
flag. For example, the following command runs only the sampleTestMethod
tests for the specified build variant. To learn more about using the —tests
flag, read Gradle's documentation on test filtering.
- ./gradlew testVariantNameUnitTest --tests *.sampleTestMethod
Multi-module reports
As described in table 1, Gradle saves test reports in the build/
directory of each module that it tests. However, when running tests across multiple modules, it may be useful to combine all the test results into a single report. To generate a single report when running tests across multiple modules, proceed as follows:
- In your project-level
build.gradle
file, add the following after all other configurations in the file.
- apply plugin: 'android-reporting'
- Invoke the
test
orconnectedAndroidTest
task with themergeAndroidReports
task. For example:
- ./gradlew connectedAndroidTest mergeAndroidReports
If you want to skip test failures in order for Gradle to finish running all remaining tests, add the —continue
option:
- ./gradlew connectedAndroidTest mergeAndroidReports --continue
When Gradle finishes running all your tests, it saves the combined reports in the PATH_TO_YOUR_PROJECT/build/
directory.
Running tests with ADB
When you run tests from the command-line with Android Debug Bridge (adb), you get more options for choosing the tests to run than with any other method. You can select individual test methods, filter tests according to their annotation, or specify testing options. Since the test run is controlled entirely from a command-line, you can customize your testing with shell scripts in various ways.
To run a test from the command-line, you run adb shell
to start a command-line shell on your device or emulator, and then in the shell run the am instrument
command. You control am
and your tests with command-line flags.
As a shortcut, you can start an adb
shell, call am instrument
, and specify command-line flags all on one input line. The shell opens on the device or emulator, runs your tests, produces output, and then returns to the command-line on your computer.
To run a test with am instrument
:
- If necessary, rebuild your main application and test package.
- Install your test package and main application Android package files (
.apk
files) to your current Android device or emulator - At the command-line, enter:
- $ adb shell am instrument -w <test_package_name>/<runner_class>
where <test_package_name>
is the Android package name of your test application, and <runner_class>
is the name of the Android test runner class you are using. The Android package name is the value of the package
attribute of the manifest
element in the manifest file (AndroidManifest.xml
) of your test package. The Android test runner class is usuallyAndroidJUnitRunner
.
Your test results appear in STDOUT
.
This operation starts an adb
shell, then runs am instrument
with the specified parameters. This particular form of the command will run all of the tests in your test package. You can control this behavior with flags that you pass to am instrument
. These flags are described in the next section.
Using the am instrument command
The general syntax of the am instrument
command is:
- am instrument [flags] <test_package>/<runner_class>
The main input parameters to am instrument
are described in the following table:
Parameter | Value | Description |
---|---|---|
<test_package> | The Android package name of the test package. | The value of the package attribute of the manifest element in the test package's manifest file. |
<runner_class> | The class name of the instrumented test runner you are using. | This is usuallyAndroidJUnitRunner . |
The flags for am instrument
are described in the following table:
Flag | Value | Description |
---|---|---|
-w | (none) | Forces am instrument to wait until the instrumentation terminates before terminating itself. The net effect is to keep the shell open until the tests have finished. This flag is not required, but if you do not use it, you will not see the results of your tests. |
-r | (none) | Outputs results in raw format. Use this flag when you want to collect performance measurements, so that they are not formatted as test results. This flag is designed for use with the flag -e perf true (documented in the section Instrument options). |
-e | <test_options> | Provides testing options as key-value pairs. The am instrument tool passes these to the specified instrumentation class via its onCreate() method. You can specify multiple occurrences of -e <test_options> . The keys and values are described in the section am instrument options. You can only use these key-value pairs withAndroidJUnitRunner or with InstrumentationTestRunner and itssubclasses. Using them with any other class has no effect. |
—no-hidden-api-checks | (none) | Disables restrictions on the use of hidden APIs. For more information on what hidden APIs are, and how this can affect your app, readRestrictions on non-SDK interfaces. |
am instrument options
The am instrument
tool passes testing options toAndroidJUnitRunner
or InstrumentationTestRunner
in the form ofkey-value pairs, using the -e
flag, with this syntax:
- -e <key> <value>
Some keys accept multiple values. You specify multiple values in a comma-separated list. For example, this invocation ofAndroidJUnitRunner
provides multiple values for the package
key:
- $ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 \
- > com.android.test/android.support.test.runner.AndroidJUnitRunner
The following table lists the key-value pairs you can use with your test runner.
Key | Value | Description |
---|---|---|
package | <Javapackage_name> | The fully-qualified _Java package name for one of the packages in the test application. Any test case class that uses this package name is executed. Notice that this is not an Android package name; a test package has a single Android package name but may have several Java packages within it. |
class | <classname> | The fully-qualified Java class name for one of the test case classes. Only this test case class is executed. |
<class_name>#method name | A fully-qualified test case class name, and one of its methods. Only this method is executed. Note the hash mark (#) between the class name and the method name. | |
func | true | Runs all test classes that extend InstrumentationTestCase . |
unit | true | Runs all test classes that do _not extend either InstrumentationTestCase or PerformanceTestCase . |
size | [small | medium | large ] | Runs a test method annotated by size. The annotations are @SmallTest , @MediumTest , and @LargeTest . |
perf | true | Runs all test classes that implement PerformanceTestCase . When you use this option, also specify the -r flag for am instrument , so that the output is kept in raw format and not re-formatted as test results. |
debug | true | Runs tests in debug mode. |
log | true | Loads and logs all specified tests, but does not run them. The test information appears in STDOUT . Use this to verify combinations of other filters and test specifications. |
emma | true | Runs an EMMA code coverage analysis and writes the output to /data/<app_package>/coverage.ec on the device. To override the file location, use the coverageFile key that is described in the following entry.Note: This option requires an EMMA-instrumented build of the test application, which you can generate with the coverage target. |
coverageFile | <filename> | Overrides the default location of the EMMA coverage file on the device. Specify this value as a path and filename in UNIX format. The default filename is described in the entry for the emma key. |
-e
Flag Usage Notes
am instrument
invokesonCreate(Bundle)
with aBundle
containing the key-value pairs.- The
package
key takes precedence over theclass
key. If you specifiy a package, and then separately specify a class within that package, Android will run all the tests in the package and ignore theclass
key. - The
func
key andunit
key are mutually exclusive.
Usage examples
The following sections provide examples of using am instrument
to run tests.They are based on the following structure:
- The test package has the Android package name
com.android.demo.app.tests
- Two instrumented test classes:
Foo1
which contains the test methodbar1
, andFoo2
which contains test methodsbar2
andbar3
- The test runner is
AndroidJUnitRunner
.
Running the entire test package
To run all of the test classes in the test package, enter:
- $ adb shell am instrument -w com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner
Running all tests in a test case class
To run all of the tests in the class UnitTests
, enter:
- $ adb shell am instrument -w \
- > -e class com.android.demo.app.tests.Foo \
- > com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner
am instrument
gets the value of the -e
flag, detects the class
keyword, and runs all the methods in the UnitTests
class.
Selecting a subset of tests
To run all of the tests in Foo1
, and the bar3
method in Foo2
, enter:
- $ adb shell am instrument -w \
- > -e class com.android.demo.app.tests.Foo1,com.android.demo.app.tests.Foo2#bar3 \
- > com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner