CTS/GTS问题分析7

以前没有完整记录过cts框架造成的问题，以这个问题记录分析方法

问题初探

测试命令： run cts-on-gsi -m CtsMediaTestCases

在host log中报错如下;

09-29 17:24:12 I/RemoteAndroidTest: Running am instrument -w -r --no-hidden-api-checks --abi arm64-v8a -e testFile /data/local/tmp/ajur/includes.txt -e debug false -e newRunListenerMode true -e log true -e notAnnotation android.platform.test.annotations.RestrictedBuildTest,android.platform.test.annotations.AppModeInstant -e notTestFile /data/local/tmp/ajur/excludes.txt -e timeout_msec 1800000 android.media.cts/android.support.test.runner.AndroidJUnitRunner on unknown-aosp_on_arm64-8bec9538

09-29 17:24:12 I/InstrumentationResultParser: test run failed: 'Instrumentation run failed due to 'Process crashed.''

在device log中报错:

09-26 20:22:55.132 18904 18904 D AndroidRuntime: Shutting down VM

09-26 20:22:55.132 18904 18904 E AndroidRuntime: FATAL EXCEPTION: main

09-26 20:22:55.132 18904 18904 E AndroidRuntime: Process: android.media.cts, PID: 18904

09-26 20:22:55.132 18904 18904 E AndroidRuntime: java.lang.RuntimeException: Exception thrown in onCreate() of ComponentInfo{android.media.cts/android.support.test.runner.AndroidJUnitRunner}: java.lang.IllegalArgumentException: "android.media.cts.DecoderTest#testH265HDR10StaticMetadata " not recognized as valid package name

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5868)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread.access$1100(ActivityThread.java:199)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1650)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.os.Handler.dispatchMessage(Handler.java:106)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.os.Looper.loop(Looper.java:193)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread.main(ActivityThread.java:6669)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at java.lang.reflect.Method.invoke(Native Method)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: Caused by: java.lang.IllegalArgumentException: "android.media.cts.DecoderTest#testH265HDR10StaticMetadata " not recognized as valid package name

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.internal.runner.RunnerArgs$Builder.validatePackage(RunnerArgs.java:506)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.internal.runner.RunnerArgs$Builder.parseFromFile(RunnerArgs.java:449)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.internal.runner.RunnerArgs$Builder.fromBundle(RunnerArgs.java:242)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.runner.AndroidJUnitRunner.parseRunnerArgs(AndroidJUnitRunner.java:336)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.runner.AndroidJUnitRunner.onCreate(AndroidJUnitRunner.java:275)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5863)

09-26 20:22:55.132 18904 18904 E AndroidRuntime: ... 8 more

09-26 20:22:55.136 18904 18904 I Process : Sending signal. PID: 18904 SIG: 9

其实看到这里很明显就可以看出是google的问题了，不是case问题就是框架的问题，如果着急的时候可以直接反馈，无需走到我这里来判断

问题分析

如果确定是case问题还是框架问题呢，首先将v3和v4的CtsMediaTestCases.apk这个测试apk都拉出来反编译即可，必要时借助010-Editor改一下魔数头，首先将apk反编成jar包，先了解case中出错位置大致逻辑

对比了下case这里没有修改，那么就是框架问题了。这里出错的逻辑在AndroidJUnitRunner.java中，没有源码，否则这个问题就非常好办了；没有源码我们只能通过smali来加log，有点麻烦；先将出错栈理出来

public void onCreate(Bundle paramBundle)

{

this.mArguments = paramBundle;

parseRunnerArgs(this.mArguments);

if (waitForDebugger(this.mRunnerArgs))

{

Log.i("AndroidJUnitRunner", "Waiting for debugger to connect...");

Debug.waitForDebugger();

Log.i("AndroidJUnitRunner", "Debugger connected.");

}

if (isPrimaryInstrProcess(this.mRunnerArgs.targetProcess)) {

this.mUsageTrackerFacilitator = new UsageTrackerFacilitator(this.mRunnerArgs);

} else {

this.mUsageTrackerFacilitator = new UsageTrackerFacilitator(false);

}

super.onCreate(paramBundle);

paramBundle = this.mRunnerArgs.appListeners.iterator();

while (paramBundle.hasNext())

{

ApplicationLifecycleCallback localApplicationLifecycleCallback = (ApplicationLifecycleCallback)paramBundle.next();

ApplicationLifecycleMonitorRegistry.getInstance().addLifecycleCallback(localApplicationLifecycleCallback);

}

addScreenCaptureProcessors(this.mRunnerArgs);

if ((this.mRunnerArgs.orchestratorService != null) && (isPrimaryInstrProcess(this.mRunnerArgs.targetProcess)))

{

this.mOrchestratorListener = new OrchestratedInstrumentationListener(this);

this.mOrchestratorListener.connect(getContext());

return;

}

start();

}

private void parseRunnerArgs(Bundle paramBundle)

{

this.mRunnerArgs = new RunnerArgs.Builder().fromManifest(this).fromBundle(this, paramBundle).build();

}

public Builder fromBundle(Instrumentation paramInstrumentation, Bundle paramBundle)

{

this.debug = parseBoolean(paramBundle.getString("debug"));

this.delayInMillis = parseUnsignedInt(paramBundle.get("delay_msec"), "delay_msec");

this.tests.addAll(parseTestClasses(paramBundle.getString("class")));

this.notTests.addAll(parseTestClasses(paramBundle.getString("notClass")));

this.testPackages.addAll(parseTestPackages(paramBundle.getString("package")));

this.notTestPackages.addAll(parseTestPackages(paramBundle.getString("notPackage")));

RunnerArgs.TestFileArgs localTestFileArgs = parseFromFile(paramInstrumentation, paramBundle.getString("testFile"));

this.tests.addAll(RunnerArgs.TestFileArgs.access$2900(localTestFileArgs));

this.testPackages.addAll(RunnerArgs.TestFileArgs.access$3000(localTestFileArgs));

paramInstrumentation = parseFromFile(paramInstrumentation, paramBundle.getString("notTestFile"));

this.notTests.addAll(RunnerArgs.TestFileArgs.access$2900(paramInstrumentation));

this.notTestPackages.addAll(RunnerArgs.TestFileArgs.access$3000(paramInstrumentation));

this.listeners.addAll(parseLoadAndInstantiateClasses(paramBundle.getString("listener"), RunListener.class, null));

this.filters.addAll(parseLoadAndInstantiateClasses(paramBundle.getString("filter"), Filter.class, paramBundle));

this.runnerBuilderClasses.addAll(parseAndLoadClasses(paramBundle.getString("runnerBuilder"), RunnerBuilder.class));

this.testSize = paramBundle.getString("size");

this.annotation = paramBundle.getString("annotation");

this.notAnnotations.addAll(parseStrings(paramBundle.getString("notAnnotation")));

this.testTimeout = parseUnsignedLong(paramBundle.getString("timeout_msec"), "timeout_msec");

this.numShards = parseUnsignedInt(paramBundle.get("numShards"), "numShards");

this.shardIndex = parseUnsignedInt(paramBundle.get("shardIndex"), "shardIndex");

this.logOnly = parseBoolean(paramBundle.getString("log"));

this.disableAnalytics = parseBoolean(paramBundle.getString("disableAnalytics"));

this.appListeners.addAll(parseLoadAndInstantiateClasses(paramBundle.getString("appListener"), ApplicationLifecycleCallback.class, null));

this.codeCoverage = parseBoolean(paramBundle.getString("coverage"));

this.codeCoveragePath = paramBundle.getString("coverageFile");

this.suiteAssignment = parseBoolean(paramBundle.getString("suiteAssignment"));

this.classLoader = ((ClassLoader)parseLoadAndInstantiateClass(paramBundle.getString("classLoader"), ClassLoader.class));

this.classpathToScan = parseClasspath(paramBundle.getString("classpathToScan"));

if (paramBundle.containsKey("remoteMethod")) {

this.remoteMethod = parseTestClass(paramBundle.getString("remoteMethod"));

}

this.orchestratorService = paramBundle.getString("orchestratorService");

this.listTestsForOrchestrator = parseBoolean(paramBundle.getString("listTestsForOrchestrator"));

this.targetProcess = paramBundle.getString("targetProcess");

this.screenCaptureProcessors.addAll(parseLoadAndInstantiateClasses(paramBundle.getString("screenCaptureProcessors"), ScreenCaptureProcessor.class, null));

this.shellExecBinderKey = paramBundle.getString("shellExecBinderKey");

this.newRunListenerMode = parseBoolean(paramBundle.getString("newRunListenerMode"));

return this;

}

/* Error */

private RunnerArgs.TestFileArgs parseFromFile(Instrumentation paramInstrumentation, String paramString)

{

// Byte code:

// 0: aconst_null

// 1: astore 4

// 3: aconst_null

// 4: astore 5

// 6: aconst_null

// 7: astore_3

// 8: new 379 android/support/test/internal/runner/RunnerArgs$TestFileArgs

// 11: dup

// 12: aconst_null

// 13: invokespecial 382 android/support/test/internal/runner/RunnerArgs$TestFileArgs: (Landroid/support/test/internal/runner/RunnerArgs$1;)V

// 16: astore 7

// 18: aload_2

// 19: ifnonnull +6 -> 25

// 22: aload 7

// 24: areturn

// 25: aload_0

// 26: aload_1

// 27: aload_2

// 28: invokespecial 384 android/support/test/internal/runner/RunnerArgs$Builder:openFile (Landroid/app/Instrumentation;Ljava/lang/String;)Ljava/io/BufferedReader;

// 31: astore_1

// 32: aload_1

// 33: astore_3

// 34: aload_1

// 35: astore 4

// 37: aload_1

// 38: astore 5

// 40: aload_1

// 41: invokevirtual 387 java/io/BufferedReader:readLine ()Ljava/lang/String;

// 44: astore 8

// 46: aload 8

// 48: ifnull +165 -> 213

// 51: aload_1

// 52: astore_3

// 53: aload_1

// 54: astore 4

// 56: aload_1

// 57: astore 5

// 59: aload 8

// 61: invokestatic 208 java/lang/String:valueOf (Ljava/lang/Object;)Ljava/lang/String;

// 64: astore 6

// 66: aload_1

// 67: astore_3

// 68: aload_1

// 69: astore 4

// 71: aload_1

// 72: astore 5

// 74: aload 6

// 76: invokevirtual 182 java/lang/String:length ()I

// 79: ifeq +24 -> 103

// 82: aload_1

// 83: astore_3

// 84: aload_1

// 85: astore 4

// 87: aload_1

// 88: astore 5

// 90: ldc_w 389

// 93: aload 6

// 95: invokevirtual 231 java/lang/String:concat (Ljava/lang/String;)Ljava/lang/String;

// 98: astore 6

// 100: goto +23 -> 123

// 103: aload_1

// 104: astore_3

// 105: aload_1

// 106: astore 4

// 108: aload_1

// 109: astore 5

// 111: new 168 java/lang/String

// 114: dup

// 115: ldc_w 389

// 118: invokespecial 232 java/lang/String: (Ljava/lang/String;)V

// 121: astore 6

// 123: aload_1

// 124: astore_3

// 125: aload_1

// 126: astore 4

// 128: aload_1

// 129: astore 5

// 131: ldc_w 391

// 134: aload 6

// 136: invokestatic 397 android/util/Log:i (Ljava/lang/String;Ljava/lang/String;)I

// 139: pop

// 140: aload_1

// 141: astore_3

// 142: aload_1

// 143: astore 4

// 145: aload_1

// 146: astore 5

// 148: aload 8

// 150: invokestatic 399 android/support/test/internal/runner/RunnerArgs$Builder:isClassOrMethod (Ljava/lang/String;)Z

// 153: ifeq +30 -> 183

// 156: aload_1

// 157: astore_3

// 158: aload_1

// 159: astore 4

// 161: aload_1

// 162: astore 5

// 164: aload 7

// 166: invokestatic 403 android/support/test/internal/runner/RunnerArgs$TestFileArgs:access$2900 (Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;)Ljava/util/List;

// 169: aload 8

// 171: invokestatic 407 android/support/test/internal/runner/RunnerArgs$Builder:parseTestClass (Ljava/lang/String;)Landroid/support/test/internal/runner/RunnerArgs$TestArg;

// 174: invokeinterface 198 2 0

// 179: pop

// 180: goto -148 -> 32

// 183: aload_1

// 184: astore_3

// 185: aload_1

// 186: astore 4

// 188: aload_1

// 189: astore 5

// 191: aload 7

// 193: invokestatic 410 android/support/test/internal/runner/RunnerArgs$TestFileArgs:access$3000 (Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;)Ljava/util/List;

// 196: aload 8

// 198: invokestatic 413 android/support/test/internal/runner/RunnerArgs$Builder:validatePackage (Ljava/lang/String;)Ljava/lang/String;

// 201: invokestatic 417 android/support/test/internal/runner/RunnerArgs$Builder:parseTestPackages (Ljava/lang/String;)Ljava/util/List;

发现最关键的一步没有反编出来，只有字节码，无法很方便的判断，只能结合smali来梳理逻辑了

@VisibleForTesting

static String validatePackage(String paramString)

{

if (paramString.matches("^([\\p{L}_$][\\p{L}\\p{N}_$]*\\.)*[\\p{L}_$][\\p{L}\\p{N}_$]*$")) {

return paramString;

}

throw new IllegalArgumentException(String.format("\"%s\" not recognized as valid package name", new Object[] { paramString }));

}

validatePackage报错，根据堆栈，得到的packagename是"android.media.cts.DecoderTest#testH265HDR10StaticMetadata "

那么为什么会报这个呢，在validatePackage中通过smali给没有源码的地方加log，发现只有报错时会有加的log，说明只有报错时才会执行validatePackage，这个信息很关键

在parseFromFile中搜索validatePackage发现只有一处调用，梳理逻辑，以下是关键信息

.line 445

invoke-static {v3}, Landroid/support/test/internal/runner/RunnerArgs$Builder;->isClassOrMethod(Ljava/lang/String;)Z

move-result v2

if-eqz v2, :cond_2

.line 446

invoke-static {v0}, Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;->access$2900(Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;)Ljava/util/List;

move-result-object v2

invoke-static {v3}, Landroid/support/test/internal/runner/RunnerArgs$Builder;->parseTestClass(Ljava/lang/String;)Landroid/support/test/internal/runner/RunnerArgs$TestArg;

move-result-object v4

invoke-interface {v2, v4}, Ljava/util/List;->add(Ljava/lang/Object;)Z

goto :goto_0

.line 449

:cond_2

invoke-static {v0}, Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;->access$3000(Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;)Ljava/util/List;

move-result-object v2

invoke-static {v3}, Landroid/support/test/internal/runner/RunnerArgs$Builder;->validatePackage(Ljava/lang/String;)Ljava/lang/String;

move-result-object v4

换句话说调用isClassOrMethod返回false导致的问题

@VisibleForTesting

static boolean isClassOrMethod(String paramString)

{

return paramString.matches("^([\\p{L}_$][\\p{L}\\p{N}_$]*\\.)*[\\p{Lu}_$][\\p{L}\\p{N}_$]*(#[\\p{L}_$][\\p{L}\\p{N}_$]*)?$");

}

"android.media.cts.DecoderTest#testH265HDR10StaticMetadata "没有通过正则表达式，进入了validatePackage逻辑，自然报错

那么这个"android.media.cts.DecoderTest#testH265HDR10StaticMetadata "是在哪赋值的呢，在框架中vts-tradefed.jar

见vts-tradefed.jar，其中的cts-on-gsi-exclude.xml，果然多了一个空格

问题修复与验证

发现带以上change编包，放到9.0R4的tools文件夹里替换，居然不可用，google这里经常有这种问题，其框架工具不知道用哪个版本编的，这个也是可以与google沟通的一个点 Constance Wu 吴晓丹

当然我们可以本地解，让编出来的vts-tradefed.jar可用，但是幸运的是这里不需要这么麻烦，因为我们解析的是从excludes.txt里面解析出的错误结果

我们新建两个文件includes1.txt (同includes.txt)， excludes1.txt (将excludes.txt中多余空格去掉)

然后

adb install CtsMediaTestCases.apk

adb push includes1.txt /data/local/tmp/ajur

adb push excludes1.txt /data/local/tmp/ajur

adb shell am instrument -w -r --no-hidden-api-checks --abi arm64-v8a -e testFile /data/local/tmp/ajur/includes1.txt -e debug false -e newRunListenerMode true -e log true -e notAnnotation android.platform.test.annotations.RestrictedBuildTest,android.platform.test.annotations.AppModeInstant -e notTestFile /data/local/tmp/ajur/excludes1.txt -e timeout_msec 1800000 android.media.cts/android.support.test.runner.AndroidJUnitRunner

case正常运行，说明change绝对可以生效

总结

google的case放出来绝对没有测试啊；而且虽然是框架问题，其实逻辑还在case里面