以前没有完整记录过cts框架造成的问题，以这个问题记录分析方法
问题初探
测试命令： run cts-on-gsi -m CtsMediaTestCases
在host log中报错如下;
|
1 2 |
09-29 17:24:12 I/RemoteAndroidTest: Running am instrument -w -r --no-hidden-api-checks --abi arm64-v8a -e testFile /data/local/tmp/ajur/includes.txt -e debug false -e newRunListenerMode true -e log true -e notAnnotation android.platform.test.annotations.RestrictedBuildTest,android.platform.test.annotations.AppModeInstant -e notTestFile /data/local/tmp/ajur/excludes.txt -e timeout_msec 1800000 android.media.cts/android.support.test.runner.AndroidJUnitRunner on unknown-aosp_on_arm64-8bec9538 09-29 17:24:12 I/InstrumentationResultParser: test run failed: 'Instrumentation run failed due to 'Process crashed.'' |
在device log中报错:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
09-26 20:22:55.132 18904 18904 D AndroidRuntime: Shutting down VM 09-26 20:22:55.132 18904 18904 E AndroidRuntime: FATAL EXCEPTION: main 09-26 20:22:55.132 18904 18904 E AndroidRuntime: Process: android.media.cts, PID: 18904 09-26 20:22:55.132 18904 18904 E AndroidRuntime: java.lang.RuntimeException: Exception thrown in onCreate() of ComponentInfo{android.media.cts/android.support.test.runner.AndroidJUnitRunner}: java.lang.IllegalArgumentException: "android.media.cts.DecoderTest#testH265HDR10StaticMetadata " not recognized as valid package name 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5868) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread.access$1100(ActivityThread.java:199) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1650) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.os.Handler.dispatchMessage(Handler.java:106) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.os.Looper.loop(Looper.java:193) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread.main(ActivityThread.java:6669) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at java.lang.reflect.Method.invoke(Native Method) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: Caused by: java.lang.IllegalArgumentException: "android.media.cts.DecoderTest#testH265HDR10StaticMetadata " not recognized as valid package name 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.internal.runner.RunnerArgs$Builder.validatePackage(RunnerArgs.java:506) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.internal.runner.RunnerArgs$Builder.parseFromFile(RunnerArgs.java:449) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.internal.runner.RunnerArgs$Builder.fromBundle(RunnerArgs.java:242) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.runner.AndroidJUnitRunner.parseRunnerArgs(AndroidJUnitRunner.java:336) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.support.test.runner.AndroidJUnitRunner.onCreate(AndroidJUnitRunner.java:275) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5863) 09-26 20:22:55.132 18904 18904 E AndroidRuntime: ... 8 more 09-26 20:22:55.136 18904 18904 I Process : Sending signal. PID: 18904 SIG: 9 |
其实看到这里很明显就可以看出是google的问题了，不是case问题就是框架的问题，如果着急的时候可以直接反馈，无需走到我这里来判断
问题分析
如果确定是case问题还是框架问题呢，首先将v3和v4的CtsMediaTestCases.apk这个测试apk都拉出来反编译即可，必要时借助010-Editor改一下魔数头，首先将apk反编成jar包，先了解case中出错位置大致逻辑
对比了下case这里没有修改，那么就是框架问题了。这里出错的逻辑在AndroidJUnitRunner.java中，没有源码，否则这个问题就非常好办了；没有源码我们只能通过smali来加log，有点麻烦；先将出错栈理出来
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 |
public void onCreate(Bundle paramBundle){ this.mArguments = paramBundle; parseRunnerArgs(this.mArguments); if (waitForDebugger(this.mRunnerArgs)) { Log.i("AndroidJUnitRunner", "Waiting for debugger to connect..."); Debug.waitForDebugger(); Log.i("AndroidJUnitRunner", "Debugger connected."); } if (isPrimaryInstrProcess(this.mRunnerArgs.targetProcess)) { this.mUsageTrackerFacilitator = new UsageTrackerFacilitator(this.mRunnerArgs); } else { this.mUsageTrackerFacilitator = new UsageTrackerFacilitator(false); } super.onCreate(paramBundle); paramBundle = this.mRunnerArgs.appListeners.iterator(); while (paramBundle.hasNext()) { ApplicationLifecycleCallback localApplicationLifecycleCallback = (ApplicationLifecycleCallback)paramBundle.next(); ApplicationLifecycleMonitorRegistry.getInstance().addLifecycleCallback(localApplicationLifecycleCallback); } addScreenCaptureProcessors(this.mRunnerArgs); if ((this.mRunnerArgs.orchestratorService != null) && (isPrimaryInstrProcess(this.mRunnerArgs.targetProcess))) { this.mOrchestratorListener = new OrchestratedInstrumentationListener(this); this.mOrchestratorListener.connect(getContext()); return; } start(); } private void parseRunnerArgs(Bundle paramBundle){ this.mRunnerArgs = new RunnerArgs.Builder().fromManifest(this).fromBundle(this, paramBundle).build(); } public Builder fromBundle(Instrumentation paramInstrumentation, Bundle paramBundle){ this.debug = parseBoolean(paramBundle.getString("debug")); this.delayInMillis = parseUnsignedInt(paramBundle.get("delay_msec"), "delay_msec"); this.tests.addAll(parseTestClasses(paramBundle.getString("class"))); this.notTests.addAll(parseTestClasses(paramBundle.getString("notClass"))); this.testPackages.addAll(parseTestPackages(paramBundle.getString("package"))); this.notTestPackages.addAll(parseTestPackages(paramBundle.getString("notPackage"))); RunnerArgs.TestFileArgs localTestFileArgs = parseFromFile(paramInstrumentation, paramBundle.getString("testFile")); this.tests.addAll(RunnerArgs.TestFileArgs.access$2900(localTestFileArgs)); this.testPackages.addAll(RunnerArgs.TestFileArgs.access$3000(localTestFileArgs)); paramInstrumentation = parseFromFile(paramInstrumentation, paramBundle.getString("notTestFile")); this.notTests.addAll(RunnerArgs.TestFileArgs.access$2900(paramInstrumentation)); this.notTestPackages.addAll(RunnerArgs.TestFileArgs.access$3000(paramInstrumentation)); this.listeners.addAll(parseLoadAndInstantiateClasses(paramBundle.getString("listener"), RunListener.class, null)); this.filters.addAll(parseLoadAndInstantiateClasses(paramBundle.getString("filter"), Filter.class, paramBundle)); this.runnerBuilderClasses.addAll(parseAndLoadClasses(paramBundle.getString("runnerBuilder"), RunnerBuilder.class)); this.testSize = paramBundle.getString("size"); this.annotation = paramBundle.getString("annotation"); this.notAnnotations.addAll(parseStrings(paramBundle.getString("notAnnotation"))); this.testTimeout = parseUnsignedLong(paramBundle.getString("timeout_msec"), "timeout_msec"); this.numShards = parseUnsignedInt(paramBundle.get("numShards"), "numShards"); this.shardIndex = parseUnsignedInt(paramBundle.get("shardIndex"), "shardIndex"); this.logOnly = parseBoolean(paramBundle.getString("log")); this.disableAnalytics = parseBoolean(paramBundle.getString("disableAnalytics")); this.appListeners.addAll(parseLoadAndInstantiateClasses(paramBundle.getString("appListener"), ApplicationLifecycleCallback.class, null)); this.codeCoverage = parseBoolean(paramBundle.getString("coverage")); this.codeCoveragePath = paramBundle.getString("coverageFile"); this.suiteAssignment = parseBoolean(paramBundle.getString("suiteAssignment")); this.classLoader = ((ClassLoader)parseLoadAndInstantiateClass(paramBundle.getString("classLoader"), ClassLoader.class)); this.classpathToScan = parseClasspath(paramBundle.getString("classpathToScan")); if (paramBundle.containsKey("remoteMethod")) { this.remoteMethod = parseTestClass(paramBundle.getString("remoteMethod")); } this.orchestratorService = paramBundle.getString("orchestratorService"); this.listTestsForOrchestrator = parseBoolean(paramBundle.getString("listTestsForOrchestrator")); this.targetProcess = paramBundle.getString("targetProcess"); this.screenCaptureProcessors.addAll(parseLoadAndInstantiateClasses(paramBundle.getString("screenCaptureProcessors"), ScreenCaptureProcessor.class, null)); this.shellExecBinderKey = paramBundle.getString("shellExecBinderKey"); this.newRunListenerMode = parseBoolean(paramBundle.getString("newRunListenerMode")); return this; } /* Error */ private RunnerArgs.TestFileArgs parseFromFile(Instrumentation paramInstrumentation, String paramString){ // Byte code: // 0: aconst_null // 1: astore 4 // 3: aconst_null // 4: astore 5 // 6: aconst_null // 7: astore_3 // 8: new 379 android/support/test/internal/runner/RunnerArgs$TestFileArgs // 11: dup // 12: aconst_null // 13: invokespecial 382 android/support/test/internal/runner/RunnerArgs$TestFileArgs:<init> (Landroid/support/test/internal/runner/RunnerArgs$1;)V // 16: astore 7 // 18: aload_2 // 19: ifnonnull +6 -> 25 // 22: aload 7 // 24: areturn // 25: aload_0 // 26: aload_1 // 27: aload_2 // 28: invokespecial 384 android/support/test/internal/runner/RunnerArgs$Builder:openFile (Landroid/app/Instrumentation;Ljava/lang/String;)Ljava/io/BufferedReader; // 31: astore_1 // 32: aload_1 // 33: astore_3 // 34: aload_1 // 35: astore 4 // 37: aload_1 // 38: astore 5 // 40: aload_1 // 41: invokevirtual 387 java/io/BufferedReader:readLine ()Ljava/lang/String; // 44: astore 8 // 46: aload 8 // 48: ifnull +165 -> 213 // 51: aload_1 // 52: astore_3 // 53: aload_1 // 54: astore 4 // 56: aload_1 // 57: astore 5 // 59: aload 8 // 61: invokestatic 208 java/lang/String:valueOf (Ljava/lang/Object;)Ljava/lang/String; // 64: astore 6 // 66: aload_1 // 67: astore_3 // 68: aload_1 // 69: astore 4 // 71: aload_1 // 72: astore 5 // 74: aload 6 // 76: invokevirtual 182 java/lang/String:length ()I // 79: ifeq +24 -> 103 // 82: aload_1 // 83: astore_3 // 84: aload_1 // 85: astore 4 // 87: aload_1 // 88: astore 5 // 90: ldc_w 389 // 93: aload 6 // 95: invokevirtual 231 java/lang/String:concat (Ljava/lang/String;)Ljava/lang/String; // 98: astore 6 // 100: goto +23 -> 123 // 103: aload_1 // 104: astore_3 // 105: aload_1 // 106: astore 4 // 108: aload_1 // 109: astore 5 // 111: new 168 java/lang/String // 114: dup // 115: ldc_w 389 // 118: invokespecial 232 java/lang/String:<init> (Ljava/lang/String;)V // 121: astore 6 // 123: aload_1 // 124: astore_3 // 125: aload_1 // 126: astore 4 // 128: aload_1 // 129: astore 5 // 131: ldc_w 391 // 134: aload 6 // 136: invokestatic 397 android/util/Log:i (Ljava/lang/String;Ljava/lang/String;)I // 139: pop // 140: aload_1 // 141: astore_3 // 142: aload_1 // 143: astore 4 // 145: aload_1 // 146: astore 5 // 148: aload 8 // 150: invokestatic 399 android/support/test/internal/runner/RunnerArgs$Builder:isClassOrMethod (Ljava/lang/String;)Z // 153: ifeq +30 -> 183 // 156: aload_1 // 157: astore_3 // 158: aload_1 // 159: astore 4 // 161: aload_1 // 162: astore 5 // 164: aload 7 // 166: invokestatic 403 android/support/test/internal/runner/RunnerArgs$TestFileArgs:access$2900 (Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;)Ljava/util/List; // 169: aload 8 // 171: invokestatic 407 android/support/test/internal/runner/RunnerArgs$Builder:parseTestClass (Ljava/lang/String;)Landroid/support/test/internal/runner/RunnerArgs$TestArg; // 174: invokeinterface 198 2 0 // 179: pop // 180: goto -148 -> 32 // 183: aload_1 // 184: astore_3 // 185: aload_1 // 186: astore 4 // 188: aload_1 // 189: astore 5 // 191: aload 7 // 193: invokestatic 410 android/support/test/internal/runner/RunnerArgs$TestFileArgs:access$3000 (Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;)Ljava/util/List; // 196: aload 8 // 198: invokestatic 413 android/support/test/internal/runner/RunnerArgs$Builder:validatePackage (Ljava/lang/String;)Ljava/lang/String; // 201: invokestatic 417 android/support/test/internal/runner/RunnerArgs$Builder:parseTestPackages (Ljava/lang/String;)Ljava/util/List; |
发现最关键的一步没有反编出来，只有字节码，无法很方便的判断，只能结合smali来梳理逻辑了
|
1 2 3 4 5 6 7 8 |
@VisibleForTesting static String validatePackage(String paramString) { if (paramString.matches("^([\\p{L}_$][\\p{L}\\p{N}_$]*\\.)*[\\p{L}_$][\\p{L}\\p{N}_$]*$")) { return paramString; } throw new IllegalArgumentException(String.format("\"%s\" not recognized as valid package name", new Object[] { paramString })); } |
validatePackage报错，根据堆栈，得到的packagename是"android.media.cts.DecoderTest#testH265HDR10StaticMetadata "
那么为什么会报这个呢，在validatePackage中通过smali给没有源码的地方加log，发现只有报错时会有加的log，说明只有报错时才会执行validatePackage，这个信息很关键
在parseFromFile中搜索validatePackage发现只有一处调用，梳理逻辑，以下是关键信息
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
.line 445 invoke-static {v3}, Landroid/support/test/internal/runner/RunnerArgs$Builder;->isClassOrMethod(Ljava/lang/String;)Z move-result v2 if-eqz v2, :cond_2 .line 446 invoke-static {v0}, Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;->access$2900(Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;)Ljava/util/List; move-result-object v2 invoke-static {v3}, Landroid/support/test/internal/runner/RunnerArgs$Builder;->parseTestClass(Ljava/lang/String;)Landroid/support/test/internal/runner/RunnerArgs$TestArg; move-result-object v4 invoke-interface{v2, v4}, Ljava/util/List;->add(Ljava/lang/Object;)Z goto :goto_0 .line 449 :cond_2 invoke-static {v0}, Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;->access$3000(Landroid/support/test/internal/runner/RunnerArgs$TestFileArgs;)Ljava/util/List; move-result-object v2 invoke-static {v3}, Landroid/support/test/internal/runner/RunnerArgs$Builder;->validatePackage(Ljava/lang/String;)Ljava/lang/String; move-result-object v4 |
换句话说调用isClassOrMethod返回false导致的问题
|
1 2 3 4 |
@VisibleForTesting static boolean isClassOrMethod(String paramString){ return paramString.matches("^([\\p{L}_$][\\p{L}\\p{N}_$]*\\.)*[\\p{Lu}_$][\\p{L}\\p{N}_$]*(#[\\p{L}_$][\\p{L}\\p{N}_$]*)?$"); } |
"android.media.cts.DecoderTest#testH265HDR10StaticMetadata "没有通过正则表达式，进入了validatePackage逻辑，自然报错
那么这个"android.media.cts.DecoderTest#testH265HDR10StaticMetadata "是在哪赋值的呢，在框架中vts-tradefed.jar
见vts-tradefed.jar，其中的cts-on-gsi-exclude.xml，果然多了一个空格
问题修复与验证
https://partnerissuetracker.corp.google.com/u/1/issues/116722365
https://android-review.googlesource.com/c/platform/test/vts/+/773225
发现带以上change编包，放到9.0R4的tools文件夹里替换，居然不可用，google这里经常有这种问题，其框架工具不知道用哪个版本编的，这个也是可以与google沟通的一个点 Constance Wu 吴晓丹
当然我们可以本地解，让编出来的vts-tradefed.jar可用，但是幸运的是这里不需要这么麻烦，因为我们解析的是从excludes.txt里面解析出的错误结果
我们新建两个文件includes1.txt （同includes.txt）， excludes1.txt （将excludes.txt中多余空格去掉）
然后
|
1 2 3 4 5 6 7 |
adb install CtsMediaTestCases.apk adb push includes1.txt /data/local/tmp/ajur adb push excludes1.txt /data/local/tmp/ajur adb shell am instrument -w -r --no-hidden-api-checks --abi arm64-v8a -e testFile /data/local/tmp/ajur/includes1.txt -e debug false -e newRunListenerMode true -e log true -e notAnnotation android.platform.test.annotations.RestrictedBuildTest,android.platform.test.annotations.AppModeInstant -e notTestFile /data/local/tmp/ajur/excludes1.txt -e timeout_msec 1800000 android.media.cts/android.support.test.runner.AndroidJUnitRunner |
case正常运行，说明change绝对可以生效
总结
google的case放出来绝对没有测试啊；而且虽然是框架问题，其实逻辑还在case里面