针对CTS,GTS测试过程中,在profile_provisioning流程分析与常见解决方案

[DESCRIPTION]

由于目前所有的手机都要求支持Android for work, 客户在测试CTS,GTS过程中,经常在profile_provisioning的过程中出错,这边对经常出错的地方进行分析,并提供对应的解决方案

[SOLUTION]

继续阅读“针对CTS,GTS测试过程中,在profile_provisioning流程分析与常见解决方案”

广升ADUPS FOTA 过认证信息更新

– ADUPS has recently released a patched version for com.adups.fota (version 5.22), that must be used for all new builds submitted to Google

– Google will continue rejecting and blocking any new build(s) submitted with com.adups.fota version 5.16 or older

– If you are using ADUPS as your FOTA provider, updates will be pushed to devices already active in the ecosystem automatically and no further action is required for those devices

– All new builds submitted to Google from hereon need to use ADUPS com.adups.fota version 5.22 or newer, or need to use an alternative FOTA solution

什么是BTS?Build Test Suite (System Image Scanning)

Build Test Suite (System Image Scanning)

The Build Test Suite (BTS) operates directly on software builds. It scans device images for preloaded malware and other Potentially Harmful Applications (PHAs) that may have been introduced by OEMs’ vendors and suppliers. BTS incorporates the same technology currently used for security scanning and analysis of applications installed via Google Play.

Starting April 1st, 2018, all partners MUST begin uploading their software builds to Google as described in their partnership agreements. Submission of software builds will be in addition to the existing test reports required for Google build approval.

Note: Partners are still responsible for vetting and reviewing all software that is distributed on their devices. Google continuously updates our databases and detection techniques for malware and security threats, but we may not always be able to detect an issue at the time of your build submission.

Submitting builds for BTS

Partners must submit whole software builds. Submissions of individual APKs are not supported. Google only scans APKs in the context of the whole build. As BTS develops, we will look beyond just the APKs in the filesystem images.

Enrollment for MADA partners

Partners can upload software builds in one of the following ways:

  1. Google Drive: Your TAM will share a Google Drive folder that you can copy your builds to.
  2. SFTP Dropbox: You can ask your TAM to set up a Secure File Transfer Protocol (SFTP) account for you so that you can upload your build software.
  3. Submission via 3PL: Submit firmware to your 3PL certifier using one of the supported formats.

Submit software builds

For each build that you submit to APFE, you should upload a single compressed archive (.zip, .rar, or .tgz) containing the Software Build in a supported formatThe archive should be named according to the build fingerprint, substituting forward slashes (“/”) and colons (“:”) with tildes (“~”). This helps reconcile the file with the reports in APFE.

Example:
If you have a zip file containing partition images that can be flashed using fastboot for a build with the fingerprint: acme/acme_1/acme_3g:7.0/NRD90M/123456789:user/release-keys
the file should be named:
acme~acme_1~acme_3g~7.0~NRD90M~123456789~user~release-keys.zip

Partners may submit additional software builds, although these submissions should be accompanied by at least a signature CTS report in APFE so that the results are available. Early submissions may help identify issues early on in the build process. We discourage submitting daily or continuous integration builds as those provide limited usefulness.

Build Test Suite analysis

BTS runs automatically on Google’s infrastructure in the background. The process depends on internal services that cannot be made available offline.

When BTS finishes, the results appear on APFE.

Build approval

If BTS does not detect any issues, then the build approval process will proceed as normal, subject to the results of other test reports uploaded.

Any suspected PHAs or other issue will be treated as test failure and will prevent build approval. Google can only share the package name and class of malware for any PHA. We cannot provide details on the possible impact of the PHA, its behavior, or how that behavior was detected.

If Google finds an issue after the build has been approved, your TAM will contact you. Partners must address new issues promptly and release an updated build that fixes the issue in accordance with their partnership agreements. New builds that still contain the issue will not be approved.

If you believe that an APK was incorrectly flagged, escalate to your TAM and provide as much detail as possible about the intended functionality of the app. Include details about the file’s origin, particularly if it was made in-house or by a supplier. Google will investigate and deal with these apps on a case-by-case basis.

Supported software build formats

Fastboot package

BTS requires the following partition images in a .zip (PKZIP) or .tgz (gzipped unix .tar) file:

  • system.img – The filesystem image mounted at /system on the device.
  • vendor.img – The filesystem image mounted at /vendor on the device.
  • oem.img – The filesystem image mounted at /oem on the device.
  • userdata.img – The filesystem image mounted at /data on the device. This is especially important if you pre-load or cache applications in the data partition to improve the out-of-box (OOB) boot time.
  • boot.img – The Linux Kernel and Ramdisk used to boot Android.
  • recover.img – The recovery image that the Linux Kernel and Ramdisk use for the device to boot to during recovery or to handle the command adb reboot recovery.
Note: The vendor.img and oem.img are only required if the build uses separate vendor and oem partitions, otherwise it is expected this content is part of the system image.

The images and filenames above are produced by the AOSP build system, and they can be applied to an unlocked OEM device using fastboot if the device’s bootloader support fastboot.

These filesystem images should be in Ext4, SquashFS, or F2FS format. Partners should use the sparse format for the partition image to reduce the space and upload size for filesystems with free space on them.

Other firmware formats

Google supports the following firmware formats used by various OEM, ODM, and SOC flashing tools:

  • Mediatek firmware used with Mediatek’s spflashtool
  • Spreadtrum PAC file format1
  • Qualcomm’s QFIL format
  • .NB0 format

Partners whose firmware is available in one of these formats may upload those formats, in a .zip, .tgz, or .rar file, instead of a fastboot archive. If you cannot provide firmware in one of the known formats, reach out to your TAM.

Note: Google does not accept firmware in self-extracting executables and does not accept password protected archive files.

Requesting an SFTP account

Larger OEMs may prefer submitting builds using SFTP because

  • SFTP does not require Google Drive quota.
  • BTS begins as soon as the upload completes.
  • Partners can use a wide range of SFTP client software.
  • SFTP is easier to integrate into automated build, QA review, and submission processes.

Contact your TAM to set up an SFTP account for uploading firmware images. You will need to generate an SSH2 RSA identity key pair for authentication and share the public key with your TAM. Once your SFTP account and dropbox have been configured, your TAM will provide you with the connection details for your SFTP account.

Generating an SSH Identity key pair

On Linux, use the following command, replacing “my_key” with a suitable key name:

$ ssh-keygen -t rsa -f my_key

This command will generate a private key, my_key, and a public key, my_key.pub. Your TAM will need the public key to set up your SFTP account.

Partners using a Microsoft Windows system can use the PuTTY client to generate SSH keys following these instructions.

Notes


  1. It is strongly recommended to compress Spreadtrum ‘.pac’ files to reduce the amount of data that needs to be uploaded. 

CTS 9.0 R1 released, new Android 9 platform and partner docs available

Android 9

Android 9 is the release of the development milestone code-named P. The source code for the following tests, including tests for instant apps, can be synced with the ‘android-cts-9.0_r1’ tag in the open-source tree.

android.hardware.cts.CameraTest#testJpegExif

[DESCRIPTION]
CTS测试时,android.hardware.cts.CameraTest 中的testJpegExif 测项fail ,测试报告一般会报出如下类似的错误:

junit.framework.AssertionFailedError: expected:<3.5> but was:<3.3> at android.hardware.cts.CameraTest.testJpegExifByCamera(CameraTest.java:843)

[ANALYSE]

该项cts测试主要是测试拍照时,拍出来的jpeg图片中的focal length的值(最终读取sensor 的相关设定)是否与parameters中的值一致,若不一致则fail。

由于目前平台上面parameters中的focal length只有一个默认的初始赋值,而没有真正从sensor driver去query,所以若要修改sensor设定的focal length的值,也要同步修改parameters中的focal length值,并且前后摄像头的这个值也需要保持一致。

继续阅读“android.hardware.cts.CameraTest#testJpegExif”

android.jvmti.cts.JvmtiHostTest911#testJvmti Fail

[DESCRIPTION]
D/ModuleListener: ModuleListener.testFailed(android.jvmti.cts.JvmtiHostTest911#testJvmti, junit.framework.AssertionFailedError: [android.jvmti.cts.JvmtiRunTestBasedTest#testRunTest java.lang.IllegalStateException: ###################
### Same thread ###
###################
 这项测试失败主要是因为手机里面的zygote进程创建进程时,所属进程中的线程不符合Google的要求。
一般是由客户自己加的功能引起的。
[SOLUTION]

继续阅读“android.jvmti.cts.JvmtiHostTest911#testJvmti Fail”

Test suites GMS July release GMS Client ID update

These versions will be required for GMS approvals as described in the table below:

Release Release version Required-use date
CTS 8.1 R7, 8.0 R11, 7.1 R19, 7.0 R23, 6.0 R30 2018-8-21
VTS 8.1_r4 and 8.0_r8 w August SPLs 2018-8-21
GMS 8.1_201807, 8.0_201807 2018-9-18

GMS Client ID update
Updated H3G country list to include ITWe’ve updated the GMS Client ID mapping (direct link), effective for new devices only. Check with specific carriers to confirm which values to set. Here’s a summary of the changes:

  • Removed AT, BE, CH, FR from Vodafone list of countries
  • Added a row for Vodafone AU

谷歌近期政策通知

谷歌最新政策如下:

1、express+设备MR也可以享受谷歌资金支持;

2、谷歌将在8月份开始释放Android 9.X版本, go的版本也同时放出;

3.Android 8.X的截止日期为2018-12-31,2019年起仅能送测Android 9.X项目;

4.2018年10月1日起,谷歌强制执行STS及BTS,相应的补丁会每月更新一次。

CtsIntentSignatureTestCases android.signature.cts.intent.IntentTest#shouldNotFindUnexpectedIntents FAIL

[DESCRIPTION]
CtsIntentSignatureTestCases android.signature.cts.intent.IntentTest#shouldNotFindUnexpectedIntents

Fail:
java.lang.AssertionError: [Package: com.android.systemui Invalid Intent: [android.intent.action.ACTION_SUBSIDYLOCK_STATE]]

[SOLUTION]

继续阅读“CtsIntentSignatureTestCases android.signature.cts.intent.IntentTest#shouldNotFindUnexpectedIntents FAIL”

Android 8.1 Google issue

Android 8.1 Google issue
特别注意:
1.有link的Google issue ,不需要再来申请分析报告,用link申请waive.
2.已得到Google回复的: You can get a waiver .

3.已提交google等待google 回复的: Waiting for google feedback.
所有Waiting for Google feedback的Google issue, 均需要客户与Google 确认是否可以waive, MTK 亦在努力与Google 沟通中,一旦拿到Google waiver, 会修改成:You can get a waiver
以下Google issue,不是全部Google issue, 其它fail 需要提供Log 来确认
同一个google issue可能在多个CTS版本 一直存在,因此豁免版本不仅限于标注的CTS版本。同一条case在CTS/CTS-ON-GSI测试下,fail root cause相同,则是同一个问题,同一个waive ID;

继续阅读“Android 8.1 Google issue”

CtsNetTestCases android.net.wifi.cts.NsdManagerTest#testNDSManager FAIL

[DESCRIPTION]
CtsNetTestCases android.net.wifi.cts.NsdManagerTest#testNDSManager
04-05 15:12:49 I/ConsoleReporter: [1/1 arm64-v8a CtsNetTestCases HGAE5WKC] android.net.wifi.cts.NsdManagerTest#testNDSManager fail: junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at android.net.wifi.cts.NsdManagerTest.testNDSManager(NsdManagerTest.java:361)
at java.lang.reflect.Method.invoke(Native Method)
[SOLUTION]

继续阅读“CtsNetTestCases android.net.wifi.cts.NsdManagerTest#testNDSManager FAIL”

android.hardware.consumerir.cts.ConsumerIrTest#test_timing fail

[DESCRIPTION]
run cts -m CtsHardwareTestCases -t android.hardware.consumerir.cts.ConsumerIrTest#test_timing
fail
06-12 15:34:34 D/ModuleListener: ModuleListener.testStarted(android.hardware.consumerir.cts.ConsumerIrTest#test_timing)
06-12 15:34:34 D/ModuleListener: ModuleListener.testFailed(android.hardware.consumerir.cts.ConsumerIrTest#test_timing, junit.framework.AssertionFailedError: Pattern length pattern:499995000, actual:926538
at junit.framework.Assert.fail(Assert.java:50)
at junit.framework.Assert.assertTrue(Assert.java:20)
at android.hardware.consumerir.cts.ConsumerIrTest.test_timing(ConsumerIrTest.java:94)
at java.lang.reflect.Method.invoke(Native Method)
[SOLUTION]

继续阅读“android.hardware.consumerir.cts.ConsumerIrTest#test_timing fail”

android.media.cts.CamcorderProfileTest#testGetWithId fail

[DESCRIPTION]
run cts -m CtsMediaTestCases – t android.media.cts.CamcorderProfileTest#testGetWithId
fail :
06-11 11:31:08 D/ModuleListener: ModuleListener.testStarted(android.media.cts.CamcorderProfileTest#testGetWithId)
06-11 11:31:09 D/ModuleListener: ModuleListener.testFailed(android.media.cts.CamcorderProfileTest#testGetWithId, junit.framework.AssertionFailedError: expected:<4500000> but was:<24000000>
at junit.framework.Assert.fail(Assert.java:50)
at junit.framework.Assert.failNotEquals(Assert.java:287)
at junit.framework.Assert.assertEquals(Assert.java:67)
at junit.framework.Assert.assertEquals(Assert.java:199)
at junit.framework.Assert.assertEquals(Assert.java:205)
at android.media.cts.CamcorderProfileTest.assertProfileEquals(CamcorderProfileTest.java:115)
at android.media.cts.CamcorderProfileTest.checkSpecificProfiles(CamcorderProfileTest.java:240)
[SOLUTION]

继续阅读“android.media.cts.CamcorderProfileTest#testGetWithId fail”

android.bluetooth.cts.BluetoothLeScanTest#testBasicBleScan fail

[DESCRIPTION]

android.bluetooth.cts.BluetoothLeScanTest

— testBasicBleScan

–testBatchScan

— testOpportunisticScan

— testScanFilter

Fail信息如下:

fail junit.framework.AssertionFailedError: Scan results shouldn’t be empty at junit.framework.Assert.fail(Assert.java:50)

[SOLUTION]
首先,请确保合入patch:ALPS03646285

继续阅读“android.bluetooth.cts.BluetoothLeScanTest#testBasicBleScan fail”

关于USB Accessory Test测试事项

[DESCRIPTION]
一、USB Accessory Test测试,不要求测试机端支持OTG;
因此取消usb.host不会影响此项测试;

<meta-data android:name=”test_category” android:value=”@string/test_category_hardware” />

<meta-data android:name=”test_required_features” android:value=”android.hardware.usb.accessory” />

<meta-data android:name=”test_excluded_features”

二、USB Accessory Test测试方法:
1、Install the Cts Vefifier USB Companion app on a separate helper device;
2、Start the accessory test companion in the Cts Verifier USB compannion;
3、Connect the two devices, if using a otg adapter make sure the adapter directly conected to the helper device.if using an type-c cable make sure that the helper device is set as supply power to the attached device.

[SOLUTION]

继续阅读“关于USB Accessory Test测试事项”

android.media.cts.AudioTrackLatencyTest#testPlaySmallBuffer test fail

[DESCRIPTION]

android.media.cts.AudioTrackLatencyTest#testPlaySmallBuffer test fail
junit.framework.AssertionFailedError: testPlaySmallBuffer: did it play all the data? expected:<1539> but was:<1024>
at junit.framework.Assert.fail(Assert.java:50)
at junit.framework.Assert.failNotEquals(Assert.java:287)
at junit.framework.Assert.assertEquals(Assert.java:67)

[SOLUTION]

继续阅读“android.media.cts.AudioTrackLatencyTest#testPlaySmallBuffer test fail”

[CTS Test]项目不支持OTG,如何去除CtsVerifier测试中USB相关的测试项

[DESCRIPTION]
AudioFrequencyMicrophoneTest
AudioFrequencySpeakerTest
AudioFrequencyUnprocessedTest
这三条需要连接USB microphone麦克风才能测试
USB Audio Peripheral Attributes Test
USB Audio Peripheral Buttons Test
USB Audio Peripheral Play Test
USB Audio Peripheral Record Test
这四条需要连接USB Audio headset耳机才能测试

继续阅读“[CTS Test]项目不支持OTG,如何去除CtsVerifier测试中USB相关的测试项”

android-cts-8.1_r3 execution is not starting

Q:

On starting, facing the below issue[followed all the precondition steps]

03-29 04:49:11 W/DynamicConfigPusher: Cannot download and parse json config from URL https://androidpartner.googleapis.com/v1/dynamicconfig/suites/CTS/modules/CtsMediaStressTestCases/version/8.1_r3?key=AIzaSyAbwX5JRlmsLeygY2WWihpIJPXFLueOQ3U 03-29 04:49:11 I/MediaPreparer: Instrumenting package android.mediastress.cts.preconditions.app: 03-29 04:49:17 I/MediaPreparer: Downloading media files from https://dl.google.com/dl/android/cts/android-cts-media-1.4.zip 03-29 05:01:42 E/ModuleDef: TargetSetupError in preparer: com.android.compatibility.common.tradefed.targetprep.MediaPreparer 03-29 05:01:42 E/ModuleDef: Precondition class com.android.compatibility.common.tradefed.targetprep.MediaPreparer failed 03-29 05:01:44 I/MediaPreparer: Instrumenting package android.mediastress.cts.preconditions.app: 03-29 05:01:50 I/MediaPreparer: Downloading media files from https://dl.google.com/dl/android/cts/android-cts-media-1.4.zip

This is happening continuously. No test case run

继续阅读“android-cts-8.1_r3 execution is not starting”

Android GO GMS认证 

一、2018年最新的GMS需提供5份测试报告,其中包括以下测试:

1、正式版本的CTS测试  (正式版本指的是要拿来送认证的的版本)

2、正式版本的CtsVerifier测试

3、正式版本的GTS测试

4、GSI user 版本的VTS测试

5、GSI user版本的CTS测试

具体测试步骤查请看后面几点

继续阅读“Android GO GMS认证 ”

[CTS_8.1] android.autofillservice.cts.PreSimpleSaveActivityTest#testTapLink_changeOrientationThenTapBack android.autofillservice.cts.SimpleSaveActivityTest#testTapLink_changeOrientationThenTapBack

如下2条Case 可以申请豁免
android.autofillservice.cts.PreSimpleSaveActivityTest#testTapLink_changeOrientationThenTapBack
android.autofillservice.cts.SimpleSaveActivityTest#testTapLink_changeOrientationThenTapBack
Fail info
java.lang.AssertionError: negative button (NO THANKS): Not true that the subject is a non-null reference
[SOLUTION]

继续阅读“[CTS_8.1] android.autofillservice.cts.PreSimpleSaveActivityTest#testTapLink_changeOrientationThenTapBack android.autofillservice.cts.SimpleSaveActivityTest#testTapLink_changeOrientationThenTapBack”

IPV6 Configuration For Google GMS Test

谷歌官网对于CTS测试网络要求如下:(查看官网要求

WLAN 和 IPv6

CTS 测试需要满足以下要求的 WLAN 网络:支持 IPv6,可以将被测设备 (DUT) 视为隔离客户端,并可以连接到互联网。隔离客户端是一种配置,可使 DUT 无法接收子网络上的广播/多网消息;这种配置可通过 WLAN AP 配置或通过在未连接其他设备的隔离子网络上运行 DUT 来实现。

如果您无法访问原生 IPv6 网络、IPv6 运营商网络或 IPv6 VPN,以致无法通过基于 IPv6 的一些测试,则可以改为使用 WLAN 接入点和 IPv6 隧道。请参阅维基百科 IPv6 隧道代理列表

实际测试中很多小伙伴们没有符合要求的网络,大多数连能连接谷歌的网络也没有,就造成很多网络原因的测试失败项。

怎么配置符合要求的IPV6网络?

继续阅读“IPV6 Configuration For Google GMS Test”

no permissions (udev requires plugdev group membership)

在 ubuntu 64位機終端上輸入  adb devices

顯示如下的不愉快:

0123456789ABCDEF    no permissions (udev requires plugdev group membership); see [http://developer.android.com/tools/device.html]

這會導致無法測試 CTS 和 GTS。查找 android 官方資料,解決方法如下:

继续阅读“no permissions (udev requires plugdev group membership)”

Android8.1(O1)CTS失败项纪录

1、CtsLibcoreTestCases libcore.java.net.SocketTest#testSocketTestAllAddresses

  • 网络问题,需要在IPV6的环境下进行测试

2、CtsLocationTestCases android.location.cts.GnssPseudorangeVerificationTest#testPseudoPosition

  • 在测试之前确保工模下有搜到卫星信号,可以在笔记本上设定cts 环境,然后在户外测试。或者使用室内GPS信号放大器

3、CtsKeystoreTestCases android.keystore.cts.KeyAttestationTest#testEcAttestation

  • 该项测试需要申请google key

继续阅读“Android8.1(O1)CTS失败项纪录”

配置CTS测试环境

设置 CTS

物理环境


蓝牙 LE 信标

如果 DUT 支持蓝牙 LE 功能,则应在与 DUT 的距离不超过五米的范围内放置至少三个蓝牙 LE 信标,以进行蓝牙 LE 扫描测试。这些信标可以为任何类型,不需要进行配置或发射任何特定信号,并且可以包括 iBeacon、Eddystone,甚至模拟 BLE 信标的设备。

GPS/GNSS

如果 DUT 支持全球定位系统 (GPS)/全球导航卫星系统 (GNSS) 功能,则应该以合适的信号电平向 DUT 提供 GPS/GNSS 信号(GPS 部分符合 ICD-GPS-200C 标准),以便其接收到相应信号并计算 GPS 位置。GPS/GNSS 信号源的种类不限(可以是卫星模拟器,也可以是室外 GPS/GNSS 信号中继器),只需将 DUT 放在距离窗口足够近的位置以使其可以直接接收到足够强的 GPS/GNSS 信号即可。

WLAN 和 IPv6

CTS 测试需要满足以下要求的 WLAN 网络:支持 IPv6,可以将被测设备 (DUT) 视为隔离客户端,并可以连接到互联网。隔离客户端是一种配置,可使 DUT 无法接收子网络上的广播/多网消息;这种配置可通过 WLAN AP 配置或通过在未连接其他设备的隔离子网络上运行 DUT 来实现。

如果您无法访问原生 IPv6 网络、IPv6 运营商网络或 IPv6 VPN,以致无法通过基于 IPv6 的一些测试,则可以改为使用 WLAN 接入点和 IPv6 隧道。请参阅维基百科 IPv6 隧道代理列表

台式机设置

继续阅读“配置CTS测试环境”