First step on Firebase Test Lab with iOS Project

Firebase Test Lab is a cloud-based app-testing infrastructure. With one operation, you can test your Android or iOS app across a wide variety of devices and device configurations, and see the results — including logs, videos, and screenshots — in the Firebase console.

To ensure your app quality, Firebase Test Lab provides you with physical and virtual devices that allow you to run tests that simulate actual usage environments.

Build your project to test

To test Firebase Test lab, you can find a sample project here :

kevinabrioux/Testlab

This app is composed of a label incremented by a button.

To make the application testable by Firebase, we need to compile it for Generic iOS Device

Ensure to select ‘Generic iOS Device’

After that, you can create a build for testing :

When your build is successful, go in your derived data, usually in

/Users/<your account>/Library/Developer/XCode/DerivedData

In your derived data, find the current project and open Build/Product :

Example of path

You will find all the files generated :

Exemple of content

Testing your build

TestLab uses physical devices to run your test. To make sure your build works correctly, try it with this command line :

xcodebuild test-without-building \
    -xctestrun <yourfile.xctestrun> \
    -destination id=<iphoneID>

where the Xctestrun’s file is found in Build/Product folder.

To get the iPhoneID, use instruments -s devices command to get the list of all devices. Select the physical devices where the code must be run.

Example of device’s list

With xcodebuild test-without-building command, you will see your physical device launch your tests and you will be able to see logs and results in your console.

Example of test running on physical device

Create your iOS matrix’s testlab

Go in your firebase console and create a new XCTest

Select Run an XCTest

Compress your Xctestrun’s file and Debug-iphoneos folder and upload it to firebase.

Upload your test’s build

Select the physical devices where code must be run, beware of the iOS minimum version supported by your project :

Some available OS are deprecated and will be removed

Actually, (17/12/19) the max OS version supported is iOS 12.3. So if your app is targeting iOS 13, test lab will skip your test :

Test will be skipped if iOS’s version is too high

Finally select the orientation and device’s locales

Select orientations and locales

Start your test and wait for results

At the end of the test, you can see your test’s results :

More interesting if you have UI’s tests, you can see see the video of all your UI tests performed :

UI tests scenario

If you go to Test results, you will be redirected to the Google cloud Bucket where all the generated files associated to these tests are stored :

Bucket associated to this test

So you can download the generated video, crashlogs or testlogs to open it in your XCode IDE.

Conclusion

Firebase test lab is a wonderful tool to release your mac from UI tests and scale on multiple device’s size, type and OS.

You can integrate it to your actual CI to improve your quality.

It’s easy to use and all the generated reports are very useful to find where and when your app failed.