AWS Device Farm Cheat Sheet

• AWS Device Farm allows you to examine and interact with your Android, iOS, and web applications on actual, physical devices maintained by Amazon Web Services (AWS).

Key Features

• Automated App Testing – Device Farm provides the functionality to either upload your personalized tests or utilize the pre-existing, script-free compatibility tests. The testing process is executed concurrently, which enables tests on various devices to commence within minutes. Upon the completion of tests, a test report is updated, which includes a summary of results, detailed logs, pixel-by-pixel screenshots, and performance metrics.
• Remote Access Interaction – The remote access feature enables you to interact with a device in real-time through your web browser. This capability can be particularly useful in various scenarios, such as assisting customers in navigating or setting up their devices.
• Real-World Environments – You can fine-tune your test environment by configuring location, language, network connection, and application data and installing prerequisite apps to simulate real-world customer conditions.
• Integration with Development Workflow – Device Farm supports integration with your development workflow using service plugins and API to automatically initiate tests and get results from IDEs and continuous integration environments.
• Custom Environments for Web Testing – Device Farm supports testing web apps on desktop browsers (Chrome, Firefox, Edge) with configurable versions and environments.
• Screenshot Comparison & Baseline Tracking – Automated tests can perform visual regression using step-by-step screenshot comparisons.
• Video Capture – Device Farm automatically records high-resolution videos of every automated and manual session.

Terminology

• Device Pool: This refers to a collection of devices that usually have common features, like the same platform, made by the same manufacturer, or of the same model.
• Job: A request for Device Farm to test a single app against a single device.
• Metering: Refers to billing for devices.
• Project: This refers to a logical workspace that encompasses runs, with each run representing a test of a single application against one or more devices.
• Report: Contains information about a run, which is a request for Device Farm to test a single app against one or more devices.

Device Selection

• • • Provides a broad selection of desktop browsers and mobile devices, including iOS, Android, and Fire OS devices.
    • Choose from a large, constantly updated collection of physical devices in the AWS Cloud, or supply and manage your own device fleet.
    • Private Devices: You can reserve dedicated devices that are isolated for your team only, ensuring consistent results and higher availability.
    • Instance Types: Device Farm allows you to choose between standard and high-performance device types for performance-heavy workloads.

Test Types

• • • Supports a variety of automated testing frameworks such as Appium (Java, Python), Calabash, Espresso, XCTest, etc.
    • Supports manual testing with Remote Access.
    • Built-in Fuzz Testing: Device Farm can simulate random user events (taps, swipes, rotations) to discover crashes.
    • Built-in Explorer (Scriptless Testing): A fully automated crawler that navigates your app, detects screens, and logs crashes without writing scripts.
    • Web App Testing via Selenium: Supports Selenium WebDriver scripts for desktop browser automation.

Test Reports

• • • Provides detailed reports for each run, job, suite, and test.
    • The reports include logs, screenshots, videos, performance data, and more.
    • You can view the reports in the Device Farm console or download them for offline use.
    • Performance Metrics Included:
      • CPU usage over time
      • Memory consumption
      • Battery drain
      • Frame rendering time
      • Network throughput
    • Timeline View: Reports include a chronological breakdown of actions, logs, and issues to make debugging easier.
    • Downloadable Raw Logs: Device logs, performance traces, and network logs can be exported for custom analysis.

Pricing

• • • Pricing is based on the total time that your tests take to run.
    • You pay for device minutes at the end of the month.
    • The service is not subject to any upfront commitments or long-term contracts.

Security

• • • Ensures the security of your tests and data.
    • After a test run, AWS Device Farm cleans up the device so the next run starts in a known state.
    • The service also isolates each app during testing to prevent interactions with other apps.
    • Network Security: Traffic to/from devices can be controlled using VPC endpoints with AWS PrivateLink.
    • Device Sanitization: Device Farm wipes user data, resets network configuration, and reinstalls OS images where required.
    • IAM Integration: Fine-grained permissions allow developers, QA testers, and admins to have different access roles.

AWS Device Farm Cheat Sheet Reference:

https://docs.aws.amazon.com/devicefarm/latest/developerguide/welcome.html