Testing Salesforce with Playwright and Generative AI

Introduction

If you’re writing automated end-to-end tests for the Salesforce platform, you’re likely using Robot Framework, an open source automation framework that exposes its own DSL for creating automated tests.

In this blog post, we’ll cover how developers can get the benefits of a natural language approach like with Robot Framework, but without needing to conform to any predefined syntax, and with a more modern test automation framework.

Robot Framework

To start, let’s review an example Robot Framework test that creates a Contact in Salesforce. I took this example from the CumulusCI docs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


*** Settings ***
Resource        cumulusci/robotframework/Salesforce.robot
Documentation   A simple Robot test

Suite Setup     Open test browser
Suite Teardown  Delete records and close browser

*** Test Cases ***
Take screenshot of list of Contacts
   [Setup]  Create a test Contact

   Go to object home  Contact
   Capture page screenshot

*** Keywords ***
Create a test Contact
   [Documentation]  Create a temporary Contact and return the ID
   [Return]         ${contact id}

   ## Generate a name to use for Contact
   ${first name}=   Get fake data  first_name
   ${last name}=    Get fake data  last_name

   ## Create a new Contact
   ${contact id}=   Salesforce Insert  Contact
   ...  FirstName=${first name}
   ...  LastName=${last name}

Robot Framework exposes a natural language DSL, and includes a Salesforce module that includes a host of pre-built actions. As you can see, the syntax of this DSL is very specific to Robot Framework, and quite different from writing code in a language like JavaScript and TypeScript.

Unfortunately, popular testing tools like Playwright have poor support for Salesforce. Let’s cover how with a dose of Generative AI, it is possible to write Playwright tests for Salesforce that incorporate some of the natural language aspects of Robot Framework.

Challenges with using Playwright

Even though Playwright is a popular and well-supported testing framework, it’s historically been quite difficult to make it work when testing Salesforce applications. This has to do with the underlying complexity of Salesforce’s Lightning component framework, which is used by both the Salesforce platform itself, as well as custom applications running in the Salesforce ecosystem.

Lightning, among other things, overrides core JavaScript APIs and implements its own non-standards-compliant version of Shadow DOM. These architectural choices all contribute to making it incredibly difficult to target elements on the page using CSS/XPath selectors. Further, because Salesforce is a closed ecosystem, you often can’t add data-test-ids or other markup to make it easier for test automation to locate elements.

This is unfortunate, because Playwright has some nice benefits over Robot Framework:

Playwright uses common web technologies like JavaScript/TypeScript, meaning it’s easy for web developers to learn. Whereas, Robot Framework abstracts away the underlying code behind a custom DSL.
Playwright has first-class IDE support, including well-supported plugins for VSCode.
Playwright includes its own Test Runner, which provides a lot of convenience functions for building tests and lots of options for viewing test results.

Writing Salesforce tests with Playwright + AI

Using the new ZeroStep AI library for Playwright, we can execute actions and assertions in Playwright in natural language, while still having the flexibility of a code-first workflow.

Below is a complete Playwright test that uses ZeroStep’s ai() function to create a new Opportunity within the Salesforce Sales app:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48


import { test, expect } from '@playwright/test'
import { ai } from '@zerostep/playwright'

const email = 'test@example.com'
const password = 'passwordhere'
const hostname = 'realhostnamehere.develop.lightning.force.com'

test.describe('Salesforce', () => {
  test('create an opportunity', async ({ page }) => {
    await page.goto('https://login.salesforce.com')
    await ai(`Enter the username ${email}`, { page, test })
    await ai(`Enter the password ${password}`, { page, test })
    await page.click('text="Log In"')

    // Only reaches here if we are successfully authenticated
    await page.waitForSelector('text="Home"')

    // Navigate directly to Sales app
    await page.goto(`https://${hostname}/lightning/page/home`)
    await page.waitForSelector('text="Quarterly Performance"')

    await ai('Click on Opportunities link', { page, test })
    await page.click('text="New"')

    // Wait for 'New Opportunity' form to be displayed
    await page.waitForSelector('text="New Opportunity"')

    await ai(`Enter '12000' in the Amount field.`, { page, test })
    await ai('Enter Test in the opportunity name input', { page, test })

    const thirtyDaysFromNow = new Date()
    thirtyDaysFromNow.setDate(thirtyDaysFromNow.getDate() + 30)

    const closeDate = thirtyDaysFromNow.toLocaleDateString('en-US', {
      month: '2-digit',
      day: '2-digit',
      year: 'numeric',
    })

    await ai(`Input ${closeDate} into the Close Date field`, { page, test })
    await ai('Click on the Stage dropdown', { page, test })
    await ai('Click on the Needs Analysis option', { page, test })
    await ai('Click Save', { page, test })

    const result = await ai('What is the current stage of the opportunity?', { page, test })
    expect(result).toEqual('Needs Analysis')
  })
})

As you can see, this examples uses both built-in Playwright functions (page.waitForSelector, page.goto) and the ZeroStep ai() function to drive the browser and create an Opportunity in Salesforce.

Here’s how the ai() function works:

The “prompt” passed in the ai() function is sent to the ZeroStep backend, along with metadata about the current state of the page
The ZeroStep AI first determines whether the prompt is one of the following types:

Action: Executes one or more actions in the web browser (e.g. Click on the Stage dropdown)
Assertion: Evaluates the prompt as a boolean operation (e.g. Assert that the current stage is "Needs Analysis")
Query: Answers a question about the state of the application (e.g. What is the current stage of the opportunity?)

Next, the ZeroStep AI determines what actions should be taken within the browser if any, and emits that to the JavaScript library.
The library executes those actions as Playwright commands.
If the prompt was evaluated as an assertion or query, the ZeroStep AI returns the true/false response or answer to the question.

Using this approach, developers can get the benefits of a natural language approach like with Robot Framework, but without needing to conform to any predefined syntax. And since this is just a JavaScript/TypeScript application, you can include additional npm packages like the jsforce library for calling the Salesforce API.

Thanks for reading, and happy testing!

Table of contents

Introduction

Robot Framework

Challenges with using Playwright

Writing Salesforce tests with Playwright + AI