Run tests in Elastic Execution Grid

Elastic Execution Grid is an early access feature and available as a tech preview.

Click here if you'd like to participate in the tech preview.

Once an admin has enabled the integration, Tosca users can run their Tosca tests in Elastic Execution Grid.

Basic workflow

To run tests in Elastic Execution Grid, follow the steps below:

  1. Create ElasticExecutionGridEvents out of your ExecutionLists.

  2. Set Characteristics to determine which Elastic Execution Grid Agents execute your tests.

  3. Define how Elastic Execution Grid Agents should run your tests.

  4. Push your ElasticExecutionGridEvents to Elastic Execution Grid.

  5. View results.

Create ElasticExecutionGridEvents

An ElasticExecutionGridEvent is a collection of ExecutionLists that you want to execute in Elastic Execution Grid.

To create ElasticExecutionGridEvents, follow the steps below:

  1. In Tricentis Tosca, check out the project root element.

    This is an important step. Checking out the project root element ensures that Tricentis Tosca adds all necessary property definitions when you create the ElasticExecutionGridEvents folder.

  2. Create an ElasticExecutionGridEvents folder. To do so, right-click the Execution folder and select Create ElasticExecutionGridEvents folder from the mini toolbar. You only need to do this once.

Create a new ElasticExecutionGridEvents folder

  1. Create a new ElasticExecutionGridEvent. To do so, right-click the ElasticExecutionGridEvents folder and select Create ElasticExecutionGridEvent from the mini toolbar.

  2. Drag and drop the ExecutionLists that you want to run in Elastic Execution Grid onto the ElasticExecutionGridEvent.

  3. Create as many ElasticExecutionGridEvents as you need.

Set Characteristics

Characteristics determine which Elastic Execution Grid Agents execute your tests.

When admins set up Elastic Execution Grid Agents (see the Elastic Execution Grid manual), they define Agent Characteristics. Typically for the operating system and installed applications.

Let's say your tests require Microsoft Excel, but only two Elastic Execution Grid Agents have Excel installed. In this case, the admin gives both an "Excel" Characteristic.

In Tricentis Tosca, you specify which Characteristics you want to apply to your tests. If you have tests that involve Microsoft Excel, apply the "Excel" Characteristic to them. Elastic Execution Grid forwards these tests only to the "Excel" Elastic Execution Grid Agents.

Only Elastic Execution Grid Agents that match all Characteristics that you specify for your tests can execute them.

Make sure that the combination of Characteristics you select in Tosca is available in Elastic Execution Grid.

To set Characteristics for your tests, follow the steps below:

  1. Select an ElasticExecutionGridEvent.

  2. Open the Properties pane.

  3. For the property AgentCharacteristics, enter the following information into the Value field: <Name of the Characteristic>=<Characteristic value>.

    For instance, Browser=Firefox.

    If you're unsure about Characteristic names and values, contact an Elastic Execution Grid admin. Names and values depend on what the admin defined during the Elastic Execution Grid Agent setup.

  1. You can specify multiple values for the same Characteristic name. To do so, separate the values by comma. For instance, Browser=Firefox,Chrome.

    You can also specify multiple Characteristics with different names. To do so, separate the names by semicolon. For instance, Browser=Chrome,Explorer;InstalledApps=SAP.

Let's say you have an ElasticExecutionGridEvent called GUI_tests. To execute the tests in this ElasticExecutionGridEvent, Elastic Execution Grid Agents must have Salesforce, Chrome, and Firefox installed.

You have 3 Elastic Execution Grid Agents.

Agent A has the following Characteristics:

  • One Characteristic named InstalledApps with the value SAP.

  • Two Characteristics named Browser, one with the value Chrome and one with the value Firefox.

Agent B has the following Characteristics:

  • One Characteristic named InstalledApps with the value Salesforce.

  • One Characteristic named Browser with the value Chrome.

Agent C has the following Characteristics:

  • One Characteristic named InstalledApps with the value Salesforce.

  • Two Characteristics named Browser, one with the value Chrome and one with the value Firefox.

To find the right Agent for execution, you add the following value to the property AgentCharacteristics of the ElasticExecutionGridEvent GUI_tests:

InstalledApps=Salesforce;Browser=Chrome,Firefox

At run-time, Agent C receives the tests for execution, since it's the only Agent with all required Characteristics.

Define how Elastic Execution Grid Agents should run your tests

With Characteristics, you define which Elastic Execution Grid Agents should execute your tests.

You can also define how these Agents should execute your tests.

Let's say you have two Elastic Execution Grid Agents that are suitable for a particular ElasticExecutionGridEvent. Should only one Agent execute all TestCases? Or can you split tests between Elastic Execution Grid Agents?

Elastic Execution Grid offers the following execution types:

  • SequentialSingleMachine: one Agent executes all TestCases in sequence.

  • SequentialMultipleMachines: TestCases are split between Agents, but executed in sequence. This speeds up execution by distributing test efforts between Agents.

  • Parallel: multiple Agents execute the TestCases within the ExecutionList in parallel. This is the fastest way to execute. However, Agents don't execute tests in sequence. Only choose this option if there are no dependencies between your TestCases.

By default, all ExecutionLists use the type SequentialSingleMachine.

If you want Elastic Execution Grid to execute an ExecutionList differently, follow the steps below:

  1. Select the ExecutionList.

  2. In the Properties pane of the ExecutionList, select a different value for ExecutionFlow.

You need to set the property in the original ExecutionList.

You can’t do this in the ExecutionList within the ElasticExecutionGridEvent.

Select a value for the property ExecutionFlow

Push ElasticExecutionGridEvents to Elastic Execution Grid

How you push tests to Elastic Execution Grid depends on how the admin has setup up the integration:

  • Integration via Elastic Execution Grid Addin.

  • Integration via Automation Object Service (AOS).

If your admin has set up integration via Elastic Execution Grid Addin, follow the steps below:

Now that you have associated your ElasticExecutionGridEvents to ExecutionLists and the environment setup is complete, you can push them to the Elastic Execution Grid. To do so, follow the steps below:

  1. Ensure that the tests you want to push aren't empty.

    • Each ElasticExecutionGridEvent must have at least one ExecutionList.

    • Each ExecutionList must have at least one TestCase.

  2. Right-click one or more ElasticExecutionGridEvents and select Execute from the context menu.

  3. In the subsequent window, sign in to Elastic Execution Grid with your Elastic Execution Grid credentials.

    You can also sign in or out separately. To do so, right-click the ElasticExecutionGridEvents folder and select Log in to Elastic Execution Grid or Log out from Elastic Execution Grid from the context menu.

  1. If the push was successful, the next screen shows the following information:

    • Which ElasticExecutionGridEvents were pushed to Elastic Execution Grid.

    • Their corresponding ExecutionFlow IDs in Elastic Execution Grid.

  2. Click OK.

If your admin has set up integration via AOS, follow the steps below:

  1. Ensure that the tests you want to push aren't empty.

    • Each ElasticExecutionGridEvent must have at least one ExecutionList.

    • Each ExecutionList must have at least one TestCase.

  2. If you have made changes to your tests, check in your changes.

  3. To push your tests, right-click one or more ElasticExecutionGridEvents and select Execute and import results from the context menu.

Please note: Tricentis Tosca doesn't display any Elastic Execution Grid error messages. Instead, AOS logs all Elastic Execution Grid issues in C:\ProgramData\TRICENTIS\Logs\ToscaAutomationObjectService.

View results

Where you can view your results depends on how the admin has setup up the integration:

You can view your results in Elastic Execution Grid only.

You can view your results in Elastic Execution Grid and in Tricentis Tosca. Tricentis Tosca displays the results on the ExecutionLists.

Tricentis Tosca only displays the results of tests that you triggered in Tricentis Tosca. If you trigger executions directly in Elastic Execution Grid, Tricentis Tosca does not display those results.