Work with test runs and test cycles

In Tricentis Test Management for Jira, you use test runs to document the execution of a test case. They contain all the information about the execution state and test result of test steps at a given time. For that reason, you can have multiple test runs for the same test case.

To group your test runs into different categories, you use test cycles. These work like folders that help you organize your tests. For instance, these can be features, a specification, or any area of your development cycle.

Test cycles and test runs

Usually, test cycles are assigned to a product version. This can be one of the versions (releases) you have in your Jira project. If you don't work with product versioning, you have the option to leave a test cycle unversioned.

This topic takes you through the steps to work with test cycles and test runs.

Create test cycles

You can create test cycles in one of the following ways:

If you create test cycles from your test cases, Tricentis Test Management for Jira automatically creates test runs for each of the selected test cases. These test runs are then stored in the new test cycle.

To create a new test cycle with test runs, follow these steps:

  1. In the Apps section of your Jira project, select Test Management to open the Tricentis Test Management for Jira.

  2. Go to the Test Design tab.

  3. Select the checkboxes next to the test cases that you want to create test runs for.

  4. Select Create test run.

  5. In the subsequent dialog, choose Create a new test cycle.

  6. Fill out these details:

    • Name: specify a name for your test cycle. This can be a high-level objective, a feature, or a specification.

    • Version: specify the version (release) for this test cycle. If you don't work with product versioning, choose Unversioned.

    • Planned start date specify the date in which you plan to start your testing activities for this test cycle.

    • Planned end date specify the date in which you plan to finish all progress for this test cycle.

    • Assign cycle's test runs to a tester (optional): specify a Jira user to assign these test runs to.

    • Environment (optional): select the test run environment from the dropdown.

    • Build (optional): select the build from the dropdown.

  7. Select the Apply existing test case folder structure to new test runs option, if you want to maintain the same folder structure leading to the test case also in the test cycle. The folder that includes the selected test case, and all the folders that are above the selected test case arewill be copied to the test cycle. However, the test cases inside the folders that are above the selected test case aren'twill not be included in the test cycle.

  8. Select Create. You'll find your new test cycle in the Test Execution tab in Tricentis Test Management for Jira. It contains test runs for the test cases you selected.

  9. Optionally, create additional test runs for this test cycle.

To create an empty test cycle, follow these steps:

  1. In the Apps section of your Jira project, select Test Management to open the Tricentis Test Management for Jira.

  2. Go to the Test Execution tab.

  3. Select New test cycle.

  4. In the subsequent dialog, fill out these details:

    • Name: specify a name for your test cycle. This can be a high-level objective, a feature, or even a specification.

    • Version: specify the version (release) for this test cycle. If you don't work with product versioning, choose Unversioned.

    • Planned start date specify the date in which you plan to start your testing activities for this test cycle.

    • Planned end date specify the date in which you plan to finish all progress for this test cycle.

    • Assign cycle's test runs to a tester (optional): specify a Jira user to assign these test runs to.

    • Environment (optional): select the test run environment from the dropdown.

    • Build (optional): select the build from the dropdown.

  5. Select Create. You'll find your new empty test cycle in the Test Execution tab in Tricentis Test Management for Jira.

  6. Optionally, create test runs for this test cycle.

To create a linked test cycle, follow these steps:

  1. Open the Jira issue that you want to link to a new test cycle. You can link all Jira issue types except test cases.

  2. Select Cycle Traceability.

  3. In the Cycle Traceability section, use the dropdown to select the project you want to create the test cycle in.

  4. Select Create new linked test cycle.

  5. In the subsequent dialog, fill out these details:

    • Name: specify a name for your test cycle. This can be a high-level objective, a feature, or even a specification.

    • Version: specify the version (release) for this test cycle. If you don't work with product versioning, choose Unversioned.

    • Planned start date specify the date in which you plan to start your testing activities for this test cycle.

    • Planned end date specify the date in which you plan to finish all progress for this test cycle.

    • Assign cycle's test runs to a tester (optional): specify a Jira user to assign these test runs to.

    • Environment (optional): create or select an existing environment that covers all of your testing variables:
      • Select existing: choose an existing test run environment from the dropdown.
      • Create new: to add a new environment, enter a name for the environment and select the newly created name from the drop-down menu.

    • Build (optional): create or select an existing build to organize your tests by software builds:

      • Select existing - choose the desired build from the dropdown.

      • Create new - to add a new Build number, enter the Build number and select the newly Build number from the drop-down menu.

  6. Select Create. You'll find your new empty test cycle in the Test Execution tab in Tricentis Test Management for Jira.

  7. Optionally, create test runs for this test cycle.

If you create test runs recursively, Tricentis Test Management for Jira automatically creates test runs for the test cases in the current folder and the test cases in the folders below it. These test runs can be stored in the new test cycle or an existing one.

To create test runs recursively, follow these steps:

  1. In the Apps section of your Jira project, select Test Management to open the Tricentis Test Management for Jira.

  2. Go to the Test Design tab.

  3. Click the menu on the top folder for which you want to create test runs recursively (i.e., test runs for the test cases in the selected folder and the test cases in the folders below it) and select Create test runs recursively.

  4. In the subsequent dialog, do one of the following:

    • To use an existing test cycle, choose Select an existing test cycle and fill out these details:

      • Select version - choose the version (release) for this test cycle. If you don't work with product versioning, choose Unversioned.

      • Select an existing cycle/folder - choose the test cycle or folder you want to add these test runs to. You can only choose test cycles which are assigned to the specified version.

      • Assign cycle’s test runs to a tester (optional) - choose a Jira user to assign these test runs to.

      • Environment (optional) – create or select an existing environment that covers all of your testing variables:

        • Select existing- choose an existing test run environment from the dropdown.

        • Create new - to add a new environment, enter a name for the environment and select the newly created name from the drop-down menu.

      • Build (optional) – create or select an existing build to organize your tests by software builds:

        • Select existing - choose the desired build from the dropdown.

        • Create new - to add a new Build number, enter the Build number and select the newly Build number from the drop-down menu.

    • To create a new test cycle, choose Create a new test cycle, click Next and fill out these details:

      • Name: specify a name for your test cycle. This can be a high-level objective, a feature, or a specification.

      • Version: specify the version (release) for this test cycle. If you don't work with product versioning, choose Unversioned.

      • Planned start date specify the date in which you plan to start your testing activities for this test cycle.

      • Planned end date specify the date in which you plan to finish all progress for this test cycle.

      • Assign cycle's test runs to a tester (optional): specify a Jira user to assign these test runs to.

      • Environment (optional): create or select an existing environment that covers all of your testing variables:

        • Select existing - choose an existing test run environment from the dropdown.

        • Create new - to add a new environment, enter a name for the environment and select the newly created name from the drop-down menu.

      • Build (optional): create or select an existing build to organize your tests by software builds:

        • Select existing - choose the desired build from the dropdown.

        • Create new - to add a new Build number, enter the Build number and select the newly Build number from the drop-down menu.

  5. Select the Apply existing test case folder structure to new test runs checkbox, if you want to maintain the same folder structure of the test cases also in the test cycle.

  6. Select Create. You'll find your new empty test cycle in the Test Execution tab in Tricentis Test Management for Jira. It contains test runs for the test cases you selected.

Link existing test cycles to issues

As described above, you can create empty test cycles that are already linked to any Jira issue, except for test cases. But, if you already have a test cycle, you can still link it to an issue to track the connections between your testing needs and your test runs:

  • In the Test Execution tab, open an existing test cycle and select Link to issue. From here, you can attach as many Jira issues as you need to.

  • Go to the Jira issue you want to link to a test cycle and select Cycle Traceability, then select the project that contains the test cycle you want to attach. From here, you can attach as many test cycles as you need to.

Create test runs for a test cycle

To run your test cases in Tricentis Test Management for Jira, you need to create a test run for every test case you want to execute. Test runs are stored in test cycles, which you need to create before proceeding.

Note that you can select multiple test cases and use the floating actions bar to quickly create a test run.

To create test runs for an existing test cycle, follow these steps:

  1. In the Apps section of your Jira project, select Test Management to open the Tricentis Test Management for Jira.

  2. Go to the Test Design tab.

  3. Select the checkboxes next to the test cases that you want to create test runs for.

  4. Select Create test run.

    Note that you can also create and view test runs for an individual test case in the Test Case Panel.

  5. In the subsequent dialog, choose Select an existing test cycle.

  6. Fill out these details:

    • Select version: specify the version (release) for this test cycle. If you don't work with product versioning, choose Unversioned.

    • Select an existing cycle/folder: choose the test cycle or folder you want to add these test runs to. You can only choose test cycles which are assigned to the specified version.

    • Assign cycle's test runs to a tester: specify a Jira user to assign these test runs to. This is optional.

    • Environment (optional): create or select an existing environment that covers all of your testing variables:

      • Select existing: choose an existing test run environment from the dropdown.
      • Create new - to add a new environment, enter a name for the environment and select the newly created name from the drop-down menu.

    • Build (optional): create or select an existing build to organize your tests by software builds:

      • Select existing - choose the desired build from the dropdown.

      • Create new - to add a new Build number, enter the Build number and select the newly Build number from the drop-down menu.

  7. Select the Apply existing test case folder structure to new test runs option, if you want to maintain the same folder structure leading to the test case also in the test cycle. The folder that includes the selected test case and all the folders that are above the selected test case will be copied to the test cycle. However, the test cases in the folders that are above the selected test case will not be included in the test cycle.

  8. Select Done.

Note that you can define the Build and Test run environment dropdown options in execution settings.

You can also create test runs directly from a requirement, which helps focus your testing on the requirements you've already set.

Organize test cycles

If you have a large number of test runs in your test cycles, it is recommended to organize them in folders. You can add folders within test cycles, move them between cycles, move test cycles between versions, and reorder test cycles within the same version.

Organize test cycles with folders

If you have a large number of test runs in your test cycles and are finding it hard to keep a good overview, add an additional organizational layer: folders within your test cycles.

To add a folder to a test cycle, follow these steps:

  1. Go to the Test Execution tab and hover over the test cycle in the list.

  2. Select > New Folder and follow the instructions on the screen.

  3. To add one or more test runs to the folder, select them. Then select Move to folder from the floating actions bar.

Each folder has its own menu, where you can do these things:

  • Rename or delete the folder.

  • Create subfolders.

  • Clone the folder. This creates a duplicate of the folder and its contents in the same test cycle.

You can also drag and drop folders within the same test cycle. For example, to change their order or move them into a different parent folder.

Move folders to another test cycle

You can move a folder and all its subfolders to another test cycle. The test cycle can be in the same version or in a different version.

To move a folder to another test cycle, follow these steps:

  1. Go to the Test Execution tab and select the desired folder. If the folder includes subfolders, they will be moved together with the selected folder.

  2. Select > Move to.

  3. In the Select Version field, select the version to which you want to move the folder. This can be the current version or another version.

  4. In the Select an existing test cycle field, select the test cycle to which you want to move the folder. Tricentis Test Management for Jira only displays test cycles of the selected version. If you are looking for another test cycle, check if this test cycle is located in a different version.

  5. Click Move.

Move test cycles between versions

You can move test cycles between versions, by following these steps:

  1. Go to the Test Execution tab and hover over the test cycle in the list.

  2. Select > Edit.

  3. In the Version field, select the desired version.

  4. Select Update.

Reorder test cycles within the version

You can reorder the test cycles list within the version, by following these steps:

  1. Go to the Test Execution tab.

  2. In the left pane, expand the desired version to reveal its underlying test cycles then drag and drop your test cycles into the desired order.

Tricentis Test Management for Jira automatically saves the new order.

Reordering test cycles

Bulk update test runs

Whether you need to update the status of multiple test runs, or change who executed them, you can bulk update your test runs from within Tricentis Test Management for Jira.

If you need to make updates to more than one test run at a time, follow these steps:

  1. Go to the Test Execution tab.

  2. Select the test cycle you want to modify.

  3. Select the checkboxes next to the test runs that you want to update one or more fields in.

  4. Select Edit from the floating actions bar.

  5. In the subsequent dialog, choose the Attribute to update and then what to Replace current value with.

    You can update any of the following fields: Assignee, Components, Executed by, Planned dates, Priority, Status, Build, or Test run environment.

  6. When you're done with all the fields, select Update to finish.

Note that you can update all fields for test runs from within Tricentis Test Management for Jira, but you can't use the Jira bulk update function for test runs.

Clone test runs

Do you need to re-run a test? Maybe your last test run failed. Or, you want to copy a test run before you edit a few steps, so you don't have to start from scratch.

To quickly duplicate an existing test run, navigate to the Test Runs tab on the Test Case. Hover over the test case you want to clone to see the menu, then select Clone from the menu.

The test run you selected copies to a new test run with the same steps, settings, and attachments.

Delete test runs

No one's perfect. Maybe it was an inaccurate testing scenario, or maybe it was just a duplicate. Whatever the case is, we've made it easy to find and delete test runs you no longer need.

Delete test runs from a test cycle

To delete test runs from a test cycle, follow these steps:

  1. In the Apps section of your Jira project, select Test Management to open Tricentis Test Management for Jira.

  2. Go to the Test Execution tab.

  3. Select the test cycle you want to modify.

  4. Select the checkboxes next to the test runs that you want to delete.

  5. Select to delete the test run.

Delete test runs from a test case

To delete test runs from an individual test case, follow these steps:

  1. In the Apps section of your Jira project, select Test Management to open Tricentis Test Management for Jira.

  2. Go to the Test Design tab.

  3. Select the test case you want to modify and go to the Test Runs tab.

  4. Select the checkboxes next to the test runs that you want to delete.

  5. Select to delete the test run.

Test cycle execution data

You can additionally track test execution data directly from test cycles. It allows you to see test results of all test runs in this test cycle:

  • Total Test Runs: displays the total number of test runs in the test cycle.

  • Executed: displays the total number of test runs which you already executed.

  • Unexecuted: displays the total number of test runs which you didn't execute yet.

  • Status bar: graphically distributes test runs of the test cycle according to their status.

Overview of test cycle execution data

Filter your test runs

Sometimes, you need to focus on a small part of the bigger picture. In this case, use Filters to investigate specific test runs.

To filter your test runs, follow these steps:

  1. In the Test Execution tab, select Filters.

  2. Select Add filter.

  3. Chose a parameter from the dropdown list. For example, you could filter test runs by Priority.

  4. Select your chosen filter type to drill down further. For example, you could now filter for any test runs that are labeled as High Priority.

Save filters

If you find yourself applying the same group of filters over and over, save yourself some work in the future. Just apply your filter(s), select Save filter, and follow the prompts on the screen. You can access all saved filters any time via the button.

Share your test runs

Data isn't as useful if you're the only one that sees it. You can easily share your test run versions and cycles:

  • To share test versions, cycles, or folders with people who have access to your Jira project, give them the URL of your test run version, cycle, or folder.

  • To share test cycles with people who don't have access— or people who just prefer to have an Excel file— just open the test cycle you want to share. Then, select and Export.

What's next

Now that you have your test cycles and test runs, check out how you can work with them.