Best Practices for Integrating with Broadcom Rally® Software

Structuring qTest Projects Integrated with Rally Overview

This article focuses on best practices for integrating your qTest projects with Rally and how to establish an efficient process and workflow between the two tools. When thinking about how to best integrate your Rally projects with qTest Manager, it is important to understand which development milestones best drive your testing efforts: releases or sprints.

For detailed instructions on how to connect your project with your Rally instance, please refer to the setup steps described in Integration with Rally.

First, it is important to fully understand the major objects that you and your teams will be working with. Take a look at the two definitions below:

  • A Release is a set of features (Backlog Items of Stories) deployed together as a single update to your application being developed. Assigning features to Releases will determine which new features will be available to your customers when the Release is deployed. To ensure teams Release new versions on a regular basis, Releases are often scheduled on a fixed date.

  • Sprints define a period of time in which development teams implement and deliver a prescribed amount of product functionality. A sprint is comprised of a collection of features you ideally will complete in a fixed period of time, but you may not necessarily do so by the Release date. Features and issues may change sprints during a Release (or even change releases), and, once a team reaches a Release date, any incomplete features may move out of their active Sprints into the product backlog to be reassigned.

Also, it is important to understand the basic motivation behind integrating qTest with Rally. Rally, unlike other tools such as Jira, has native Test Management functionality. Inside Rally, you will find Test Cases and Test Runs and the ability to track and modify them. However, qTest offers much more Test Management functionality. For example, in Rally, there are no dedicated Test Steps attached to a Test Case. In order to track Test Steps, you would have to manually create them and then dump them all into a basic description field, whereas in qTest there are dedicated fields and functions to track Test Steps for a more complete approach to Test Case Management.

So, the qTest integration is the right step to bolster and simplify your team's overall Test Management abilities by taking the place of Rally's more limited Test Management offerings. All the while, Rally will still be used for Development Backlog, Requirement, and Defect management.

Example Rally Structure

In order to make the best use of your Rally and qTest integration, you must organize and structure your Rally environment in an efficient manner.

To begin with, let's take a look at the basic hierarchical structure of a Rally environment:

  • Workspace (This is the highest level container available)

    • Projects

      • Child Projects

A workspace is made up of Projects, and those Projects can have child projects. How you make use of this structure is going to heavily weigh on how you organize and structure your development efforts in Rally. Whichever way you structure your Rally environment will then influence how you should integrate Rally with qTest.

In this article, we will be referring to a standard example where all development efforts are structured as a Project under one Workspace in Rally. The Development Project is broken up into various initiatives housed within Child Projects.

Each Release is divided into Sprints. This Release has three Sprints, as seen below in the Iterations section of the screen:

Sprints, also called iterations, comprise of items from the Backlog. As teams begin their Sprint or Iteration scheduling and planning, the team could:

  • Group them by Portfolio

  • Assign work to specific teams

  • Further break down Backlog Items into specific tasks

  • Estimate their effort to complete them

Any items not assigned to a Sprint remain in the Backlog until they can be committed to a Sprint effort.

Configuring the Integration in qTest

After you successfully set up the connection with your Rally instance, you can begin the integration configuration by setting up the defect integration.

Defect Integration

For detailed instructions on how to configure defects with Rally, refer to the setup steps described in Configure Defects with Rally.

In this example, we are configuring the integration to allow our qTest users to log defects both at the root project level, "Sample Project," and at the Child Project level, "Sample Child Project."

We recommend you add the overall project and each child-project as a defect type so that your testing team can select the appropriate option when logging an issue at the time of test failure. But, if you want to limit users to only log defects at the root project folder or solely into a specific Release, this can be configured here.

Next, we will configure the Requirement integration.

Requirement Integration

For detailed instructions on how to configure Requirements with Rally, please refer to the setup steps described in Configure Requirement Integration with Rally.

In this example, we are only adding requirement types for our Release, "Sample Project" and not for the entire Workspace, "Workspace 1." The reason for this is because we only want our testing team to work on items that are committed to a release. When, and only when, Backlog Items are assigned to a specific Release will they be imported in qTest automatically.

Organizing your qTest Project

Create your Releases in Test Plan

You will initially want to create a Release folder in Test Plan for each Release in your Rally project. In this example, we have one Release and we will title it "Sample Release 1". For each new Release that is created in Rally, your qTest team will need to create a Release folder in the Test Plan tab to ensure that your qTest project is aligned to the same Release cadence as your development team in Rally.

View Imported Rally Requirements

Upon initial import, Rally objects will be treated as Requirements in qTest Manager. By default, these integrated requirements will be loaded into a common Module folder named "Imported from Rally", as shown below.

Reorganizing your Imported Rally Requirements

You will first want to reorganize your initial imported requirements. When integrating with active Rally projects, this folder could contain hundreds of requirements, but, ideally, this integration and initial import should be configured at the very beginning of each new Rally project or Release. Therefore, the process is more concentrated on managing the integration than an initial organization effort.

When organizing the initial import, we suggest bucketing your requirements at a single property level so that the management efforts are mitigated. In this example, we are going to organize our Requirements by their specific Sprint value. Backlog Items are less likely to change Sprint values as often as they may change status or team, so if you organize requirements by more than one level ( such as by Sprint, Team, and Status), your qTest users will be spending too much time keeping your requirement folders up to date.

We are going to create a folder outside of the "Imported from Rally" folder for each of our Sprints. This way, you can think of the "Imported from Rally" folder as the container for any new Backlog Item that comes into scope for your Release that then needs to be placed in the appropriate Sprint folder. You want to create your Sprint folders outside of the "Imported from Rally" folder and not nested within as subfolders. Create your Sprint folders, as seen below.

Next, you need to drag and drop your imported Rally requirements into the appropriate Sprint folders.

Maintaining your Imported Rally Requirements

Now that you have updated your existing Backlog Items into their appropriate Sprint folders, your qTest users will need to manage existing issues and organize any new Backlog Items that come into scope. This should require minimal effort if maintained properly.

The image below shows a new backlog item being created in Rally.

When your development team creates new stories in the Rally backlog and saves, the stories will automatically appear in qTest in the "Imported from Rally" folder, as seen below.

This story will remain in the Backlog in Rally until it has been assigned to a Sprint. In the same vein, the Requirement should remain in the "Imported from Rally" folder in qTest until it has been assigned to a Sprint. At that point, you should drag and drop it into the appropriate Sprint folder in qTest.

Your qTest users will need to periodically check the Requirements in the backlog ("Imported from Rally" folder) and move them to the appropriate Sprint folder once they have been assigned a Sprint value.

You do not have to retrieve each Requirement again using the Retrieve Data button to get the updated data of Rally objects on the requirement. Doing this would work, but the integration should pull new objects from Rally into qTest every minute or two.
Currently, no Rally fields can be used to search or query in qTest Manager, but you can search for the name of a Rally Requirement by using quotation marks.

Clicking the search result will take you to that object in qTest Manager.

Optional: Maintaining your Release Scope in Test Plan

In the Test Plan, you may want to take the time to populate and maintain the Release Scope for each Release object.

For detailed instructions on how to add Requirements to your Release, refer to the setup steps in Add a Requirement to a Release.

By adding Backlog Items to their appropriate Release folder in Test Plan, you are ensuring you have full end-to-end traceability between objects in qTest. Several reports in qTest Insights are driven off the Release Scope, and you can also plan your Test Runs in the Test Execution tab of qTest Manager using this Release traceability.

One limitation may be the amount of time this may take to manage if you have large Releases. You can still generate the same reports in qTest Insights that are driven from this traceability, but they may need to be manually created and not provided out-of-the-box.

The other limitation is that, when planning Test Runs, your qTest users will not be able to add Tests based on the Release Traceability. They will not have the option to add Test Runs based on their linked Requirements in a specific Release. Any requirement that has linked test cases that are part of the Release Scope will be available for selection.

This is not a critical piece of functionality but simply something to consider when deciding if it’s worth taking the time and effort to manage the Release Scope in Test Plan.

However, if all Backlog Items in a Sprint are tied to a single Release, this can easily and quickly be managed by selecting the entire Sprint folder when adding Requirements, as seen in the screenshot above.

Linking Test Cases to Rally Requirements

Imported Rally objects can be linked to qTest Manager Test Cases. The linking with qTest Manager Test Cases will be generated in the corresponding Rally object's Links section.

For detailed instructions on how to link Test Cases and Requirements, refer to the setup steps described in Link Test Cases and Requirements.

When you link or create associated Test Cases to an Imported Rally object, qTest Manager also generates a link under the corresponding Rally object. To quickly navigate to the Backlog Item in Rally, click the ID link on the Requirement in qTest.

Submitting Defects to Rally

You can submit defects to Rally from qTest Manager during TestPad Execution or from the Test Log.

For detailed instructions on how to submit Defects to Rally, refer to the setup steps described in Submitting Defects to Rally.

qTest users can submit a defect during Test Execution using the Quick Run or Test Pad window. Optionally, they could use the Test Log in Execution History to submit a defect after executing a Test Run, as seen below.

Test Pad window:

Test Log in Execution History:

The Rally Defect Submission page will display.

Specific fields could be auto-populated with qTest information, such as the Test Steps in the Description field, depending on your integration configuration.

After clicking Save and submitting the Defect into Rally, your qTest users will be returned back to qTest Manager.

qTest users should be able to quickly reference the linked Rally Requirements in their active qTest Manager window.

Once the Backlog item has been assigned to the Defect, it should be visible as a link on the Rally Defects tab.

Now your qTest users can return to the Defect window on the Test Pad or Test Execution tab of qTest Manager to close the defect and return to testing.

Review Rally Defects After Submission

In qTest Test Execution, you can find Rally defect information in many areas.

  • To view Rally defects for a single Test Run, select the Defect icon from the Test Run grid on the Statistics tab.

  • To view an aggregate list, select the Defect Summary tab for a roll-up of all Rally defects related to associated Test Runs. The Defect Summary tab can be customized so that you can display additional Rally Defect fields as columns.