Manager 8.7.3 Release Notes

October 23, 2017

Installation/Deployment Packages

Linux Docker deployment packages

  • One Docker deployment package to deploy qTest Manager, qTest Sessions. qTest Insights (no new update to Parameters)

Linux non-Docker installation/upgrade package

  • One package for installing qTest Manager, qTest Sessions. qTest Insights and qTest Parameters (no new update to Parameters)

Windows installation packages

  • One package for installing qTest Manager, qTest Sessions. qTest Insights and qTest Parameters (no new update to Parameters)

Plug-in/Tool Updates

  • Automation Agent Tool v1.3.6

Enhanced Test Execution UI

A key highlight of qTest Manager 8.7.3 OP is an enhanced UI for Test Execution (excluding the TestPad and some pop-up dialogs). This is the first major phase of the enhanced UI for qTest Manager as a whole, which will be rolled out in phases in subsequent releases.

Test Execution Root page

Execution Status Graph

  • The graph is located on top of the root page. It shows the overall status of Test Runs in the project

  • Hover over the graph to view percentage of Test Runs by Statuses

Execution Summary

  • The Execution Summary table shows the status of Test Runs grouped by the highest level of containers (whether that is by Releases, Cycles, or Test Suites)

  • Test Runs that are located directly under Test Execution root are grouped in “No Group”

  • This table can be exported to Excel

Test Run Assignment view

  • This grid is deprecated in the new UI. To view Test Run assignments, you can still use the Data Query or qTest Insights

Test Execution Container pages

A container is either a Release, a Test Cycle or a Test Suite, since these are groupings of Test Runs. One goal of the enhanced UI is to deliver both high-level summary and actionable details across Releases, Cycles, and Test Suites.

Statistics tab of Release, Test Cycle, and Test Suite

Execution Status Graph

  • The graph is located on top of the page. It shows the overall status of Test Runs located under the container

  • Hover over the graph to view percentage of Test Runs by Statuses

“Super-Charged” Test Run grid of Release, Test Cycle, and Test Suite

Left Navigation Panel vs Test Run Grid

  • If you click on a Release, Cycle or Suite from the left navigation panel of Test Execution, the Test Run grid will display all Test Run contents in a “flat list.” For example, if a Release is selected from the left navigation panel, the Test Run grid displays all Test Runs within that Release, regardless of whether the Test Runs are located directly within the Release or sub-contained in other Cycles and/or Test Suites within that Release

  • To view linked Defects of a Test Run, click on the number in the Defects column

Test Run Grid Pagination

  • If there are more than 50 Test Runs, the grid is paginated. You can move through pages or change the total number of items in one page up to 200 using the navigator at the top left or bottom left of the grid

Test Run Grid views specific to the user

  • The Test Run grid can be customized by each user to add, remove, and rearrange columns. This custom view will not impact other users in the project

  • The Test Run grid view will be retained, so that the user can see the same columns displayed when viewing pages for the same container type. In other words, the user has the ability to customize the view for all Test Suites, a different view for all Cycles, and another view for all Releases within the same project (without impact to other users)

Customize Columns in Test Run Grid View

  • To add/remove columns, click the Gear button and select the columns from the dropdown. A few examples of helpful columns to display:

    • Custom Fields of Test Runs

    • Custom Fields of Test Cases

    • Requirements

  • To rearrange columns, hover over the column header. Then drag the dotted handle icon to move it to the preferred position

  • Columns that must be displayed by default: ID, Name, Logs, Defects

Filters on Test Run grid

  • You can filter Test Run grid in Release, Test Cycle and Test Suite pages using any Test Run fields, except for multiple picklist typed fields

  • The applied filters will not be retained after you logged out or close your browsers

Sort test runs in Test Run grid

  • To sort the rows by any column, click on the column header. The order of Test Runs after sort affects the user only

  • Default behavior:

    • Test Runs in a Test Suite page will be default sorted by # (Test Run order within suite)

    • Test Runs in a Release or Cycle page will be default sorted by ID (Test Run ID)

  • After you sort, if you click into a different module (such as Test Design or Requirements) and return back to Test Execution, the sort will be retained

Selecting Test Runs in Test Run Grid

  • To select individual Test Runs from the grid, select the check boxes in the first column next to the Test Runs

  • To quickly select all Test Runs, use these options from the Checkbox dropdown in the grid header:

    • Select all results: to select all Test Runs within the selected container (including Test Runs that are displayed in subsequent pages)

    • Select all on current page: to select all Test Runs currently displayed on the page

    • None: to clear the selection

  • (If you click directly into the check box, you will select all Test Runs on the current page)

Action Buttons from the Test Run Grid

  • Quick Run

  • Run

     

    • Select Test Runs from the grid and click on the Run button to execute based on the default execution mode (TestPad only)

    • You can also select Test Runs from the grid and click on the other execution options from the Run dropdown (which will Launch the execution and also save that default execution mode for the next time you click on the Run button:

      • TestPad only

      • TestPad + Desktop Explorer

      • TestPad + Web Explorer

  • Add Test Run(s)

    • Click Add to create new Test Runs directly under the container (Release, Cycle, or Test Suite) that is selected from the left navigation panel

  • Delete Test Run(s)

    • In addition to the right-click selection from the left navigation panel, you can also delete Test Runs from the grid using the Delete button

  • Export Test Run reports from the grid

    • In addition to the Export Test Run Reports button from the left navigation panel’s toolbar, you can also export these reports for the selected Test Runs from the Test Run grid

    • To export selected Test Run from the grid, select one of the options from the Export dropdown

      • Test Run Details: to export the selected Test Runs with the selected columns from the grid in addition to test step details to Excel

      • Test Execution and Defects: to export for the selected Test Run(s) to Excel. The columns in this report are the same found in the report from the left navigation toolbar

      • Test Execution and Defects - No Cell Merging: to export for the selected Test Run(s) to Excel. The columns in this report are the same found in the report from the left navigation toolbar

      • Automation Test Runs: to export selected Automation Test Run to TestNG xml file

Drag / Drop to reorder Test Runs in a Test Suite

  • In a Test Suite page, you are able to drag and drop the table rows to change Test Run order. This function is disabled in the Release and Test Cycle page

Cut or Copy Test Runs

  • This actions are still possible as a right-click option from the left navigation of Test Execution and not possible from the Test Run grid

Other tabs of Release, Test Cycle, and Test Suite

For easy access you can find other contents related to the selected Release, Cycle or Suite in these sections:

  • Properties (Properties of the Release are read-only as they’re only editable in the Test Plan module)

  • Defect Summary

  • Execution Summary (not available in Test Suite page)

  • History (not available in Release page since its Properties are not editable in Test Execution)

Test Run page

Quick Run Run

  • Click on the Run button to execute based on the default execution mode (TestPad only)

  • You can also click on the other execution options from the Run dropdown (which will Launch the execution and also save that default execution mode for the next time you click on the Run button:

    • TestPad only

    • TestPad + Desktop Explorer

    • TestPad + Web Explorer

Test Run Properties

  • This section can be collapsed and expanded to view all properties of the Test Run

  • Gray fields are in read-only mode and cannot be edited

  • Remember to click Save after you modify any properties

Execution History tab

  • List all of previous executions (Test Logs) of the Test Run

  • Test Log grid can be customized for additional columns

  • Click on a Test Log to view the details of its executed test steps

  • To link Defects to an existing Test Log or Test Step Log, click on the number in the Defects column to open the Link Defects dialog

Attachments

  • Lists all attachments in the associated Test Case (from Test Design)

  • (Attachments from the Test Log are displayed in the Test Log Details for the corresponding Test Log instead of the attachments tab)

Other tabs on Test Run page

  • Test Case Details: To view the associated Test Case properties and Test Steps. These grayed-out fields are read-only and can only be edited on the Test Case page

  • Defects: all Defects which linked to all executed Test Logs and Steps

  • Requirements: all Requirements which are covered by the Test Run

  • Sessions: all Sessions which are associated with the Test Run

  • History: changes made on the Test Run

  • Comments

New Features

Site-level Field Settings

This feature allows standardized fields which will be shared across your qTest Manager projects.

Managing Site Fields

Site fields are defined and configured in Administration panel. To manage site fields, site administrators must have Manage Site-Level Field Settings permission.

  • The Manage Site Fields page is the same as Field Settings page under a project. You will be able to:

    • Browse through the fields of different qTest Manager’s Artifact types

    • Modify system fields

    • Create and modify custom fields

    • Define values for picklist typed fields

  • Instead of defining all custom site fields, you are able to clone field settings from an existing project to the site fields

Managing Field Templates

  • A Field Template includes some or all of site fields. Once you apply a Field Template to projects, these projects will use the same set of site fields included in the Field Template. Any update on the site fields will flow down into the projects

  • A Field Template can be applied to multiple Projects, but one Project can only be associated with on Field Template

Applying a Field Template

  • You are able to apply a Field Template to your existing projects with their own Field Settings. The system will merge the site fields and values with the project fields and values. You will have a chance to preview the mapping between site fields and project fields before proceeding

    • System fields are merged using the original names

    • Custom fields are merged using field names and field types

      • Custom site fields which do not match with any project custom fields will be created as new custom fields in the applied projects

      • Custom project fields which do not match with any site custom fields will remain as internal project fields

    • For picklist typed fields, values from the site fields and projects fields are merged using their display name

      • Project values which do not match with any site values are added to the site fields

      • Site values which do not match with any project values will remain in the site fields

  • You can also select a template when creating a new project

Updating site fields

  • Once a template is applied to projects, any updates on the site fields will flow down into the project field settings

  • Site fields which are removed from a Field Template will become inactive in the associated projects, but you are still able to reactivate them

  • Site fields which are added to a Template will be merged with project internal fields using the same rules as for applying Field Template

Managing Project-level Field Settings

  • In Field Settings page of a project which has been associated with a Field Template, the project admin will be able to view which fields are site fields and which ones are internal project fields

  • Site fields cannot be modified or deactivated within a project

    • You are able to mark a site field Searchable within a project

    • You are able to reorder all fields

    • For picklist field types, you are able to select which values are inactive within your project and select other default values than the selected on the Site-level Field Settings

Refer to Site Fields for more details of how to use the Site-level Field Settings

Audit Log

  • qTest Manager starts tracking some key events for security purposes starting from the date this feature was released (May 24, 2017).

  • The tracked events can be exported to csv file. For now, you will be able to export the following events:

    1. User login

    2. Failed user login

    3. View Explorer Web Session

    4. Export Explorer Web Session

    5. User session ended

  • Data in the Audit logs can be retained up to one year, starting from May 24, 2017 (the Launch date of this new feature)

Internal Defect Workflow

  • Project admins can limit their users’ permission to update Defect Status by defining Transition Rules of the Defect Workflow

  • The Defect Workflow defines Defect Status transitions and which User Profile can perform these transitions

  • Under the Gear icon of each project, select Defect Workflow Settings. From there, you can define the Transition Rules and enable the workflow

  • To add or remove Transition Rules, click on the Add or Remove icons on the Action columns

  • To temporarily disable a Transition Rule, deselect its Active check box

  • To define a Transition Rule, you will need to select appropriate From status values, To status values and User Profiles

    • Any: This is a special value which represent any Status or User Profile values irrespective of whether they have been created or not

  • When creating a new Defect, Status field is not editable. The Status default value as configured in Field Settings is selected

  • When cloning a Defect,

    • If the original Defect's Status is defined as a "From" value in one of the transition rules and the your user profile can perform the transition, the Status value is remained

    • Otherwise, it is reset to the default value and it is not editable

  • You can preview the Workflow when updating Defect Status on the Defect detail page. It will only list out the Transition Rules which associated with your login user profile

  • Updating Defects using Batch Update or Import Defects functions will also need to follow the Transition Rules

Test Automation Scheduling Updates

New plug-in for UFT

  • qTest Manager features Automation Scheduling which allows you to schedule and kick off automation tests that reside on your local machine and report the results of these tests back to your qTest server. Refer to Test Automation Scheduling Quick Start Guide for more background on this existing Automation V2 Integration

  • In addition to the existing support for TestNG. JUnit. Cucumber for Java. and JBehave. we now provide an out of the box plug-in for UFT. This integration will follow the existing Automation V2 Integration

  • Test Cases created to qTest Manager by the plug-in’s scanning function will have no Test Step. They will be updated when your UFT.scripts are executed and results are reported to qTest Manager

New Shell Agent

  • We are introducing a new plug-in, the Shell Agent plug-in, which is capable of kicking off shell scripts (Linux and MacOS. or batch scripts (Windows)

  • The Shell Agent is the newest option in the Test Automation Scheduler that offers a high level of flexibility including:

    • Ability to schedule automated Test Execution (for tests in any framework and language, including custom frameworks)

    • Automation scripts can be stored in a file that is located in the local machine or in a shared network directory

    • Ability to kick off commands, shell / batch scripts, or executable files that are executable from your console or terminal.

    • Ability to define which test runs get executed and how the results will be passed into Test Execution (by using APIs)

Using information passed from qTest Manager in your scripts

  • When Test Runs are scheduled from qTest Manager UI, extra data of these Test Runs and their Test Suite are passed along with the schedule to the agent

  • You can pass those data using command arguments to your kick-off scripts

Enhancements

  • Unlimited custom fields (previously limited to 20 custom fields maximum per object)

  • Default landing page jumps to most recently visited page:

    • When logging in to qTest Manager, instead of the default landing to the Test Plan page, you will be navigated to the most recent page (Most recent page for the user is defined by the most recently visited page where the user spent more than 10 seconds)

  • Deprecated syntax search. Free-text search is still available from the search bar

  • Native browser alert before page close to remind of unsaved changes

  • Required check box is displayed across all system fields and custom fields

  • Ability to deselect default value of Defect Target Release/Build and Assigned To fields

  • The limitation of 15 Defect Status values has been removed. In case there are more than 15 Defect Status values, only the first 15 values are displayed in Execution Status with Defect report

API Updates

  • Submits multiple test results and specifies Test Design and Test Execution tree structures API:

    • The enhancement allows you to update Test Run system and custom fields, except for the read-only fields, before submitting results to the Test Runs

    • The submitted Test Log will inherit the latest field values from its associated Test Run

    • There is a difference between updating Test Runs using qTest Manager UI and using this API

      • In qTest Manager UI: once a Test Run is executed, its Environment field becomes read-only

      • Using API: Environment field is editable even when the Test Run has been executed

  • The following updates are applied to these 2 APIs: Submit A Test Log and Submit An Automation Test Log:

    • The enhancement allows you to update Test Run system and custom fields, except for the read-only fields, before submitting results to the Test Runs

    • The submitted Test Log will inherit the latest field values from its associated Test Run

    • There is a difference between updating Test Runs using qTest Manager UI and using this API

      • In qTest Manager UI: once a Test Run is executed, its Environment field becomes read-only

      • Using API: Environment field is editable even when the Test Run has been executed

  • Email notifications are disabled by default when creating projects via API

  • Add ability to pass Build Number and Build URL when submitting test results using APIs

  • Added ability to paginate results from Get Test Runs API

Bug Fixes

  • 9064 - API - Issue with querying Test Runs by Status

  • 8000 - UFT.Integration - Issue with verifying the signature of SSL configuration for qTest Manager

  • 9170 - Notification Emails being sent to inactive qTest accounts

  • 6789 - Issue with searching imported Jira issue by ID in qTest Manager

  • 9197 - Test Run Schedules with a long title causing errors

  • 9237 - Incorrect information in Clone Project notification email

  • 8868 - Fail to execute Test Runs with Automation Integration v2

  • 9258 - When copy and paste a Test Suite, the Test Run order of the new Test Suite will match with the

  • Test Run order on the Test Run grid of the original Test Suite

  • 9921 - On the Test Run grid, only the unapproved Test Case versions are in red 9889 - Test Log fields are populated with corresponding values in Test Run fields when submitting test results using API

  • 9974 - Detach Defects which are linked with Test Step logs from Test Log level

  • 10059 - Schedule multiple Test Runs from a Test Suite

  • 9867 - When Status of a Jira defect is updated, it will be synced back to qTest Manager

  • 10144 and 9892 - Notification emails with correct comment contents

  • 9948 - Copy and paste a Test Suite - the order of Test Runs on the new Test Suite's grid is the same as in the original Test Suite's grid

  • 10793 - Fix the issue with create Test Runs with custom fields using API

  • 10633 - Issue when submitting Defect to VersionOne

  • 10808 - Issue when linking existing Jira issues to Test Runs

  • 11275 - Issue of displaying Attachment section in Defect pages

  • 11261 - Issue when bulk editing objects with a required Multiple Selection Combo Box field

  • 11287 - Issue of displaying Defect pages in IE11

  • 11299 - Issue of retrieving Test Case custom fields using API

  • 11346 - Issue of loading History pages

  • 11215 - Fix the issue of parsing a HistoryChange entry using our generated SDKs

  • 11334 - Ability to whitelist emails with apostrophes

Known Issues

  • When a user logs in with his SSO account for the first time and merge it with an existing qTest Manager account, there will be 2 events logged: Failed user login (with the qTest Manager account) and User login (with the external account)

  • Test Run order in the grids is not consistent with the order on the Test Execution tree

  • Test Case Properties section in Test Run page: Test Case custom fields always display the latest values, regardless of the Test Run's associated Test Case version

  • Customized view of Test Run grid:

    • Only these fields are inline-editable: Name, Assigned To, Planned Start Date, Planned End Date

    • For Test Runs created from shared Test Cases: except for Test Case name and ID, other Test Case information is not displayed in the grid

  • Currently you will not be able to enter multiple commands into the Shell Agent

  • If Defect Workflow is enabled in your project, it is required that your Defect Status field has a default value. For some reasons, the default value is removed when your project is associated with a Site Field Template (eg: Defect Status field configured in Site Administration panel has no default value, or the selected default value at site-level is deactivated in your project's Field Settings). If you encounter an OOPS page when creating a new Defect in qTest Manager, please ask your Project Admin to access the Field Settings page and select a default value for Defect Status field