Launch 1.6.3 Release Notes

October 29, 2020

qTest Launch and Tricentis Tosca integration enhancements

The following enhancements have been made in the 1.6.3 release of qTest Launch to improve the integration between qTest Launch and Tricentis Tosca.

Monitor Service now used for Tosca integration

Monitor Service is now used to integrate qTest Launch and Tricentis Tosca. This service must be configured for each qTest Project that is integrated with Tosca. To configure the Monitor Service connection, enter the Monitor Service URL in the new Monitoring Service URL field in the Tosca Integration section on the Automation Settings screen. For information about locating the Monitor Service URL, refer to Locate the Monitor Service URL.

New setting to display only jobs that contain Tosca tests

A new setting has been added to the Jobs view filters, allowing you to display only jobs that contain Tosca tests. Select the new Show Tosca Jobs check box to refresh the Jobs grid and display only jobs that contain Tosca tests.

Easily track progress of Tosca jobs

A new Progress column now displays in the Jobs grid when the Show Tosca Jobs check box is selected. The Progress column displays a progress bar, allowing you to easily determine the progress of a Tosca job. Hover your cursor over the progress bar to display a color legend that indicates the estimated percentage of completion. For example, the scheduled job could be 20% executed and 80% not executed. The new progress bar is also available on the Schedule Details dialog.

Currently, the progress of a job will not automatically update in real time. To display the most recent progress estimate for a job, click the Refresh icon on the Jobs grid.

New execution timeout configuration available for Tosca integration

The Tosca Integration tab has been added to the qTest LaunchSettings dialog. This tab includes a new execution timeout setting, which allows you to define a threshold after which qTest Launch should fail pending executions from Tosca. This setting is designed to stop communication with Tosca on a specific job if there is not progress update from Tosca within the defined timeout period, preventing the connection from qTest Launch to Monitor Service from being maintained for an unnecessary amount of time. If any progress is made on executing the job, the timeout for that job will reset. Progress is considered to be the time between completion of individual Execution Entries (Test Runs) in a Tosca Test Event.

For example, suppose you enter a threshold of 60 minutes. After 60 minutes, Job A has not made progress. Based on the threshold value, Job A will be failed, and server resources being used for that Job will be allocated to another Job. However, suppose after 15 minutes, progress is made on Job A. The threshold is reset, and Job A will be allowed another 60 minutes with no progress before it is canceled.

The default value for this setting is 480 minutes and is applied to all Projects in your qTest site. You can enter a value from 1 to 10,000 minutes. It is recommended that you set the timeout to the maximum amount of time that is required for a single Test Case execution in your Tosca TestEvents. For example, if there is a Test Case that requires 24 hours to execute, the recommended timeout configuration would be a minimum of 1,440 minutes.

In addition, if the timeout value is changed, the new value only applied to upcoming jobs and will not be applied to currently pending Jobs.

New timeout for integration tasks between qTest and Tosca

The Tosca Workspace Timeout field has been added to the Tosca Integration section on the Automation Settings screen. This new setting allows you to define a threshold after which qTest Manager should fail pending integration tasks from qTest to your integrated Tosca workspace. Integration tasks include converting a qTest Test Case to Tosca, linking Test Cases from Tosca to qTest, linking TestEvents from Tosca to qTest, and updating linked Test Cases from Tosca to qTest. Integration tasks typically occur within seconds; however, for larger Tosca workspaces where a complete update (Update All) requires significantly more time, this setting can be used to fail integration tasks that are taking too long to complete. The default value for this field is five minutes and can be set to a maximum of 30 minutes.

Run multiple TestEvents simultaneously

Multiple TestEvents can now be run simultaneously in qTest Launch. Previously, TestEvents were run sequentially, which caused issues when one TestEvent execution could not be run, preventing the remaining TestEvents from starting. With this enhancement, you can schedule multiple TestEvents from the same Tosca workspace and the TestEvents will be run simultaneously. If one TestEvent cannot be run, the remaining TestEvents will not be affected.

Quickly access Tosca Event Monitor

A new link has been added to the Job Details dialog, allowing you to easily access Tosca Event Monitor, where you can monitor test executions for scheduled Tosca tests. The new View Execution Detail in Tosca Event Monitor link displays beneath the progress bar on the Job Details dialog. Click the link to open Tosca Event Monitor in a separate browser tab.

The Tosca Event Monitor URL must be accessible from your internet browser. Contact your IT department for additional assistance, if needed.

Cancel a running execution at any time

Executions of scheduled Tosca tests can now be canceled if the executions are in Not Run or in Running status. Previously, an execution could not be canceled after the execution had been started.

To cancel an execution, click the Action icon for the job in the Jobs grid. Then select Cancel on the menu that appears. You can also cancel a job by clicking the Cancel button on the Job Details dialog.

Please note that it may take some time for an execution to be completely canceled. Once an in progress execution is canceled, any test results that were completed are collected and are submitted to qTest Manager, where they can be reviewed.

View Tosca execution cancellations from qTest Launch

When an execution is canceled in Tricentis Tosca, the execution status is now updated to qTest Launch, providing improved integration between Launch and Tosca.

View execution summaries for Tosca jobs

Once a job is executed in Tosca, you can now view an execution summary for the job directly in qTest Launch. To view the summary, access the Job Details dialog for a completed, failed, or canceled Tosca job. The execution summary appears at the bottom of the Last Execution Logs section and includes details such as the status of the job, any errors that occurred while executing the job, the reasons for the errors, and the number of test logs submitted to qTest Manager.

Bug fixes

Ticket # Component Description
    An issue was fixed in which Test Runs failed when test scripts were run in the Eclipse IDE, but passed when the tests were integrated with qTest.
  Launch An issue was fixed in which Test Runs within Test Suites in a Release could not be added to qTest Launch for scheduling or were not being added in the correct folder location on qTest Launch's schedule screen.

Known limitations

  • The consumption of Monitor Service is not secure. If the Monitor Service endpoint is public on the internet, any user who knows the endpoint can access the service and perform operations that the service provides, such as viewing a list of running executions or canceling a specific execution.

  • The format of execution results from Tosca are free text, and in some cases are not consistent, allowing for potential incorrect parsing of execution results in qTest Launch.

  • The exact duration of test executions cannot be calculated. The time it takes to execute a TestEvent can vary based on the number of Test Cases in the TestEvent and the size and complexity of each Test Case. Execution progress is considered to be the time required for a single Test Case to fully complete execution.

  • A linked Tosca workspace is checked out when integration and execution tasks are triggered from qTest to Tosca and is facilitated using the Tosca Commander REST API. This API call will fail if the workspace is left open or if the workspace is locked. This behavior is similar when using Tosca CI for triggering Tosca executions. The integrated Tosca workspace for your qTest Project must only be accessed by the integration tasks and not be opened by an end user during the linking of objects between Tosca and qTest and the scheduling or execution of Tosca TestEvents from qTest Launch.

  • If the Launch server is restarted when there are several running jobs, all running jobs will be marked as failed after the server restarts. Launch will not resume checking for statuses or execution results for any previously running jobs. In addition, if Monitor Service is restarted, all executions in Monitor Service will be removed, and Launch will fail the job.

  • In qTest, users can no longer create Test Cycles or Test Suites with linked Tosca TestCases and schedule for execution using the native integration. By design, the grouping of Tosca TestCases needs to be performed in Tosca as the creation and linking of TestEvents before the TestCases can be scheduled in qTest Launch. This process ensures that the proper configurations and Agent setup are adhered to and can only be controlled from Tosca. If previously created qTest Test Cycles and Test Suites with Tosca TestCases have been scheduled on a recurring cadence, they will continue to execute and allow the user time to properly configure a TestEvent in Tosca before linking to qTest. This process also eliminates ExecutionLists and TestEvents from being dynamically and exponentially created in qTest.

  • If a Tosca user deletes sub-objects of a TestEvent, such as individual ExecutionLists or ExecutionEntries (Test Runs), and then performs an update, these objects are not deleted in qTest. This design is to maintain data integrity and prevent the loss of execution history in qTest. Currently, there is no visual indication to the user in qTest that specific sub-objects have been deleted. When a TestEvent is scheduled in qTest, Tosca will execute the up-to-date TestEvent despite additional objects being displayed in qTest at the time of execution. If users plan to delete sub-objects of a linked TestEvent in Tosca, it is recommended to first unlink the TestEvent, make the appropriate changes, and then relink the TestEvent to qTest. Tricentis plans to enhance this user experience in a subsequent release with a visual indication in qTest of the specific sub-objects that have been deleted in Tosca.