Configure the level of detail in your execution results

By default, Tosca Commander logs all execution results on a TestStepValue level. It also reports all possible test states: Passed, Failed, or Paused (available in manual tests only).

If you are looking for different information, you can change this level of detail in your execution results:

  • On which level Tosca Commander should log.

  • Which test states Tosca Commander should consider.

For instance, you can specify you only want results on a TestCase level and only for failed or paused tests.

Specify the level of detail

You can specify the detail level in the following ways:

  • In the settings. In this case, Tosca Commander applies the behavior to all ExecutionLists.

  • With a test configuration parameter (TCP). In this case, Tosca Commander applies the behavior only to the object(s) for which you have set the parameter.

Note that TCPs override the settings for a particular object. For instance, you can modify the settings to specify a certain behavior for all ExecutionLists in the workspace and then set a parameter that specifies a different behavior for a particular ExecutionList.

Configure the settings

To change the level of detail for all ExecutionLists created with Engines 3.0, modify the setting Report successful execution of under Project->Settings->TBox->Logging.

To change the level of detail for all ExecutionLists created with Classic Engines, modify the setting Report successful execution of under Project->Settings->Engine.

Once you have modified the setting, Tosca Commander implements it in all future executions. Existing result logs stay as they are.

Create a test configuration parameter

You can set the TCP for ExecutionLists, individual ExecutionEntries, or TestMandates.

Note that this parameter only works in actual test executions and not in ScratchBook trial runs.

To change the detail level for individual objects, create a test configuration parameter named TBox_Logging_ReportSuccessfulExecutionof and give it one of the following values:

  • TestCases: Log on the TestCase level and only report the test states Failed or Paused.

  • TestSteps: Log on the TestStep level and report the test states Passed, Failed, or Paused.

  • TestStepValuesVerification: Log on a TestStep level. Only TestStepValues whose ActionMode is Verify are logged in detail.

  • TestStepValuesAll: This is the default behavior. Log on the TestStepValue level and report the test states Passed, Failed, or Paused.

Once you have added the TCP, Tosca Commander implements it in all future executions. Existing result logs stay as they are.

Example

This example shows the difference between two possible values for the setting Report successful execution of. It uses a TestCase with two TestSteps:

  • The first TestStep passed.

  • The second TestStep failed.

In the screenshot below, you have specified the default value: TestStepValue - All. With this value, Tosca Commander logs on a TestStepValue level and reports all test states.

Sample TestCase with default value TestStepValue - All for the setting Report successful execution of

In the screenshot below, you have specified the value TestCases. With this value, Tosca Commander logs on a TestCase level, and only reports the test states Failed or Paused.

Sample TestCase with value TestCases for the setting Report successful execution of