Steer controls with Tricentis Vision AI
You can steer the controls that you have scanned with Vision AI in the same way as TBox controls.
This chapter provides information on:
|
You can use Vision AI to steer controls in mobile applications if the Mobile Engine 3.0 doesn't support or recognize them. For detailed information, refer to this Knowledge Base article. |
Additional parameters for table steering
If you want to automate tables, you can modify additional parameters that specify how Vision AI detects table content. For information on how to edit these parameters in XScan, see chapter "Change the parameters for table detection and steering".
The Method parameter defines the way in which Vision AI distinguishes table cells from pixels on the screen. You can use the following values for this parameter:
-
Gridlines - Vision AI searches for long vertical and horizontal lines within the table bounds. This method might recognize the bottom of the text in a row as a possible line, which results in detecting an extra empty row. In this case, you can use the MergeEmpty parameter to get rid of the extra empty row.
-
Words - Vision AI searches for pixels that are different from their surrounding pixels but not gridlines. This method uses the vertical and horizontal gaps between words as cell boundaries.
-
CountrolBounds - Vision AI searches for the bounds of each control and a text block. This method uses the vertical and horizontal gaps between these controls as cell boundaries.
-
Auto - Vision AI tries the Gridlines method first. If it fails to find a good set of cells, it falls back to the Words method.
Steer ColumnThreshold and RowThreshold
The parameters ColumnThreshold and RowThreshold control how sensitive Vision AI is when it selects a row or column. The values for these parameters range from 0 to 100. They specify the percentage threshold that Vision AI must detect to steer the row or column. Lower values mean that more rows or columns are used while higher values mean that less rows or columns are used.
The default value Auto selects the best value based on the Method parameter:
-
For Gridlines method, it's 30.
-
For Words method, it's 15.
-
For ControlBounds method, it's 1.
Vision AI might detect extra empty rows or columns. This happens frequently, if you use the Words or ControlBounds method.
The MergeEmpty parameter specifies when these empty rows and columns are merged into the other content of the table:
-
Set this parameter to Yes to turn merging on or to No to turn it off.
-
Set this parameter to Auto if you want to turn merging off when you use the Gridlines method. For all other methods, it turns merging on.
Steer HeaderColumn and HeaderRow
These two parameters work in the same way as standard TBox tables. You can select which row and/or column number is used for headers.
To set these values, click on the cells in the Content Preview.
Sometimes the exact header row number varies depending on the detection location. In this case, you can't rely on the HeaderRow parameter. Instead, set the HeadersContain parameter to a text value from the header row of your table. Vision AI searches the table for a row that contains this text value and uses it as the header row.
This parameter works in the same way as standard TBox tables. It specifies the columns for the $lastContentRow and $firstEmptyRow selectors. The default value is * which means that all columns are used.
Steer IgnoreInvisibleTableContent
This parameter has no effect for Vision AI tables.
Steer ColumnMinGap and RowMinGap
These parameters specify the minimum number of pixels between columns and rows. The default value is 0.
Increase the values of these parameters by small amounts to reduce the number of columns or rows that have been split incorrectly.
Additional parameters for typing
After you have scanned your controls with Vision AI, you can add steering parameters that control the typing mechanism. These steering parameters apply only to controls with the ControlType property Input or Dropdown.
Add steering parameters for the Input control
The table below lists the possible steering parameters for a control with ControlType property set to Input. If you don't specify these parameters, the typing mechanism behaves according to the default value.
|
Steering parameter |
Description |
Default value |
|---|---|---|
|
KeyDelay |
Specify how long Vision AI should wait after each key-up or key-down action. Enter the value in milliseconds. |
10 |
|
HoldDelay |
Specify how long Vision AI should select and hold the key. Enter the value in milliseconds. |
0 |
|
SelectTextKeys |
Specify the key sequence that Vision AI should select before it enters the value that you specify in TestStepValue. By default, Vision AI replaces the existing content of an input box with a new value that you specify in the TestStepValue. Ensure that you enter the value in double quotes to prevent Tricentis Tosca from recognizing it as a dynamic expression. |
"{Home-Ctrl-Home-Shift-End}" |
|
DeleteTextKeys |
Specify the key sequence that Vision AI should select if the TestStepValue is empty. By default, Vision AI deletes the existing content of an input box if a TestStepValue is empty. Ensure that you enter the value in double quotes to prevent Tricentis Tosca from recognizing it as a dynamic expression. |
"{Delete}" |
|
In this example, you want to select the content of an input box with the key sequence Ctrl and A. To do so, you add the steering parameter SelectTextKeys in the Properties of this control and set its value to "{Ctrl-A}".
SelextTextKeys steering parameter |
Add steering parameters for the Dropdown control
The table below lists the possible steering parameters for a control with the ControlType property set to Dropdown. If you don't specify these parameters, the typing mechanism behaves according to the default value.
|
Steering parameter |
Description |
Default value |
|---|---|---|
|
KeyDelay |
Specify how long Vision AI should wait after each key-up or key-down action. Enter the value in milliseconds. |
10 |
|
HoldDelay |
Specify how long Vision AI should select and hold the key. Enter the value in milliseconds. |
0 |
|
ConfirmItemKeys |
Specify the key sequence that Vision AI should select to confirm an item and close the drop-down. By default, Vision AI selects Tab to confirm an item and close the drop-down. Ensure that you enter the value in double quotes to prevent Tricentis Tosca from recognizing it as a dynamic expression. |
"{Tab}" |
|
|
In this example, your application requires you to press Enter instead of Tab to select an item from a drop-down. To change the default behavior, you add the steering parameter ConfirmItemKeys in the Properties of this control and set its value to "{Enter}".
ConfirmItemKeys steering parameter |
Special value <Foreground> for the Caption parameter
You can edit the Caption parameter of your Module to steer the topmost window. This allows you to steer, for example, pop-up windows or windows without a caption that are in the foreground.
To do so, follow the steps below:
-
Scan a topmost window that you want to steer with XScan.
-
Open the Properties pane of your new Module.
-
Replace the value of the Caption parameter with <Foreground>.
When you use this Module in a TestCase, Vision AI steers the window that is in the foreground.
Caption parameter with value <Foreground>
Limitations in steering tables with Vision AI
Currently, Vision AI doesn't support the following options and properties for steering tables:
-
Table properties: Exists, RowCount, and ColumnCount.
-
Horizontal table scrolling within the table. If Vision AI doesn't find a specified row or cell, it scrolls through the whole page, but only vertically. Horizontal scrolling is not supported.
-
Defining embedded controls in cells via the Create Embedded <Control-Type> control option in the context menu. If you want to use an embedded control, scan it separately in XScan. In the Modules section, drag this control into the Cell ModuleAttribute.
-
Column-first steering doesn't support row labels. This means that you can't use the row label as the cell name if you use the secondary steering under a table control. For example, if you select a column and then a cell, only numeric indexes such as $1, $2, $last are supported.
-
The Content Preview pane doesn't display the cells that are merged. The grid lines in merged cells are visible.
To overcome these limitations, you can insert the VisionScript statements directly between TestSteps in Tricentis Tosca. To do so, use the Run VisionScript Module.