Vision Script Reference

Vision Script is the language used to write test cases that can be run by the Vision AI Executor. To execute Vision Script test steps from Tosca, use the Run VisionScript module.

Structure

Each Vision Script command starts with a keyword and is usually written on a single line. For long commands, you can split the command over several lines using the \ backslash character at the end of the line. Comments maybe written using // double-slash, or /* and */ for multi-line comments. A simple test case may look like this:

/* This will log the user in to the application
   Make sure the username and password are kept up to date
*/
TYPE "admin" IN the 'Username' input
TYPE "pass123" IN \
  the 'Password' input
CLICK 'Submit'    // note that the control type is optional.

Control Flow

Vision Script also supports block-level commands like IF/THEN/ELSE, WHILE, REPEAT/UNTIL, TRY/FAILURE. These commands may be nested inside each other. Space at the beginning of the line is optional, but helps improve readability.

e.g.

REPEAT
  TRY
    CLICK 'Login'
  FAILURE
    CLICK 'Refresh'
  END TRY
  IF :Count < 10 THEN
    STORE :Count + 1 INTO :Count
  END IF
UNTIL 'Username' is visible

Control Specification

Many Vision Script commands expect a control specification. In the syntax reference below, that is indicated by <control-specification>.

[THE] ['<name>'] [<control type>] [#<index>] [WITH <property expression>] [HAVING <child expression>] [THAT IS <relative expression>]

All parts of the control specification are optional, but there must be at least a <name> or a <control type> specified.

Examples:

THE 'Username' input
THE button WITH color == "Red"
'OK'
'Cancel' button
'Submit' button #2
THE row HAVING the 'Type' cell == "Production"
row HAVING 'Type' == "Production" and 'Status' == "Active"
THE 'Street' input THAT IS BELOW 'Home Address' text and ABOVE 'Business Address' text
THE 'Submit' button THAT IS LEFTMOST

Quotes

Important Note!

When using quotes, control names use single quotes e.g. 'OK', and string values use double quotes e.g. "Red". This allows Vision AI to distinguish between referencing a control or comparing a value.

Control Names

A control has a name, and for everything but Text controls, this will match the control's label (or labels if multiple have been assigned to the control). For Text controls, the name matches against the Content of the Text (i.e. the text value itself).

e.g.

CLICK the 'Account' input                 // matches an input next to the "Account" label
CLICK the 'Account' text                  // matches the text "Account"
CLICK the text WITH label == "Account"    // matches the text next to the "Account" label
CLICK the input WITH label ~= "Acc*"      // wildcard matches are performed with the ~= operator

Control Types

A control may be one of the following types:

Button
Input
Dropdown
RadioButton
CheckBox
Table
List
Menu
Icon
ToolBar
TabBar
Panel
Tree
Scrollbar
TitleBar
Window
Text
Row
Column
Cell
Image
Item
Node

There are two special control types: Control means any control type except for Text, and Anything which means any control type including Text.

Control
Anything

Omitting the control type altogether means any control type including Text.

Control Type Options

The Control Type may be followed by a set of options in square brackets []. These options are used in identifying or steering the control.

<control-type>[option=value, ...]

e.g.

WITH table[Method=Gridlines]
  CLICK the 'copy' image[Grayscale=true, MultiScale=true]
END WITH

Scrollable Control Options (Table, Tree, List)
- ScrollSpeed (number - a speed multiplier. Default is 1.0.)
- ScrollTimeout (number - seconds. Default is 10.)
- ScrollDownTimeout (number - seconds. Overrides ScrollTimeout)
- ScrollUpTimeout (number - seconds. Overrides ScrollTimeout)
Table Options
- Method (Auto/Gridlines/WordPixels/ControlBounds)
- HeadersAbove (true/false)
- HeadersContain (string - label of a column heading)
- (more advanced options for row/column detection are available, see Table Steering below)
Tree Options
- DepthMethod (Text/Pixles)
- MinIndent (number - minimum pixels each node must be indented from the parent)
- ExpandMethod (Click,DoubleClick,IconClick + {keys})
- SelectMethod (Click,DoubleClick,IconClick + {keys})
Menu Options
- FirstClick (true/false)
- IntermediateClick (true/false)
- SeparatorThreshold (number - percentage, default = 50)
Toolbar Options
- SeparatorThreshold (number - percentage, default = 50)
Tabbar Options
- SeparatorThreshold (number - percentage, default = 50)
Image Options
- Grayscale (true/false - find image using grayscale only)
- MultiScale (true/false - find images of different sizes to the reference image)
- Scale (number - find images at the specified scale compared to the reference image. 1.0 = same size, 2.0 = double size, 0.5 = half size etc.)
- Threshold (number - match threshold (0.0 - 1.0). Default is 0.75.)

Scrolling Options

These options can be used with any scrollable control, such as Tables, Trees, and Lists. These options can also be used for non-scrollable controls, and affects how the control is found.

ScrollSpeed - a multiplier. 1.0 (the default) is the standard scroll speed, increasing or decreasing this number makes scrolling faster or slower (e.g. 2.0 or 0.5).
ScrollTimeout - number of seconds to spend scrolling up or down looking for controls. Default = 10
ScrollDownTimeout - overrides ScrollTimeout for scrolling downwards (searching downwards)
ScrollUpTimeout - overrides ScrollTimeout for scrolling upwards (searching upwards)

Table Steering

When scanning a table with Vision AI, there are many ways the table cells can be identified from the pixels on the screen. Vision AI has several predefined methods to identify the cells: Auto, Gridlines, Words, and ControlBounds. The default Auto setting tries a combination of the Gridlines and Words methods.

The Gridlines method searches for long vertical and horizontal lines within the table bounds. Sometimes the bottom of the text in a row can seem like a possible line to Vision AI, and an extra empty row may be detected. See below for further options for tweaking this behavior.

The Words method searches for pixels that are different from their surrounding pixels, but are not gridlines. Then the vertical and horizontal "gaps" between these word pixels are used as the cell boundaries.

The ControlBounds method uses the bounds of each control and text block detected by the Neural Networks. Then the vertical and horizontal "gaps" between these controls are used as the cell boundaries.

Table Steering Options

Method - Sets the main detection method. Valid values are Auto (default), Gridlines, Words or ControlBounds. The Auto method is a combination of Gridlines and Words.
ColumnThreshold, RowThreshold - These threshold parameters control how sensitive Vision AI is to choosing a row or column. The values range from 0 to 100, and represent the percentage threshold that must be detected in order for the row or column to be used. Lower values means more rows or columns will be used, higher values mean less rows or columns will be used. The default value of Auto selects the best value based on the Method parameter: 30 for Gridlines, 15 for Words, and 1 for ControlBounds.
ColumnMinGap, RowMinGap - These parameters specify the minimum number of pixels between columns and rows. The default value is 0. Increase the values of these parameters by small amounts to reduce the number of columns or rows that have been split incorrectly.
MergeEmpty - Often there are extra empty rows or columns detected (especially when using the Words or ControlBounds method). This parameter controls when these empty rows and columns are merged into the other contents of the table. The True/False values explicitly turns merging on/off respectively. The default Auto value turns merging off when using the Gridlines detection method, otherwise turns it on.
HeaderColumn, HeaderRow - These two parameters work the same as with standard Tosca tables – you can choose which row and/or column number is used for headers. The default is 1 (i.e. the first row and column are used for the headers)
HeadersContain - Sometimes the exact header row number varies depending on the detection location. In this case the HeaderRow parameter cannot be used reliably. Instead, set this parameter to a text value that is contained in the header row. Vision AI will search the table for this text, and use that row as the header row.
HeadersAbove - Sometimes the top header row is not detected as part of the table content. Set this value to True to extent the table bounds vertically upwards to include this row.

Table Steering Properties

Row controls have the following extra properties:

RowNum - the row number. The first row (including the column headers) has RowNum == 1
RowIndex - the row number relative to the column headers row. The header row has RowIndex == 0, any rows above the header are negative, rows below are positive.
Label - contains the text of the row header cell (usually the leftmost cell)
Content - contains the all text of the entire row

Column controls have the following extra properties:

ColumnNum - the column number. The first column (including the row headers) has ColumnNum == 1
ColumnIndex - the column number relative to the row headers column. The header column has ColumnIndex == 0, any columns to the left are negative, columns to the right are positive.
Label - contains the text of the column header cell (usually the topmost cell)
Content - contains the all text of the entire column

Cell controls have the following extra properties:

RowNum - from the cell's row
RowIndex - from the cell's row
ColumnNum - from the cell's column
ColumnIndex - from the cell's column
RowSpan - for merged cells, the number of rows this cell spans
ColumnSpan - for merged cells, the number of columns this cell spans
Label - contains the text of the column header cell (usually the topmost cell)
Content - contains the all text in the cell

Table Steering Examples

Table steering uses the WITH statement to select the table, row/column and then cell to work with. Each WITH statement filters the existing controls to a more restricted set.

e.g.

WITH 'Vision AI Execution' table[Method=Gridlines, ColumnThreshold = 30, RowThreshold = 50]
  WITH row HAVING 'Capability' == "Table Steering"
    VERIFY 'Current State' == "DONE"
    VERIFY 'Limitations' == ""
  END WITH
END WITH

WITH 'All Items' Table
  WITH row #2
    TYPE "1000" in the 'Material' Cell
    TYPE "1{Enter}" in the 'Order Quantity' Cell
  END WITH
END WITH

WITH the table
  WITH row #1
    VERIFY color=='Yellow' for the 'Gold' cell
  END WITH
  WITH row #2
    VERIFY color=='White' for the 'Gold' cell
  END WITH
END WITH

WITH the 'Type' table
  WITH the 'Article' row #1
    TYPE "10/07/2014 – 21:00" in the 'Post date' cell
    WAIT 2 seconds
    VERIFY the 'Post date' cell == "10/07/2014 – 21:00"
  END WITH
END WITH

WITH the 'Summary' table
  WITH the row HAVING cell #1 == "Select Option"
    WITH the 'Gold' cell
      CLICK the RadioButton
    END WITH
  END WITH
END WITH

WITH the 'Summary' table
  WITH the 'Gold' column
    CLICK the RadioButton
  END WITH
END WITH

Tree steering

The basic steering method is to use the SELECT command with dart-syntax, where the names of each tree node are separated by "->". e.g.

SELECT "Level1->Level2->Level3" FROM the 'Navigation' Tree

WITH syntax is also supported, e.g.

WITH the 'Navigation' Tree
  WITH the 'Level1' Node        // WITH .. Node  will expand the node if it appears collapsed
    WITH the 'Level2' Node
      CLICK the 'Level3' Node
    END WITH
  END WITH
END WITH

Tree Steering Options

DepthMethod - one of Text or Pixels. Controls how different levels are identified in the tree. Text uses the left bound of the text, Pixels uses the leftmost non-background pixel (e.g. the left edge of the icon or text, whichever is first). Some trees have different indents on the text, depending on whether there is an expansion icon or not.
MinIndent - controls the number of pixels a tree node should be indented from the parent node to count as a child.
ExpandMethod - one of Click, DoubleClick or IconClick. Defaults to DoubleClick. Can also be followed with keystrokes in curly braces eg. {Right}. Curly braces must be quoted in Tosca e.g. "Click{Right}". Controls how a tree node is to be expanded.
SelectMethod - Similar to ExpandMethod, but for selecting the final node. Defaults to Click

Limitations

For SELECT/dart-syntax steering, trees are considered to be lists with expandable nodes, there is no checking that each level is strictly below the previous level. This is due to current limitations in scrolling and OCR bounds detection. Future releases may improve this behavior.
For WITH block steering, all the nodes must be visible on screen at once, if a parent node is scrolled out of view during steering, then the steering will fail.

List steering

Use the SELECT command with the name of the item to select

SELECT "Inbox" FROM the 'Folders' List

Steering Options

SeparatorThreshold - A percentage of the height of the control, default = 50. If a vertical line is detected with a length that exceeds this threshold (e.g. more than half the height of the bar) then this is considered a break in the items, no item will span over the separator. Useful if words that are close together are being merged together into one item, but there is still a vertical line between them.

Limitations:

For tab bars, the currently selected tab is not identified, and so can't be verified. This may come in a future release.

Indexing

Use the #<n> syntax to retrieve the nth control from a set of identically matching controls.

#1, #2, #3...
#-1, #-2, #-3...

The order of the controls is from top left to bottom right (i.e. control #1 is the top-left matching control).

Negative values mean "from the end": control #-1 is the last control, #-2 is the second last and so on.

Warning: indexes can be unstable, it may be better to use relative expressions.

Wildcard Matching

You can use wildcard matching in property verification expressions and control property selectors by using the ~= wildcard match operator. The wildcard character is *, and can be escaped with {*}. To use '{' or '}' in a wildcard expression, escape by doubling the character: '{{' or '}}'. To capture a portion of the text to a variable, use the syntax {:variable}. This acts just like a wildcard character, except that the matched characters are stored in the named variable.

Wildcards are not interpreted when using 'name' syntax, or the standard == operator. Only the ~= operator activates wildcard matching.

e.g.

VERIFY Content ~= "ABC*" for the 'License' input     // will match anything after ABC
VERIFY Content ~= "ABC{*}" for the 'License' input   // will match the * literally because it is escaped.
VERIFY Content == "ABC*" for the 'License' input     // will match the * literally when using ==

VERIFY Content ~= "ABC{:License}" for the 'License' input    // will match ABC, and any characters after ABC are stored in the 'License' variable

VERIFY Content == "ABC{DEF}" for the 'License' input     // will match the content ABC{DEF}
VERIFY Content ~= "ABC{{DEF}}" for the 'License' input   // will match the content ABC{DEF} - the { and } must be escaped in wildcard expressions.

VERIFY Content == "ABC123" for the 'License*' input                  // Incorrect! There is no wildcard operator, so the * will be matched exactly.
VERIFY Content == "ABC123" for the input with Label ~= "License*"    // Correct - use the property name and the ~= operator to use wildcards

Tosca Integration:
▸ When running from Tosca 14.2 and later, captured variables will also set the corresponding buffer value.

Previous versions:
▸ In Tosca 13.4 / Vision AI 1.35 and earlier, wildcard matching and the ~= operator are not available.

Property Expressions

Here are the available properties for each control. Each property type has its own specialized matching algorithm.

Type - one of the control types
Label - the label of the control
Content - the content of the control
Color - bright/dark/dim/dull red/green/blue/ etc
Position - top/middle/bottom left/center/right
IsNew - true if the control is newly detected since the last successful CLICK/MOUSE/TYPE/SELECT command.
RowNum
ColumnNum
RowSpan
ColumnSpan
Heading - same as label (for cells)

It is also possible to specify a match threshold:

<property-name> <operator> [<percentage>%] <value>

e.g.

CLICK button WITH color == 70% "blue" and color == 10% "dull red"  // button must contain (approximately) 70% blue pixels and 10% dull red pixels
CLICK button WITH content >= 95% "Automobile"  // text content must match by at least 95% confidence

Child Expressions

An expression that is evaluated in the context of the parent control. Use these expressions to find controls that have a child matching the expression.

e.g. row cells

WITH row HAVING 'Type' cell == "Production"

The child expression is 'Type' cell == "Production". This is evaluated for the children of each matching control (in this case row controls). A table may contain several rows, this expression will be true for the row where there is a cell with the label 'Type' and the value "Production".

Relative Expressions

Relative expressions can take one of three forms:

[1st|2nd|3rd...] (LEFTMOST | RIGHTMOST | TOPMOST | BOTTOMMOST)
[1st|2nd|3rd...] [ANYWHERE] [FROM] (LEFT | RIGHT | TOP | BOTTOM) OF <control-specification>
[1st|2nd|3rd...] [ANYWHERE] (ABOVE | BELOW) <control-specification>

The first form selects one of the matching controls according to the index and direction specified.

Previous versions:
▸ In Tosca 13.4 / Vision AI 1.35 and earlier, the "ANYWHERE" keyword is not available, all searches are "Anywhere" searches.

e.g.

CLICK the 'Select' button THAT IS LEFTMOST
CLICK the 'Select' button THAT IS 2nd RIGHTMOST
// alternative syntax:
CLICK the 'Select' button THAT IS 2nd FROM RIGHT

The second and third forms selects controls that are relative to another control (the "anchor" control)

e.g.

CLICK the 'Select' button THAT IS LEFT OF the 'Account' text
TYPE Auckland in the 'Street' input THAT IS BELOW the 'Home Address' text

Searching

Previous versions:
▸ In Tosca 13.4 / Vision AI 1.35 and earlier, the "Wedge" behavior is not available, all searches are "Anywhere" searches.

When searching in a particular direction, Vision AI will look in a wedge from the center of the anchor control through the corners of the anchor control. All controls where the centrepoint of the control is in the wedge are considered.

CLICK the control THAT IS right of '3'

Here the closest control to the anchor is chosen, i.e. '7'. Controls that are further away can be selected using the indexing syntax:

CLICK the control THAT IS 2nd right of '3'

Here the 2nd closest control to the anchor is chosen, which is '13'. The wedge searching is applied at each step:

The 1st right of '3' is '7'
The 2nd right of '3' is the 1st right of '7'
The 1st right of '7' is '13'
Therefore, the 2nd right of '3' is '13'

Anchors can also be combined:

CLICK the control THAT IS right of '3' AND below '4'

Here the closest control to all anchors is chosen, i.e. '6'.

The ANYWHERE keyword turns off the wedge logic, and all controls strictly in the direction specified are considered.

CLICK the control THAT IS anywhere right of '3'

Here the best matching control is chosen, no preference is made to location.

If two horizontal and/or two vertical relative conditions are specified, then ANYWHERE mode is automatically activated:

CLICK  the control THAT IS right of '3' and left of '10'

Here the best matching control is chosen, no preference is made to location.

Other combinations are possible, mixing ANYWHERE with standard directions:

CLICK  the control THAT IS anywhere right of '3' and below '4'

Control values

When used in an expression, a control specification will resolve to a value that is the content of the control.

e.g.

STORE the 'Make' input INTO :CarMake
VERIFY the 'Type' dropdown == "Truck"

To get the value for a specific property, use the FROM operator:

<property> FROM <control-specification>

e.g.

STORE color FROM the 'Make' input INTO :MakeInputColor

To verify the value of a specific property, use the FOR operator:

<property-expression> FOR <control-specification>

VERIFY color == "Blue" FOR the 'Submit' button

Datatype conversion

Sometimes properties need to be converted to other datatypes. The available datatypes are Number, DateTime, Boolean and String.

<property-name> AS <datatype>
<property-name> AS <datatype>(<format-string>)

This allows more accurate comparisons, especially for number and datetime datatypes.

e.g.

VERIFY content AS DateTime == "3 Nov 2020" FOR the 'Date of Manufacture' input
VERIFY content AS Number == 1000.0 FOR the 'Annual Mileage' input

STORE content AS Number FROM the 'Annual Mileage' input IN :Mileage

You can also call the datatype conversions as functions when using expressions:

STORE Number('Annual Mileage' input) IN :Mileage
STORE DateTime('Date of birth' input, "dd/MM/yyyy") IN :DOB

Visibility Expressions

Use IS VISIBLE or IS NOT VISIBLE to check if a control exists on screen or not.

<control-specification> IS VISIBLE
<control-specification> IS NOT VISIBLE

Expressions

Literals

Strings - use double quotes for text. Special characters are escaped with the \ backslash character
- "Red"
- "Embedded \"quote\" in string"
- "Single Backslash \\ in a string"
- "C:\\Temp\\file.txt"
- "Text with CRLF line break\r\n"
Labels - use single quotes for control labels
- 'Make'
- 'Account \'Name\''
Identifiers (e.g. control types, property names, function names etc.) must start with a letter, and can contain letters or digits
- Not
- Convert2Text
- input
Numbers - decimal, scientific, or hexadecimal numbers
- 10
- 45.01
- 1.34e5
- 56.3e-7
- 0x04123
Symbols - expression operators and other symbolic constants
- *
- <=

Functions

Utility functions
- Not(<value>) - negate a value or expression
- Format(<text-value>, <format-string>) - format text using the .Net string formatting syntax
- Env(<name>) - retrieve an environment variable
- ScriptArg(<index>) - retrieve an argument from the command line
- Now([<seconds-offset>]) - retrieve the current date and time. Optionally specify an offset in seconds.
- IsRegexMatch(<string>, <pattern>) - match a string against a regular expression pattern.
Datatype conversion functions
- Boolean(<value>) - convert to boolean
- DateTime(<value>[, <format-string>]) - convert to datetime, optionally using a .Net date-time string format
- Number(<value>) - convert to number
- String(<value>) - convert to string

Operators

Mathematical operators
- + (addition)
- - (subtraction/negation)
- * (multiply)
- / (divide)
String operators
- & (concatenate)
Boolean comparison operators
- = or == (equals)
- <> or != (not equals)
- < (less than)
- <= (less than or equal)
- > (greater than)
- >= (greater than or equal)
- ~= (wildcard match)
Boolean combination operators
- && or and (boolean and)
- || or or (boolean or)
- ! (not)

Variables

Variable names are prefixed with a colon :, must start with a letter or underscore _, and can contain letters, digits or underscores.

:Total
:Account1
:Component_count
:_Temp

Tosca Integration:
▸ When running from Tosca 14.2 and later, setting a variable in VisionScript will also set/delete the corresponding buffer value.

Steering Locators

Locators are used to find controls on screen. There are several in-built locators:

Static (wait for the control to appear - max 5 seconds)
PageUp (press Page Up - max 10 seconds)
PageDown (press Page Down - max 10 seconds)
MouseWheelUp (scroll the mouse wheel up - max 10 seconds)
MouseWheelDown (scroll the mouse wheel down - max 10 seconds)
Standard (Use the following Locators in sequence: Static, MouseWheelUp, MouseWheelDown)

Currently only the Standard Locator is used, in a future release it will be possible to specify a specific Locator, and to create your own custom Locators in Vision Script.

Steering Operators

Operators are used to steer controls in a particular way.

Each control type has a specific Operator associated with it, that performs the steering required for that control type. You can change the Operator with the USING THE "<operatorname>" OPERATOR syntax on supported commands (See the SELECT command for examples). This is handy if the control is not identified with the correct control type.

The following in-built Operators are provided:

Checkbox
Dropdown
List
Menu
TabBar
Toolbar
Tree

In a future release it will be possible to create your own custom Operator in Vision Script.

Vision Script Command Reference

ACTIVATE

Activate an already running application or window. For apps, you may specify a executable name or the name of a short cut in the start menu. Window names are found using a substring match, app names are found using exact match. The wildcard characters * and ? may also be used. If APP or WINDOW is omitted, then windows are matched first by window name, then by app name.

ACTIVATE [APP|WINDOW] "<name>"
ACTIVATE WINDOWID <window-id>

CLICK

Left-click a control or at a specific position on the screen. See also the MOUSE command for further options.

CLICK [IN] <control-specification>
CLICK AT <x>,<y> [IN] <control-specification>
CLICK AT <x>,<y>

The x and y coordinates are specified as follows:

[+/-]<offset>
[+/-]<percent>%
[+/-]<percent>%[+/-<offset>]

Specifying + means a positive offset from the right or bottom. Specifying - means a negative offset from the left or top. Omitting both + and - means a positive offset from the left or top. The percent value represents the proportion of the width or height. Percent values above 100 and below 0 are permitted - this will cause a click outside the control.

e.g.

CLICK the 'Next' button
CLICK 'Submit'
CLICK IN 'Make' input
CLICK IN the 'Price' dropdown

CLICK AT 90%,50% IN 'Make' input      // click near the right end of the control
CLICK AT 100%+20,50% IN 'Make' input  // click 20 pixels outside the right of the control
CLICK AT 50%-20,50% IN 'Make' input   // click 20 pixels to the left of the center of the control
CLICK AT 50%,150% IN 'Make' input     // click center-below the control, by 50% of the height

CLICK AT 50%,50%   // click the center of the current window
CLICK AT 120,300   // click at 120,300 pixels from the top left in the current window

CHECK

Check (or un-check) a CheckBox control.

CHECK <control-specification>
UNCHECK <control-specification>

The CHECK and UNCHECK commands are aliases for the following SELECT commands:

SELECT "Checked" FROM the <control-specification> USING the "checkbox" OPERATOR
SELECT "Unchecked" FROM the <control-specification> USING the "checkbox" OPERATOR

Note: SELECT also accepts the "Indeterminate" value for checkbox selection.

e.g.

CHECK the 'Remember Me' checkbox
UNCHECK the 'Extended Warranty' checkbox

CLOSE

Close an already running application or window. For apps, you may specify a executable name or the name of a short cut in the start menu. Window names are found using a substring match, app names are found using exact match. The wildcard characters * and ? may also be used. If APP or WINDOW is omitted, then windows are matched first by window name, then by app name.

CLOSE [APP|WINDOW] "<name>"

e.g.

CLOSE APP "Google Chrome"
CLOSE WINDOW "Automated Software Testing"
CLOSE WINDOW "Account * | Sales"
CLOSE "chrome"

DELETE

Delete a variable, or all variables.

DELETE :<variable name>
DELETE ALL VARIABLES

e.g.

DELETE :Total
DELETE ALL VARIABLES

Tosca Integration:
▸ When running from Tosca 14.2 and later, deleting a variable in VisionScript will also delete the corresponding buffer value.

ELSE

See the IF command for a full description.

ELSE
ELSE IF <expression> THEN

END

Ends a IF, WHILE, TRY, WITH or SECTION block

END IF|WHILE|TRY|WITH|SECTION

FAILURE

See the TRY command for a full description.

FAILURE

IF

Creates a conditional expression block. See the Expressions section for how to construct Vision Script expressions.

IF <expression> THEN
ELSE IF <expression> THEN
ELSE
END IF

The ELSE statements are optional. Other Vision Script commands can be placed in between, including nested IF statements. Be sure to match each IF with an END IF.

e.g.

IF the 'Next' button is visible THEN
  CLICK 'Next'
ELSE
  Click 'Done'
END IF

IF :Total < 10 THEN
  STORE "Small" IN :Size
ELSE IF :Total < 20 THEN
  STORE "Medium" IN :Size
ELSE
  STORE "Large" IN :Size
  CLICK the 'Oversize' button
END IF

IF 'Back' is visible THEN
  IF color from 'Make' input == "Red" THEN
    CLICK 'Back'
  END IF
END IF

IMAGE

Registers an image file for later identifying image controls

IMAGE "<label>" FROM "<filename>"

e.g.

IMAGE "cut" from "C:\\Images\\cut.png"
IMAGE "copy" from "C:\\Images\\copy.png"

CLICK the 'cut' image

KEYDOWN

Simulate holding down a keyboard key. The key is not released until a corresponding KEYUP command is executed.

The keys are described on the following page and must be used without the prefix VK_: https://docs.microsoft.com/en-gb/windows/win32/inputdev/virtual-key-codes Example: use RETURN instead of VK_RETURN.

KEYDOWN "<key>"

e.g.

KEYDOWN "LWIN"
KEYPRESS "R"
KEYUP "LWIN"

KEYPRESS

Simulate pressing and releasing a keyboard key.

KEYPRESS "<key>"
KEYPRESS "<key>" IN <control-specification>

e.g.

KEYPRESS "TAB"
KEYPRESS "TAB" IN the 'Make' input

If a control is specified, the control will be selected with a click before the key is pressed.

KEYUP

Simulate releasing a keyboard key. See also the corresponding KEYDOWN command.

KEYUP "<key>"

e.g.

KEYDOWN "LWIN"
KEYPRESS "R"
KEYUP "LWIN"

LOG

Add a message to the result log

LOG <expression>

e.g.

LOG "Status: " & :CurrentStatus
LOG "Make: " & the 'Make' input

MOUSE

Perform an operation using the mouse. For simple left-click commands, use the CLICK command instead.

MOUSE [<modifiers>] [<button>] [<operation>] [IN] <control-specification>
MOUSE [<modifiers>] [<button>] [<operation>] [AT <x>,<y>] [IN] <control-specification>
MOUSE [<modifiers>] [<button>] [<operation>] [AT <x>,<y>]

The modifiers can be one or more of the following keys. If not specified, no modifier keys are pressed.

ALT
CTRL
SHIFT

The button can be one of the following values. The word BUTTON is optional. If not specified, it defaults to LEFT BUTTON.

LEFT [BUTTON]
MIDDLE [BUTTON]
RIGHT [BUTTON]

The operation can be one of the following values. If not specified, it defaults to CLICK.

CLICK
LONGCLICK
DOUBLECLICK
DRAG
DROP
HOVER
DOWN
UP

The x and y coordinates are specified as follows:

[+/-]<offset>
[+/-]<percent>%
[+/-]<percent>%[+/-<offset>]

e.g.

MOUSE LEFT CLICK the 'Next' button
MOUSE HOVER IN 'Make' input
MOUSE ALT SHIFT RIGHT DOUBLECLICK AT 50%,50%

MOUSE AT 90%,50% IN 'Make' input      // click near the right end of the control
MOUSE AT 100%+20,50% IN 'Make' input  // click 20 pixels outside the right of the control
MOUSE AT 50%-20,50% IN 'Make' input   // click 20 pixels to the left of the center of the control
MOUSE AT 50%,150% IN 'Make' input     // click center-below the control, by 50% of the height

MOUSE AT 50%,50%   // click the center of the current window
MOUSE AT 120,300   // click at 120,300 pixels from the top left in the current window

MOUSEWHEEL

Perform an operation using the mouse wheel.

MOUSEWHEEL [<modifiers>] <direction> [<steps>] [IN] <control-specification>
MOUSEWHEEL [<modifiers>] <direction> [<steps>] [AT <x>,<y>] [IN] <control-specification>
MOUSEWHEEL [<modifiers>] <direction> [<steps>] [AT <x>,<y>]

The modifiers can be one or more of the following keys. If not specified, no modifier keys are pressed.

ALT
CTRL
SHIFT

The direction must be one of the following values.

UP
DOWN
LEFT
RIGHT

The steps option indicates how many detents to scroll the mouse wheel. If not specified, a value is chosen depending on the window or control size. The word STEPS is optional.

<number> [STEPS]

The x and y coordinates are specified as follows:

[+/-]<offset>
[+/-]<percent>%
[+/-]<percent>%[+/-<offset>]

If coordinates are not specified, the mouse is moved to the center of the window or control.

e.g.

MOUSEWHEEL DOWN in the 'Orders' table
MOUSEWHEEL CTRL UP 4 STEPS   // hold control, and scroll up 4 detents in the current window
MOUSEWHEEL DOWN AT 120,300   // scroll down at 120,300 pixels from the top left in the current window

OPEN

Open an application or document on the local machine. For applications, you may specify a executable name or the name of a short cut in the start menu.

OPEN "<application>"
OPEN "<application>" WITH <param1> [<param2> ...]
OPEN DOCUMENT "<document>"

Opening a document will launch the default program associated with that file type.

e.g.

OPEN "Google Chrome"
OPEN "iexplor.exe" WITH "http://www.tricentis.com"
OPEN DOCUMENT "C:\\Downloads\\invoice.pdf"

REPEAT

Repeat a set of statements until a condition is true

REPEAT
UNTIL <expression>

e.g.

REPEAT
  WAIT 2 SECONDS
  CLICK the 'Refresh' button
UNTIL the 'Operation Complete' text is visible

SECTION

Group set of statements, optionally with a string label. Sections may be nested.

SECTION [<label>]
END SECTION

During execution, the section name (if present) will be logged at the beginning and the end of the section. There is no other effect of the SECTION statement.

e.g.

SECTION
  TYPE "adam" INTO the 'Username' input
  TYPE "12345" INTO the 'Password' input
END SECTION

SECTION "Calculate"
  SECTION "Enter Data"
    TYPE '1' INTO the 'First Value' input
    TYPE '7' INTO the 'Second Value' input
  END SECTION
  SECTION "Compute"
    CLICK 'Add'
  END SECTION
END SECTION

SELECT

Select a value from a control that has multiple values. Use this command for steering dropdowns, checkboxes, lists, trees, menus, toolbars, and tabbars.

SELECT <value-expression> FROM <control-specification>
SELECT <value-expression> FROM <control-specification> USING [THE] "<operator name>" [OPERATOR]

e.g.

SELECT "Audi" FROM the 'Make' dropdown
SELECT "Accounts Payable" from the 'Navigation' toolbar
SELECT "File->Quit" from the 'File' menu

Each control type has a specific Operator associated with it, that performs the steering required for that control type. You can change the operator with the USING THE "<operatorname>" OPERATOR syntax. This is handy if the control is not identified with the correct control type. e.g. If a control was identified as an input, but needs to be steered as a dropdown, you can use the following syntax:

SELECT "BMW" FROM the 'Make' input USING the "dropdown" OPERATOR
// shorter syntax:
SELECT "BMW" FROM 'Make' USING "dropdown"

If a control type does not have a specific operator associated with it, the dropdown operator is used.

Checkboxes

There are three pre-defined values for selecting checkbox states:

"Checked"
"Unchecked"
"Indeterminate"

e.g.

SELECT "Checked" FROM the 'Remember Me' checkbox

If the current value of the checkbox doesn't match the requested value, one or two left clicks are performed in the control.

The CHECK and UNCHECK commands can also be used to steer checkboxes.

Dropdowns

Steering a dropdown is performed internally by clicking in the control, typing the requested value, then pressing TAB. If this does not work for the selected control, use separate statements to steer the control e.g.

CLICK the 'Make' dropdown
CLICK the 'Audi' text WITH IsNew = true

The IsNew = true check ensures only items that have recently appeared are considered.

Typing Options

Several options can be specified after the control type in square brackets that affect how the keys-expression is typed.

KeyDelay Specify the delay in milliseconds to wait after each key-up or key-down action. Default = 10.
HoldDelay Specify the delay in milliseconds to hold each key before releasing it. Default = 0.
ConfirmItemKeys Specify the key sequence to confirm the item and close the dropdown. Default = {Tab}.

e.g.

SELECT "BMW" from the 'Make' dropdown[KeyDelay=100,ConfirmItemKeys="{ENTER}"]

Previous versions:
▸ In Tosca 13.4 / Vision AI 1.35 and earlier, the Typing Options are not available.

Toolbars and Tabbars

Special processing is carried out to separate complicated toolbars into separate items (e.g. ribbon toolbars), and to join multi-line button labels together. The requested value is then found by matching against these items. A left-click is then performed on the item.

You can also use the WITH syntax to perform more complicated steering if required:

WITH toolbar #1
  MOUSE RIGHT CLICK the 'Customize' item
END WITH

Menus

The value for menus can be specified using a hierarchical syntax, e.g. "File->Preferences->Settings". The first item in the sequence is selected using the Toolbar operator, and the subsequent items are selected by finding and clicking matching text items that have recently appeared on screen.

If the menu reacts purely on mouse hover, and the clicks are not desired, specify Menu Steering Options to control the steering behavior.

Lists

The requested value is searched within the single-line text items contained within the list control, and then a left click is performed on the item.

Trees

The value for trees can be specified using a hierarchical syntax, e.g. "Fruits->Pears->Bosc". Each node will be expanded (only if required) by double-clicking (default), and the final node will be clicked. To change this behavior, specify Tree Steering Options on the control.

STORE

Store a value into a variable.

STORE <expression> IN :<variablename>

e.g.

STORE color == "red" for the 'Make' input in :HasMakeError
STORE color from the 'Make' input in :MakeColor
STORE 'Number of Seats' control in :Seats
STORE :Seats + :Wheels * 7 + 1 in :ComponentCount

Tosca Integration:
▸ When running from Tosca 14.2 and later, setting a variable in VisionScript will also set the corresponding buffer value.

TARGET

Target an already running application or window for subsequent commands, without activating it. For apps, you may specify a executable name or the name of a short cut in the start menu. Window names are found using a substring match, app names are found using exact match. The wildcard characters * and ? may also be used. If APP or WINDOW is omitted, then windows are matched first by window name, then by app name.

TARGET [APP|WINDOW] "<name>"
TARGET WINDOWID <window-id>

TRY

The TRY/FAILURE block is used to optionally execute some commands in case of a step failing.

TRY
FAILURE
END TRY

e.g.

TRY
  ACTIVATE WINDOW "New contact"
FAILURE
  OPEN DOCUMENT "C:\\Documents\\New contact.pdf"
  ACTIVATE WINDOW "New contact"
END TRY

TYPE

Type text using the keyboard into a control, or into the current application

TYPE [EXACT|SENDKEYS] <keys-expression>
TYPE [EXACT|SENDKEYS] <keys-expression> IN|INTO <control specification>
TYPE [EXACT|SENDKEYS] <keys-expression> IN|INTO <control specification> USING [THE] "<operator-name>" [OPERATOR]

Typing is performed by emulating keypresses.

If the keys-expression is empty, this indicates the input should be cleared. In this case, the DELETE key is pressed after the select-all sequence.

e.g.

TYPE "120" in the 'Engine Performance' input
TYPE "{Ctrl-F5}"  // Press Ctrl-F5
TYPE "{Ctrl-Shift-A}"  // Press Ctrl-Shift-A
TYPE "Joe{TAB}" in the 'First Name' input
TYPE SENDKEYS "^{F5}"  // Press Ctrl-F5

Note: the USING ... OPERATOR syntax has not yet been implemented. The default operator is the input operator, this performs a click in the control, followed by a special key sequence to select all the text, followed by the specified keys-expression.

Keys Expression

The characters in the expression are typed one by one. Special keys can be typed using the key name inside braces { }, separated by dash -. e.g. {TAB} or {CTRL-F5}.

To type a brace character, use double-braces, or the {OpenBrace} or {CloseBrace} key name. e.g.

TYPE "Count {{5}}" // will output 'Count {5}'.
TYPE "{OPENBRACE}red{CLOSEBRACE}" // will output '{red}'.
TYPE "{CTRL-OPENBRACE}" // press the 'Ctrl-{' key combination

Modifier keys inside braces are held down until the end brace, e.g. {Ctrl-A-B} will hold the Ctrl key, press A, press B, then release Ctrl. {Ctrl-A-Shift-B} will hold Ctrl, press A, hold Shift, press B, release Shift, and then release Ctrl. Newline characters in the keys-expression (\r\n or \n) will be interpreted as pressing the {ENTER} key. The modifier keys are: Ctrl, Shift, Alt, Win, Apps

To type the - character inside braces, use the {Minus} key name, e.g. {Ctrl-Minus}

The full list of key names is described on the following page and must be used without the prefix VK_: https://docs.microsoft.com/en-gb/windows/win32/inputdev/virtual-key-codes Example: use RETURN instead of VK_RETURN. Some common alternatives are also supported, e.g. {ENTER} can be used in place of {RETURN}.

Alternate syntax

If the EXACT keyword is specified, then special keys are not interpreted, and the text is output exactly as entered.

Typing can also be performed using the SendKeys syntax. For more details on the syntax, see the SendKeys documentation. To enable this mode specify the SENDKEYS keyword.

Tosca Integration:
▸ Tosca does not support the SENDKEYS command when interacting with Vision AI Controls, only the VSKEYS command.
▸ When passing keys using the Run VisionScript module, the curly braces must be escaped on the Tosca side by using {{ and }}, e.g. TYPE "Joe{{TAB}}" in the 'First Name' input

Typing Options

Several options can be specified after the control type in square brackets that affect how the keys-expression is typed.

KeyDelay Specify the delay in milliseconds to wait after each key-up or key-down action. Default = 10.
HoldDelay Specify the delay in milliseconds to hold each key before releasing it. Default = 0.
SelectTextKeys Specify the key sequence to select any existing text in the input before typing the keys-expression. Default = {Home-Ctrl-Home-Shift-End}.
DeleteTextKeys Specify the key sequence to delete text (when keys-expression is empty). Default = {Delete}.

e.g.

TYPE "ABC123" in the 'License Plate Number' input[KeyDelay=100,SelectTextKeys="{Ctrl-A}"]
TYPE "" in the 'Surname' input[DeleteTextKeys="{Backspace}"]

Previous versions:
▸ In Tosca 13.4 / Vision AI 1.35 and earlier, the Typing Options are not available.

Typing in Remote Environments

Some remote environments (e.g. Remote Desktop, Citrix, VMware Horizon etc.) only accept keypresses that match the current keyboard. Ensure that the keyboard layout of the client program matches the keyboard layout of the target system, otherwise incorrect characters could be typed.

Vision AI will attempt to send keys that are not possible to type with the current keyboard (e.g. Russian Cyrillic characters with a US keyboard), but they may not be registered on the remote system. Local applications will usually accept these non-keyboard letters without an issue.

SENDKEYS often does not work on remote environments because it does not fully emulate the keyboard. Avoid using the SENDKEYS method for these environments.

Typing in Emulated Mobile Environments

When typing into a mobile control, Vision AI will send the select-existing-text key combination (the SelectTextKeys option) before typing in the control. Usually on a mobile emulator, the default {Home-Ctrl-Home-Shift-End} key combination causes the emulator to scroll the screen around wildly, and the input focus is lost. For these environments, you should set the SelectTextKeys option to the empty string, e.g. [SelectTextKeys=""]

From a Tosca TestCase you can set the SelectTextKeys steering parameter on the module attribute, or use the {VSKEYS[""]} command to enter text.

UNCHECK

Un-check a CheckBox control. See the CHECK command for a full description.

UNCHECK <control-specification>

UNTIL

See the REPEAT command for a full description.

UNTIL <expression>

VERIFY

Verify an expression. Note that control values and properties can be used in an expression, as well as boolean operators.

VERIFY <expression>

e.g.

VERIFY 'Current State' cell == "DONE"
VERIFY color == "red" FOR the 'Make' input
VERIFY color FROM the 'Make' input == "red"
VERIFY :Total > 5 and the 'Fuel Type' dropdown == "Petrol"

When comparing property values, it is preferable to use the FOR syntax over the FROM syntax, because the FROM syntax will only retrieve the most likely value for a property. When using FOR, alternate values for the property are also considered (e.g. similar colors, or OCR variations). You can use the content property for verifying the value of a control.

e.g.

VERIFY content == "DONE" FOR the 'Current State' cell
VERIFY color == "red" FOR the 'Make' input
VERIFY color > 90% "blue" FOR the 'Make' input
VERIFY :Total > 5 && content == "Petrol" FOR the 'Fuel Type' dropdown

WAIT

Wait until a time period elapses and/or an expression is true.

WAIT <number> SECOND[S]
WAIT <number> SECOND[S] UNTIL <expression>
WAIT UNTIL <expression>

e.g.

WAIT 0.5 SECONDS
WAIT 10 SECONDS UNTIL the 'Next' button is visible
WAIT UNTIL color from 'Make' input == "blue"

If both a time and an expression are specified, then the time acts like a timeout. If the timeout elapses, the step will fail. If only an expression is specified, the default timeout is used (60 seconds).

WHILE

Repeat a set of statements while an expression is true.

WHILE <expression>
END WHILE

e.g.

WHILE :Num < 7
  STORE :Num + 1 in :Num
END WHILE

WHILE the 'Next' button is not visible
  WAIT 2 seconds
END WHILE

WITH

The WITH command finds a control on screen, and then isolates all nested commands to work only with controls that are children of the found control. This is useful for tables, dialogs etc.

WITH <control-specification> [AS <control-type-specification>]
END WITH

If the control is not detected correctly, the AS <control-type-specification> clause can be used to treat the control as a different control type.

Some control types generate their own special child controls as part of the WITH statement (e.g. Tables create row, column and cell controls, and toolbar creates item controls).

e.g.

WITH 'Vision AI Execution' table[Method=Gridlines, ColumnThreshold = 30, RowThreshold = 50]
  WITH row HAVING 'Capability' == "Table Steering"
    VERIFY 'Current State' == "DONE"
    VERIFY 'Limitations' == ""
  END WITH
END WITH

WITH 'Error' window
  CLICK 'OK'
END WITH

WITH 'Main' menu as toolbar
  CLICK the 'Copy' item
END WITH

Vision Script Reference

Structure

Control Flow

Control Specification

Quotes

Control Names

Control Types

Control Type Options

Scrolling Options

Table Steering

Table Steering Options

Table Steering Properties

Table Steering Examples

Tree steering

Tree Steering Options

Limitations

List steering

Menu steering

Menu Steering Options

Toolbar and TabBar Steering

Steering Options

Limitations:

Indexing

Wildcard Matching

Property Expressions

Child Expressions

Relative Expressions

Searching

Control values

Datatype conversion

Visibility Expressions

Expressions

Literals

Functions

Operators

Variables

Steering Locators

Steering Operators

Vision Script Command Reference

ACTIVATE

CLICK

CHECK

CLOSE

DELETE

ELSE

END

FAILURE

IF

IMAGE

KEYDOWN

KEYPRESS

KEYUP

LOG

MOUSE

MOUSEWHEEL

OPEN

REPEAT

SECTION

SELECT

Checkboxes

Dropdowns

Typing Options

Toolbars and Tabbars

Menus

Lists

Trees

STORE

TARGET

TRY

TYPE

Keys Expression

Alternate syntax

Typing Options

Typing in Remote Environments

Typing in Emulated Mobile Environments

UNCHECK

UNTIL

VERIFY

WAIT

WHILE