ISA option

The ISA (Intelligent Screen Automation) option provides extended screen recognition capabilities by automatically finding UI elements and shapes.

Do not use Intelligent Screen Automation (ISA) in Attended automation.

ISA helps you automate applications with limited or no automation API, such as Citrix. Also, use this option to automate applications that are slow, such as Remote Desktop Protocol applications, SAP, and others.

In general, use this option for any application for which the application tree is incorrectly populated or is otherwise difficult to create a finder for.

For information about automating applications with the ISO option, see the following topics:

Common attributes

To understand the common attributes of an element, review the following:

der_x, der_y, der_width, der_height: Size and coordinates of the element.
lt_16_5, rb_15_4: Calculated numbers relative to the element's left top and right bottom coordinates. Element coordinates might change in different program states.

For example, a selected option with bold text can have different coordinates than the cleared option, and therefore finders might not work correctly. This property eliminates coordinate change issues, and you can use it to create reliable finders.

If a UI element has a name property, it is used in a finder by default when you insert an action step. Otherwise, property lt_16_5 is used in a finder.

UI elements

Each recognized UI element contains several attributes to use in your finders. The following table lists supported UI elements.

UI element	Widget name	Element attributes	Notes
Dialog box, frame, pane, and others	container	N/A	A general parent element that contains other elements. For example, an entire dialog box or a form inside a dialog box can be a container. One of the special containers that Tungsten RPA can recognize is a table.
Element	element	N/A	A general UI element that does not contain other elements.
Icon	icon	label	A graphical element that does not contain any text.
Table	table	N/A	Tables are represented in an application tree as separate elements with the following restrictions: Merged cells are not recognized. Headers are the same as regular cells. Each cell is defined as an item in the application tree. Items can contain other UI elements, such as check boxes, text boxes, and so on.
Row	row	N/A	A row in the table tree.
Item	item	N/A	An item within a row in the table tree.
Text box	textbox	label	A text field.
Check box	checkbox	label	A check box.
Option button	radiobutton	label	An option button or radio button.
Text and text label	linklabel	name inverted	This is either a standalone text element such as a dialog box subtitle or a UI element label such as a check box label. name: If a text is recognized as a UI element label, the UI element contains a label property with the label name as its value. In the following example, textbox element contains label="Name" property. inverted: If the text detected by OCR is dark on a white background, inverted is set to 0. If the text is white on a dark background, inverted is set to 1.

ISA tips

To improve the recognition results, consider the following tips.

Recognition results depend on many factors such as screen fonts, text size, background and foreground color, and so on. To improve results, try using different color schemes in your interface, such as regular monospaced black fonts on a bright background and larger font sizes.
Selected text with inverted colors in a text field might not be accurately recognized. To improve results, remove the selection from the text field (such as by clicking somewhere else) before extracting its value.
When you work with dynamic content forms and text fields that might be recognized differently in different states, use the Freeze Tree step to hold the application tree state and improve recognition results.
To extract a value from an element, use the Extract Value action as you would when using Windows automation API. If the element's value is not recognized in ISA mode, try Extract Text From Image action.
Edit extended OCR settings in the ocr.cfg file. See Extended OCR Settings for more information.
Train Tesseract to recognize your character set using either TTF fonts or UI screen shots. See Train Tesseract for more information.
Before automating SAP applications, set System Dependent Theme in the SAP GUI Options for better recognition results.
If you connect remotely to a Windows automated device via RDP connection, resolution and color depth depend on the network connection speed between your computer and the remote device. Different connection speeds in different robot runs may lead to issues if robots use Image Finder and ISA.
- For Image Finder and ISA, it is very important to always receive the same picture (pixel-by-pixel) in the same application state.
- Always use the same explicitly specified resolution and color depth parameters (g: desktop geometry (WxH) and a: connection color depth) for the RDP connection in the Open step.
  
  Example: rdp://user1:MyPassword@MyDesktop?g=640x480&a=16

Change or add UI recognition language

By default ISA uses the language specified in the ocr.cfg file. To add or change the default language specified in this file, see Change OCR engine and language.

To change the ISA UI recognition language for a computer:

In a text editor, open the isa_v1.cfg file in the lib directory as follows.
- On a Windows computer used with Desktop Automation Service:
  
  C:\ProgramData\Tungsten RPA\<build number>\lib\tessdata
  
  If you have several versions of the Desktop Automation Service installed on the automated computer, locate the version of the service that automates the desktop using ISA. Refer to the "Windows tab" in the Tungsten RPA Desktop Automation Service Guide.
- On a local Windows computer used with built-in browser, built-in Excel, or PDF readers, navigate to the RPA installation directory:
  
  nativelib\hub\windows-x64\<build number>\xxx\lib
  
  The build number varies in different versions of the program.
  
  Example:
  
  C:\Program Files\Tungsten RPA 2026.1.0.0\nativelib\hub\windows-x64\622\lib
- On a local Linux computer used with built-in browser, navigate to the RPA installation directory:
  
  nativelib/hub/linux-x64/<build number>/lib
  
  Example:
  
  Tungsten_RPA_2026.1.0/nativelib/hub/linux-x64/533/lib
In the ocr_language parameter, either replace default with a language of your choice or, to use more than one language, add another language using the plus sign, such as ocr_language=eng+jpn.
Save and close the file.
Restart the Desktop Automation Service for the changes to take effect.

To return to the default recognition language, perform the previous steps, specifying default in the ocr_language parameter.

Add UI recognition language for Tesseract

If you use the Tesseract OCR engine, ISA can automate the English language UI by default.

For information on changing the recognition language, refer to "Change default OCR language" in the Tungsten RPA Desktop Automation Service Guide.

OmniPage OCR engine includes all supported languages in the Tungsten RPA installation.

Using more than one language simultaneously for screen recognition slows down robot execution and degrades recognition results.

Use the following procedure to add more languages for UI recognition.

Download the .traineddata file for the required language from GitHub.

For example, the file for the Japanese language is jpn.traineddata.

If the files are not available, search for version 3.04 from the GitHub main repository page.

Tesseract version 3.05.00 is compatible with Tungsten RPA 2026.1. However, RPA uses Tesseract 3.04.00 training files.

Make sure the downloaded .traineddata file is compatible with your version of the teserract.dll file. For compatibility information, see the README.md file published on the GitHub tesseract-ocr/tessdata traineddata page.

Using the wrong training data does not result in error messages, but may cause poor OCR results.
Copy the downloaded .traineddata file to the appropriate directory.
- On a Windows automated computer used with Desktop Automation Service:
  
  C:\ProgramData\Tungsten RPA\<build number>\lib\tessdata
  
  If you have several versions of the Desktop Automation Service installed on the automated computer, locate the version of the service that automates the desktop using ISA. Refer to the "Windows tab" in the Tungsten RPA Desktop Automation Service Guide.
- On a local Windows computer used with built-in browser, built-in Excel, or PDF readers, navigate to the RPA installation directory:
  
  nativelib\hub\windows-x64\<build number>\lib\tessdata
  
  The build number varies in different versions of the program.
  
  Example:
  
  C:\Program Files\Tungsten RPA 2026.1.0.0\nativelib\hub\windows-x64\622\lib\tessdata
- On a local Linux computer used with built-in browser, navigate to the RPA installation directory:
  
  nativelib/hub/linux-x64/<build number>/lib/tessdata
  
  Example:
  
  Tungsten_RPA_2026.1.0.0/nativelib/hub/linux-x64/533/lib/tessdata
Change the UI recognition language as described in Change or add UI recognition language.