› Integration › Generative AI › Integrate TotalAgility with Generative AI

Integrate TotalAgility with Generative AI

Use the following procedure to integrate TotalAgility with Generative AI.

You can add more than one integration for Generative AI providers.

Procedure
  1. Navigate to Integration > Generative AI.

    The Generative AI providers page appears. By default, the Tungsten AI provider is available. You can use this AI provider or configure other Generative AI providers supported by TotalAgility.

  2. Select New to configure other Generative AI providers.

    The New AI provider configuration dialog box is displayed.

  3. On the Type list, select one of the AI providers and configure it to use as a Standard or Vision model.

    • OpenAI Compatible (default)

      This provider type supports any OpenAI API-compatible endpoint. Results, performance, and feature availability may vary when using third-party providers instead of OpenAI.

      Configuration settings

      Description

      ID

      Provide a unique ID for the AI provider.

      Display name

      Enter the AI provider name to help in identifying the server's name in the list.

      Use legacy function syntax

      If selected, TotalAgility uses this "function" syntax to send the request to the AI provider; otherwise, TotalAgility makes use of the latest "tools" function syntax to send the request.

      When you execute Copilot, Generative AI activity, or Generative AI action using Data objects, the legacy or latest function syntax is used, depending on the configured setting. If the model does not support the syntax, an error occurs.

      Use provider for Copilot for development

      If selected, the AI provider is used as the Copilot for development. (Default: Clear)

      Default provider for Copilot for extraction

      If selected, the AI provider is used as the Copilot for extraction. (Default: Clear)

      • Copilot for extraction is accuracy-sensitive. For optimal results, use:

        • Tungsten AI (default) or a tested Azure OpenAI Compatible model (recommended; otherwise, it may impact extraction accuracy and performance. Refer to the Best Practices Guide for more information.

      • When multiple AI providers are configured, you can use only one AI provider (OpenAI Compatible or Azure OpenAI) as the Copilot for development or the Copilot for data extraction.
      Standard Model: Use this model when generating an item without using an image.
      API URL

      Enter the API URL for the selected provider.

      Authentication

      Select the authentication type:

      • API key: Enter the API key. This key is used for authentication/authorization.

      • OAuth: Select the OAuth server configured in TotalAgility. See Configure OAuth server.

        At runtime, when calling an AI provider using an OAuth server, the access token is automatically retrieved and included in the Authorization header. Additionally, the access token is automatically refreshed based on the refresh interval set in the OAuth Server.

      Use custom API key header

      If selected, you can specify an API key header name for API key authentication; otherwise, the default API key header name is used. (Default: Clear)

      API key

      The API key for the Generative AI provider. The supported API key version is 2024-10-21.

      The maximum supported length for the API key is 2,000 characters.

      Model

      Enter the AI provider model, such as gpt-3.5-turbo, to align with your requirement.

      Temperature

      Set the process Temperature for the AI provider. (Default: 0.5, Minimum value: 0, and Maximum value: 2)

      Temperature is a parameter that controls the level of creativity in the AI-generated text. You get a more focused or diverse text based on the temperature set. A higher temperature means the model takes more risks, giving you a mixed response.

      When the AI provider is configured as "Copilot for extraction", the system ignores the temperature parameter and always uses a fixed value of 0. This ensures deterministic, repeatable extraction results; therefore, any change in temperature does not affect the extraction output.

      Reasoning effort

      The reasoning effort parameter controls how much the model thinks before it answers. Optional. Select the reasoning effort level for reasoning models.

      Option Description
      Auto (Default) System automatically determines the reasoning effort level.
      None Use for latency-critical work.
      Low Use for efficient reasoning with tool use, planning, and search.
      Medium Use as a balanced starting point for most workloads.
      High Use for complex debugging and deep planning where quality matters.
      Extra High Use for deep research and agentic tasks with long runs.

      • For OpenAI providers:
        • During process or form creation using Copilot for development and MCP server interactions, set reasoning effort to "None" when using functions or tools.
        • In scenarios without using functions or tools, you can specify an appropriate reasoning effort value for supported reasoning models.
      • For Azure OpenAI:
        • During process or form creation using Copilot for development and MCP server interactions, the reasoning effort parameter is not applicable because these operations rely on functions and tools.
        • Azure OpenAI does not allow the reasoning effort parameter to be used together with functions or tools. As a result, the parameter is automatically excluded when using functions or tools.
      • For non-reasoning models, use "Auto".
      Timeout in seconds

      Configure the timeout period in seconds for the provider. (Default: 300, minimum: 120, and maximum: 3600)

      When you send a request to an AI model, such as a text generation or image processing task, the timeout period ensures the request does not become unresponsive indefinitely. This is helpful as some providers are slower than others, and adjusting the timeout period accordingly can optimize performance.

      For example, if you set a timeout period of 120 seconds for a text generation request, the system waits up to 120 seconds for the AI to generate the response. If it takes longer, the system terminates the request and returns an error.

      Retry count

      The number of times a system retries a failed operation for each provider before giving up and returning a failure response. (Default: 5, Minimum: 0, and Maximum: 100).

      For example, if you set a retry count of 3, the system attempts to resend a failed request up to three times before finally returning an error if the request continues to fail.

      When you use the AI models, the timeout and retry count settings are applied for Copilot, Copilot for extraction, dashboard insights, Copilot insights, and vision usage.
      Vision Model: Use this model when generating an item using an image. If the vision settings are not configured or the specified model does not support images, an error appears.
      API URL

      Enter the API URL for the vision model.

      Authentication

      Select the authentication type:

      • API key: Enter the API key. This key is used for authentication/authorization.

      • OAuth: Selected the OAuth server configured in TotalAgility. See Configure OAuth server.

        At runtime, when calling an AI provider using an OAuth server, the access token is automatically retrieved and included in the Authorization header, in line with the existing support for OAuth servers in other areas. Additionally, the access token is refreshed automatically based on the refresh interval set in the OAuth Server.

      Use custom API key header

      If selected, you can specify an API key header name for API key authentication; otherwise, the default API key header name is used. (Default: Clear)

      API key

      The API key for the Generative AI provider. The supported API key version is 2024-10-21.

      The maximum supported length for the API key is 2,000 characters.

      Model

      Optional. Select the AI provider model, such as gpt-4-vision-preview, if using an image as the input. However, you can change the model to align with your requirements.

      Token limit

      Set the token limit for the provider. (Default: 3000)

      Tokens are word fragments that are generated when the AI provider processes the input. The token limit restricts the number of tokens (usually words) sent to a model per request.

      Timeout in seconds

      Configure the timeout period in seconds for the provider. (Default: 300, minimum: 120, and maximum: 3600)

      Retry count

      Set the retry count for the provider. (Default: 5, minimum: 0, and maximum: 100)

    • Azure OpenAI

      The Azure OpenAI provider type has the same configuration options as OpenAI.

      • You do not need to set the model for the Azure OpenAI provider type, as it is not used by Azure.
      • You do not need to specify an API version in the URL. If an API version is included in the URL, it is automatically ignored.
      • TotalAgility automatically uses the correct API version and URLs based on the provider configuration.

    • Custom LLM

      Configuration settings

      Description

      ID

      Enter an ID for the AI provider.

      The system identifies the AI provider with this ID.

      Display name

      Enter the AI provider name to help identify the server's name in the list.

      Process

      Select the custom service or synchronous process that includes the following variables.

      Variable name

      Variable type

      Initialization

      Inputs:

      Document Content

      String

      Yes

      Document Type

      String

      Yes

      Input Prompt

      String

      Yes

      Seed

      Short

      Yes

      Temperature

      Double

      Yes

      Use seed

      Bool

      Yes

      Document

      Document

      Yes

      Outputs:

      Output

      String

      No

      Error Raised

      Bool

      No

      Status Code

      Short

      No

      Error Message

      String

      No

      If the required process is not available, you can select a new one by selecting Create new process. A synchronous process with the correct input and output variables is automatically created. This process is then available on the Business processes page, and the variables are initialized in the following order.

      Name ID
      Input Prompt INPUT_PROMPT
      Temperature TEMPERATURE
      Document Content DOCUMENT_CONTENT
      Document Type DOCUMENT_TYPE
      Use Seed USE_SEED
      Seed SEED
      Document DOCUMENT

      When a TotalAgility document is used, the Document variable is initialized with the document instance ID at runtime. When a form is configured with the Generative AI chat control or Copilot using AI Agent as the provider and an inline image as the source, then the Document Content variable is initialized with a base64 string of that image.

      • You can change the name of the variables if the initialization order is unchanged.

      • You cannot delete a process if it is configured as a custom LLM AI provider.

    • AI Agent

      The AI Agent provider type has the same configuration options as Custom LLM.

  4. Select Save.
    The configured AI provider appears on the Generative AI providers page.

    • When you open the existing AI provider, the API key is encrypted, and only the last five digits of the key are displayed.

    • You can update or delete an existing Generative AI provider.

More info