› Tungsten Copilot › Configure the Generative AI provider

Configure the Generative AI provider

If the Generative AI provider is not already configured in the TotalAgility Advanced Studio, you are prompted to configure it when you first open Tungsten Copilot.

Procedure

  1. On the TotalAgility Studio Home page, select the Tungsten Copilot card.

    You are prompted to configure a Generative AI provider.

  2. Select Configure.

    The New AI provider configuration dialog box is displayed.

  3. On the Type list, select one of the AI providers and configure it to use as a standard model or vision model.

    Configure the standard model settings when you do not want to use an image to generate an item.

    Configure the vision model settings when you want to use an image to generate an item. If the vision settings are not configured or the specified model does not support using images, an error appears.

    • OpenAI (default)

      Configuration settings

      Description

      ID

      Provide a unique ID for the AI provider.

      Display name

      Enter the AI provider name to help in identifying the server's name in the list.

      Use legacy function syntax

      The "function" syntax used by TotalAgility to send the request to the AI provider. (Default: Selected and read-only)

      Use provider as

      Copilot function Description

      Copilot for development

      The read-only field is selected by default as there should be at least one AI provider used as the Copilot.

      You can have multiple AI providers but use only one AI provider (OpenAI or Azure OpenAI) as the Copilot for development.

      Copilot for extraction

      If selected, the AI provider is used as an Extraction provider. (Default: Clear)

      When multiple AI providers are configured, you can use only one AI provider (OpenAI or Azure OpenAI) as Copilot for data extraction.

      Standard Model
      API URL

      Enter the API URL for the selected provider.

      Authentication

      Select the authentication type:

      • API key: Enter the API key. This key is used for authentication and authorization.

      • OAuth: Select the OAuth server configured in TotalAgility. Refer to Configure OAuth server in the TotalAgility Advanced Studio Help.

        At runtime, when calling an AI provider using an OAuth server, the access tokens are automatically retrieved and included in the Authorization header, in line with the existing support for OAuth servers in other areas. Additionally, the access token is refreshed automatically based on the refresh interval set in the OAuth Server.

      Model

      Enter the AI provider model, such as gpt-3.5-turbo, to align with your requirements.

      Temperature

      Set the process Temperature for the AI provider. (Default: 0.5, Minimum value: 0, and Maximum value: 2)

      Temperature is a parameter that controls the level of creativity in the AI-generated text. You get a more focused or diverse text based on the temperature set. A higher temperature means the model takes more risks, giving you a mixed response.

      Timeout in seconds

      Configure the timeout period in seconds for each provider. (Default: 300, minimum: 120, and maximum: 3600)

      When you send a request to an AI model, such as a text generation or image processing task, the timeout period ensures that the request does not become unresponsive indefinitely. This is helpful as some providers are slower than others and adjusting the timeout period accordingly can optimize performance.

      For example, if you set a timeout period of 120 seconds for a text generation request, the system will wait up to 120 seconds for the AI to generate the response. If it takes longer than 120 seconds, the request will be terminated, and an error will be returned.

      Retry count

      Set the retry count for each provider. (Default: 5, minimum: 0, and maximum: 100)

      The retry count is the number of times a system attempts to retry a failed operation before giving up and returning a failure response. If you set a retry count of 3, the system resends a failed request up to 3 times before finally returning an error if the request continues to fail.

      When you use the AI models, the timeout and retry count settings are applied for Copilot, Copilot for extraction, dashboard insights, Copilot insights, and vision usage.

      Vision Model
      API URL

      Enter the API URL for the vision model.

      Authentication

      Select the authentication type:

      • API key: Enter the API key. This key is used for authentication and authorization.

      • OAuth: Select the OAuth server configured in TotalAgility. Refer to "Configure OAuth server" in the TotalAgility Advanced Studio Help.

        At runtime, when calling an AI provider using an OAuth server, the access tokens are automatically retrieved and included in the Authorization header, in line with the existing support for OAuth servers in other areas. Additionally, the access token is refreshed automatically based on the refresh interval set in the OAuth Server.

      Model

      Optional. The AI provider model to generate an item. For example, gpt-4-vision-preview. You can change the model to align with your requirements.

      Token limit

      Tokens are word fragments generated when an AI provider processes the input. The token limits restrict the number of tokens sent to a model per request. (Default token limit for the provider: 3000)

      Timeout in seconds

      Configure the timeout period in seconds for each provider. (Default: 300, minimum: 120, and maximum: 3600)

      Retry count

      Set the retry count for each provider. (Default: 5, minimum: 0, and maximum: 100)

    • Azure OpenAI

      The Azure OpenAI provider type has the same configuration options as OpenAI.

      You need not set the model for the Azure OpenAI provider type, as it is not used by Azure.

    • Custom LLM

      Configuration settings

      Description

      ID

      Enter an ID for the AI provider.

      The system identifies the AI provider with this ID.

      Display name

      Enter the AI provider name to help in identifying the server's name in the list.

      Process

      Select the custom service or synchronous process that has the following variables.

      Variable name

      Variable type

      Initialization

      Inputs:

      Input Prompt

      String

      Yes

      Temperature

      Double

      Yes

      Outputs:

      Output

      String

      No

      If the required process is not available, select the Create new process link to automatically create a synchronous process with the correct input and output variables.

      You cannot delete a process if it is configured as a custom LLM AI provider.

    • AI Agent

      Configuration settings

      Description

      ID

      Enter an ID for the AI agent.

      The system identifies the AI agent with this ID.

      Display name

      Enter the AI agent name to help identify the server's name in the list.

      Process

      Select the custom service or synchronous process that has the following variables.

      Variable name

      Variable type

      Initialization

      Inputs:

      Input Prompt

      String

      Yes

      Seed

      String

      Yes

      Temperature

      Double

      Yes

      Use seed

      Bool

      Yes

      Outputs:

      Output

      String

      No

      Error Raised

      Bool

      No

      Status Code

      Short

      No

      Error Message

      String

      No

      If the required process is not available, select the Create new process link to automatically create a synchronous process with the correct input and output variables. The process appears on the business processes list page.

      You cannot delete a process if it is configured as an AI Agent provider.

  4. Select Save and then select OK.
    The AI provider is configured, and you are redirected to Copilot.

    The AI provider configured here appears on the Integration menu > Generative AI providers page in the TotalAgility Advanced Studio. When you open the AI provider integration, the API key for the standard and vision models is encrypted, and only the last five digits of the key are displayed.