› Use Transformation Designer › Manage databases › Fuzzy databases › Add a local fuzzy database

Add a local fuzzy database

Use the procedure in this topic to add a local fuzzy database

Local fuzzy databases are used in a locator or in a script. Ensure the .CSV or .TXT file for the database is formatted correctly.

It is best practice not to exceed the supported number of records for a local fuzzy database. The default number of supported records is set to five million records. Databases may exceed this limit, but more memory is required. However, even with increased memory allocated, processing speed may be decreased. Alternatively, you can use a Remote Fuzzy Database that is managed through the Tungsten Search and Matching Server.

Procedure

  1. On the Project tab, in the Configuration group, select Project Settings Project Settings icon.
  2. Select the Databases tab to view the databases settings.
  3. In the Databases group, select Add. The Add New Database window is displayed.
  4. In the Database Name field, enter a descriptive name for the new fuzzy database.
  5. On the Database Type list, select the Local Fuzzy Database type.
  6. Select OK. The Local Fuzzy Database Properties window is displayed.
  7. On the Local Fuzzy Database properties window, set the following settings.
    1. Select where the Referenced import file (text or csv file) is located, either File system or Web.

      Depending on your selection, do one of the following.

      Selection

      Action

      Web

      Select Browse URL and type in the desired web address.

      If the selected file is protected, select Yes and provide a User Name and Password.

      Select Test to ensure that the URL is available.

      A copy of the source database file is imported into the file system hierarchy for the project and a list of column is displayed in the Column Configuration group.

      File system

      Select Browse and navigate to the required .txt or .csv file.

      A copy of the source database file is imported into the file system hierarchy for the project and a list of column is displayed in the Column Configuration group.

    2. In the Column Configuration group, for each column select Search and/or Filter to determine the column behavior during the fuzzy search.
    3. Optional. In the Column Configuration group, it is possible to rename a column name by first selecting its row and then selecting on the column name. Press Enter to save the change.

      After renaming a column, the preview below is updated automatically.

    4. In the Import Options group, configure the following settings as needed.

      Setting

      Behavior

      Ignore case

      Select if case does not matter in your database. This means that all strings are converted to lower case automatically when a search is performed. This setting does not apply to filtering.

      Filtering is case sensitive

      Available when one or more fields in the Column Configuration table have the Filter setting selected.

      Select if you want the filtering value entered into the script to match the entries in the database exactly.

      First line contains caption

      Select if your Local Fuzzy Database contains the column header names.

      Field delimiter

      Enter the field delimiter character that separates entry fields in your database source file. (Default: ;)

      Optional. Select Tab to use the Tab character as an additional field delimiter.

      Word separation characters

      If your database contains compound words, enter the characters that separate compound words. (Default: -,)

      Optional. Select Tab and Space to use these characters to also separate words.

      Characters to ignore

      Enter the characters to be ignored. These characters are excluded from the database fields during import, and from the search text before processing. (Default: ."'!)

      Optional. Select Space as an additional character to ignore.

    1. Optional. Add, delete, modify, or import string substitutions in the String Substitution group.
    2. In the Optimization group, configure the following settings as needed.

      Setting

      Behavior

      Automatic update from import file

      Updates the database automatically when the source file changes. The automatic update occurs when the project gets loaded. (Default: Cleared)

      Load database index in memory

      Keeps the fuzzy database in memory that helps process the fuzzy search more efficiently. Clearing this field reduces the memory usage but also significantly slows fuzzy searches. (Default: Selected)

      If this setting is cleared, the Filter setting in the Column Configuration group is no longer available.

      Database processing

      Indicates the quality of database processing to optimize your database searches.

      If this is set to Basic, the Filter setting in the Column Configuration group is no longer available. The `Basic setting also uses an older algorithm that delivers results with less precision. especially for large databases. However, the speed of processing smaller databases may be quicker when using this setting.

  8. Select OK to save the Local Fuzzy Database.

    The properties window is closed and the new Local Fuzzy Database is displayed in the list of databases in the Databases group.

  9. Optional. Select OK to close the Project Settings window.
  10. Save the changes to your project.

More info