Configure trainable document separation

Use the procedure in this topic to configure trainable document separation.

When trainable document separation is configured, the Advanced Online Learning Options window include online learning for document separation in addition to classification online learning.

To configure trainable document separation (TDS), each class in the project needs many test documents. If you created a Project Planner TDS model in an earlier version of Tungsten TotalAgility, import that model rather than configuring TDS a second time.

Procedure
  1. On the Project tab, in the Configuration group, select Project Settings icon Project Settings.
  2. Select the Document Separation tab to view the document separation settings.
  3. Select Trainable Document Separation (TDS) and then configure the following settings.

    Setting

    Description

    Compatibility Mode

    This setting is disabled unless Trainable Document Separation (TDS) is selected. Select only if you want to use an old separation model created in Project Planner. (Default: Cleared)

    When selected, dynamic online learning for separation is not possible. Any training documents that are collected are used for dynamic classification online learning only.

    How should the TDS classification result be used for document classification

    This setting is disabled unless Trainable Document Separation (TDS) is selected.

    Select one of the following values to configure how the TDS classification result is used to classify a document:

    Keep TDS classification result also for document classification:

    This value is only available if the "Compatibility Mode" setting is selected. Used to accept the TDS classification results for the actual Transformation Designer project, where no document classification is defined.

    Run subtree classification based on TDS classification result if possible:

    Used to accept the TDS classification result and use this as the parent trigger for subtree classification. (Default: Selected)

    If there is no TDS result then the document is reclassified and this result is used for subtree classification. If no subtree classification is defined then it behaves as Keep TDS classification result also for document classification.

    Discard TDS classification result and classify document again:

    Used to ignore the TDS classification results and to reclassify the current document to document classes for the actual Transformation Designer project.

    TDS Model import path

    This setting is disabled unless Trainable Document Separation (TDS) and Compatibility Mode are selected.

    Select the location of the TDS model file and select Import TDS Model to import the file into the project.

    For projects created in Tungsten TotalAgility 5.0 or an earlier version, it is not possible to import a TDS model if there is more than one model. In Tungsten TotalAgility 5.5, you can test and train multiple models as only the most recently built classifiers are imported for each TDS classifier.

    TDS Page Classifier

    This classifier is enabled automatically if you are using Trainable Document Separation and not Compatibility Mode.

    This classifier looks at each page and classifies it and then these page classifications are evaluated to determine the best way of separating the pages into documents.

    Use the Properties icon Properties and Delete icon Reset buttons to manage the page classifier.

  4. Open the Project Tree window if it is not already open.
  5. Open the Details window if it is not already open.
  6. For each class, configure the following Standard Document Separation settings in the Details window.

    Setting

    Description

    Ignore for separation

    When document separation is enabled for a project, you can disable document separation for single classes by selecting Ignore for separation. This means the pages of a document in this class are handled as unclassified pages.

    If the setting is not selected, documents in this class are separated, and several additional settings become available. (Default: Cleared)

    Class represents

    If the "First page" setting is selected, a fixed page length can be set. By default, the value for the fixed page length is 0 (zero). This means the number of pages is unlimited.

    For example, when the fixed page length is set to 3 and document separation is processed for a multi-page document, then the following occurs during processing.

    Document separation processes the multi-page document page by page. For each page classification is performed and in case a page is classified to this class, then a new document is created and the page is added to this new document. As the fixed document length is set to three the following two pages are added to the document without classifying them and regardless if they would belong to another class. After the third page is added, the current document is closed; it contains three pages now. The next page of the multi-page document is processed until all pages of the multi-page document are processed.

    If the value is set to zero and a page of a processed multi-page document is classified to this class, then a new document is created and the page is added. The next page of the multi-page document is added to the current document when:

    • The page is unclassified

    • The page is classified as a Middle page or Last page of the same class as its Corresponding first page

    When a processed page of the multi-page document is classified to another class that is not a middle or last page of the current document, then the current document is closed and the current page is added to a new document.

    After all pages of the multi-page document are processed the next multi-page document is processed.

    If "Middle page" or "Last page" is selected, then the list for Corresponding first page is enabled, allowing a class for the middle or last page to be specified. If this is done, then a middle page (or last page) is added to the currently processed document, when the first page of the current document is classified to the class that is selected for the Corresponding first page setting. Otherwise, the document is closed and the middle (or last) page is added to a separate new document.

    If you define a middle or last page for a first page then the setting Fixed page length for the first page needs to be set to 0 (unlimited) as this setting has priority over other settings. If Fixed page length is set to 1 or higher then the settings for middle or last page are never taken into account as for a fixed page length the pages are added without classifying them.

    If <none> is selected, then the middle page is always added to the current document. For a last page, it works the same way except the document is closed after the page is added and a new document is started for the next processed page of the multi-page document.

    If you define a middle or last page on a document where a fixed page length is defined, these settings are not taken into account as the Fixed page length setting has priority over the other settings.

    Fixed page length

    Available only if you select "First Page" for the "Class represents" setting. If all documents of this type are a fixed page length, you can specify that page length here. This ensures the documents are separated correctly, and makes it easier to determine if a page contains the content for extraction. (Default: 0)

    This means there is no fixed page length, and that documents can have various page lengths.

    Corresponding first page

    Enabled only when "Class represents" is set to Middle page or Last page. Select a class from the list so a document that is classified as the selected class can have the middle or last page added to it rather than creating a new document.

  7. Optional. Enable online learning for document separation.

    Any documents that are not recognized based on your training documents require manual separation and classification during production. Those documents are then collected and the next time a document with that layout is encountered it is separated and classified successfully.

  8. Save the changes to your project.
Next steps
More info