Configuration of AutoTagging for Cumulus

Clarifai account

To connect your Cumulus server with the API of your Clarifai account you have to create an application within your Clarifai account page. Navigate with your browser to to following page to create a PAT (personal access token):

If you do not have an account yet you have to create one at first.

Create an application if not already done and save the API key we need in the next step. Also take a look at the default language and change it to english because the language used will be set later in the configuration of the Cumulus scheduler job.

Licensing

To use the plugin you need an activation key from CDS Gromke e.K.. Since we bind the usage of the plugin to your Cumulus hardware ID we need your Cumulus server hardware ID to create an activation key. You find your Cumulus server hardware ID in the Remote-Admin -> License dialog within the Cumulus Serverconsole.

Please send your Cumulus server hardware ID via ag@cds-gromke.com to CDS Gromke e.K. to receive an activation key.

Setting up the Scheduler job

To use the plugin you have to create a new Scheduler job in the Scheduler-Manager within the Cumulus Serverconsole.

For example create a new job called "AutoTagging". In the main Scheduler job dialog you have to select the type "AutoTagging" and your catalog you want to work with. Also define the timing of the Scheduler job. If you want that the job looks for new records every 30 seconds you can define a timing defined by Cron notation like this:

0,30 * * * * ?

Open the configuration dialog by clicking on "Configure..." to set up the configuration of the Scheduler job.

Settings: General

Screenshot

Scheduler job configuration: General settings

AutoTagging Activation Key

In this input field you have to enter the activation key you have received from CDS Gromke e.K. (Contact Us!).

Clarifai API key

In this input field you have to enter the Clarifai API key you can find in your application in your Clarifai account (https://clarifai.com/settings/security).

Cumulus Administrator and Password (only in previous versions; before version 2018-01-13)

The older version of AutoTagging needed access to the Cumulus server with the Cumulus admin user. If your Cumulus Scheduler is not running with the Cumulus admin user (configured in the config.xml file within the Cumulus Scheduler installation path) you have to enter the name and password of your Cumulus admin user here. Otherwise you can leave these two fields blank. Entering the Cumulus administrator credentials here will not change the user under which the actual Scheduler job runs.

Search-Query for Tagging

Here you can select the imported record query "Not autotagged yet or Pending". It is also possible to change this query in the Cumulus client or create a new one (e.g. if you want to separate different taggings into different scheduler jobs or only want to include images).

Metadata Template

If you want you can apply a metadata template to the records after the tagging was successful. This is helpful to have control over specific field values for tagged records and therefore makes it easier to have multiple Scheduler jobs running in a row.

Settings: Tagging

Screenshot

Scheduler job configuration: Tagging settings

Tagging Language

If you want to use another language than the default one within the Default Model it can be set here. Choose the language code (e.g. fi) from a list of supported languages that can be found here: https://clarifai.com/developer/guide/languages#supported-languages

Important Note: If possible use always English as the default language in the application within your Clarifai account.

Maximum Thumbnail Dimensions

This sets the size of the uploaded thumbnail. Please note that this is a maximum value and depends on the actual thumbnail size of your records in the Cumulus catalog. Very small values will give less accurate results during tagging and very large thumbnail sizes will send more data to Clarifai and may take a second longer.

Tagging Delay

Sets the optional interval (in milli seconds) between Clarifai API request to avoid the problem of having too many request that would result in blocking the request. This is sometimes needed if the application language has been set to something other than english.

Text Field for Tags (GUID) (optionally)

If you want to save the recognized keywords into a record text field then you can enter a record field GUID optionally. For example you could use the standard field "Notes":

{af4b2e0b-5f6a-11d2-8f20-0000c0e166dc}:Notes

Text Field Tag Separator (optionally)

The default separator for tags saved in a text field is a semicolon with a white space. If you want to change the default value you can enter an own separator here.

Fill mode

Depending on already existing field values it might make sense to decide how the newly recognized tags should be saved in the text field. So you can choose between the following three possibilities for saving the tags:

  • Replace: Always saves the tags to the text field
  • Fill if empty: only stores tags if there is not already an existing value
  • Append: Appends the values in addition to already existing data and separates it with the defined separator

Add Keywords to $Keywords (older versions only, before version 2018-01-13)

By activating the checkbox "add keywords" all recognized keywords will be saved under $Keywords within the category tree and assigned to the records. There will also be created a top level keyword entry for every tagging model for a better overview of the recognized keywords. It is no problem to use your own keywords together with the recognized ones from Clarifai.

Root Category for Keywords (optionally)

If you want AutoTagging to add the detected keywords to the categories tree you have to define in this configuration field the root category to use for the keywords. If you leave it empty no keywords get added to the categories tree. If the entered root category does not exist it will be created as a master category in the catalog. There will also be created a top level keyword entry for every tagging model for a better overview of the recognized keywords. It is no problem to use your own keywords together with the recognized ones from Clarifai.

Add face marker in comments (optionally)

If you want to detect faces in images the result is stored by default in a table field custom made for the list of detected faces with properties like age, gender and culture. Additionally you can store the result in the standard comments table field to visualize it with rectangles shown directly in the preview window in Cumulus. Just activate the checkbox to use this feature.

Clarifai Probability

This slider sets the probability treshold value for the recognized keywords. The best value depends on your images, how many keywords you want and how accurate they should be. A good start for testing this is 90.

Settings: Black and white listing

Screenshot

Scheduler job configuration: Black and white list settings

Black / White Listing Mode

It is possible to use the black or white list feature to exclude or exclusively include specific tags for better control of the tagging result.

List of Tags for Black / White Listing

In this text area you can enter a list of tags you want to use if black listing or white listing is enabled in this scheduler job configuration. The list can contain as many tags as you want and the tags can be separated by comma or semicolon or line breaks.

Settings: Replace Mapping

Screenshot

*Scheduler job configuration: Replace mapping settings

List of words to replace

You can define your own list of predicted tags you want to replace with other words. All tags from one line gets replaced by the first tag in that line. This can help you to control the resulting list of tags.

Running and using the Scheduler job

Now activate the new Scheduler job. At this point the plugin should not tag any image automatically since no record should have the record field value "not autotagged yet" in the record field "Autotag". New images will be tagged automatically since the default values for the control fields "Autotag", "Autotag_Model", "AutoTag Classification" and "AutoTag Color" are prepared accordingly. This can be changed in the Cumulus catalog settings for these fields.

Tag existing images

If you want the Scheduler job to automatically tag existing images then you have to change the field value of the record field "Autotag" to "Not autotagged yet" and select a tagging model in the field "AutoTag_Model" (for good results use "general-v1.3"). You can also activate the fields "AutoTagColor" and "AutoTag Classification" if you want to use these features. For easier access to these field values you can use the imported record view set "AutoTagging" and the asset information window within the Cumulus client.

Automatically tag new images

All important fields for tagging have default values and leads therefore to automatic tagging of new images. The default values are the following:

  • AutoTag: Not autotagged yet
  • AutoTag_Model: generalModel
  • AutoTagColor: true

Please note! Since all three fields are activated by default the tagging for every record will consume three requests from your Clarifai account. You can change this by changing the default values.

Only tag new images in a specific category

If you want to narrow the processed records to ones assigned to a specific category for example then you can change the imported record search query to the following with the help of the Cumulus client search window:

"{df7dd4fb-c6c0-4d54-a012-669fa9a9a166}:Autotag" == 1 && "{af4b2e0c-5f6a-11d2-8f20-0000c0e166dc}:Kategorien" == ":B 5:"

This way only records assigned to the category with the id 5 or a subcategory will be processed if they have the status of "Not autotagged yet".