Adding Annotators to the Pipeline

After adding crawlers, the next step is to add one or more annotators. Annotators extract facts or references in the text as annotations.

1. In the pipeline, click the Annotators tab.

   

2. Next, click the Add Output button. Graph Studio opens the Add Component dialog box. The New tab is selected and lists the available annotators and the Existing Components tab lists annotators that have been previously configured for other pipelines.

   

3. To add a new annotator to the pipeline, click the annotator name to select it. To add an existing annotator to the pipeline, click the Existing Components tab, and then select an annotator. The list below describes each of the default annotators:

   • Custom Relationship Annotator: Include this annotator to map relationships between annotations based on the number of characters between the annotations.
   • External Service Annotator: Include this annotator to hit an HTTP endpoint that provides annotations.
   • Keyword and Phrase Annotator: Include this annotator to create annotations based on the phrases that you specify.
   • Knowledgebase Annotator: Include this annotator to link structured and unstructured data by finding instances in data layers, graphmarts, or Graph Studio linked datasets. Based on the names and aliases of entities present or patterns that are indicative of the entities, this annotator marks up the documents with the structured entities linked.
   • LLM Annotator: Include this annotator to generate annotations (e.g., categories) to specific entities in a dataset.
   • Regex Annotator: Include this annotator to use regular expression rules to identify entities such as email addresses, URLs, phone numbers, or any other entity that can be matched using a regular expression.

4. After selecting an annotator, click OK. Graph Studio opens the Create dialog box for the component. Complete the fields to configure the annotator. The list below provides details about the settings for the annotators that are typically used in pipelines. Click an annotator name to view the details for that component:

   • External Service Annotator
   • Keyword and Phrase Annotator
   • Knowledgebase Annotator
   • LLM Annotator
   • Regex Annotator

   External Service Annotator

   

   For information about the options that are presented when you edit an External Service Annotator, see Annotator Settings Reference.

   • Title: Required field that specifies the unique name for this annotator.
   • Description: Optional field that provides a description of this annotator.
   • HTTP Request Config: Required field that specifies the HTTP source object that contains the URL and method to use when sending data for annotations.
   • Document ID Response Path: Required field that specifies where to find the document ID in the response.
   • Entity Name Path: Required field that specifies the annotation object name path.
   • Entity Class Path: Required field that specifies the class URI for an annotation.

   Keyword and Phrase Annotator

   

   For information about the options that are presented when you edit a Keyword and Phrase Annotator, see Annotator Settings Reference.

   • Title: Required field that specifies the unique name for this annotator.
   • Description: Optional field that provides a description of this annotator.
   • Phrase: Required field that specifies the terms or phrases to annotate. Type a word or phrase in the field and then click Add to add the phrase. You can add any number of phrases.

   Knowledgebase Annotator

   

   For information about the options that are presented when you edit a Knowledgebase Annotator, see Annotator Settings Reference.

   • Title: Required field that specifies the unique name for this annotator.

   • Description: Optional field that provides a description of this annotator.

   • Backing Graphmart: Optional field that specifies the graphmart or graphmarts to annotate.

     If you want the annotator to run against a linked dataset or Graph Studio knowledgebase instead of a data layer or graphmart, leave the Backed Layer and Backed Graphmart fields blank. After saving the pipeline, you can edit the pipeline and specify a Backed Dataset at that time.

   • Backing Layer: Optional field that specifies the data layer or layers to annotate.

     The Backing Layer and Backing Graphmart fields are treated independently. Layers that you select do not have to be part of the graphmart that you specify in Backing Graphmart. And specifying a layer does not mean that you must select a Backing Graphmart. However, any layers or graphmarts that you select must contain classes and properties from the Backing Ontology or the data will not be annotated.

   • Backing Ontology: Required field that specifies the model for the backing data layers and/or graphmart. Click the field and select a model from the drop-down list.

   • Term Class: Required field that specifies the class of data for the annotations.

   • Term Label Property: Required field that lists the primary name or label property of the resources.

   • Term Identifying Properties: Required field that specifies the properties that contain names, aliases, or other identifiers to use for identifying the resources.

   LLM Annotator

   

   • Title: A unique identifier for your LLM connection.
   • LLM Endpoint: The URL of the LLM service.
   • LLM Provider Name: The name of the LLM provider.
   • LLM Model Name: The name of the model to use for LLM operations.
   • LLM API Key: The authentication token granting access to the LLM service. This same key should be provided in the Confirm LLM API Key field.
   • Can Use LLM Connection: The roles granted permission to use the LLM connection.

   For information about the options that are presented when you edit an LLM Annotator, see Annotator Settings Reference.

   The LLM Annotator Service must be set up and connected to an LLM Service before the LLM Annotator functionality can be utilized.

   Regex Annotator

   

   For information about the options that are presented when you edit a Regex Annotator, see Annotator Settings Reference.

   • Title: Required field that specifies the unique name for this annotator.

   • Description: Optional field that provides a description of this annotator.

   • Regular Expression Rule: Required field that lists the regular expression rules for this annotator. To add a rule, click drop-down field and select Create New. Graph Studio opens the Create Regular Expression Rule dialog box where you can define the rule:

     

     • Title: Required field that specifies the name of the rule.
     • Class Structure: Required field that specifies the class in the model that should be created for this rule. The value should be in the format group_number:class_name, where group_number corresponds to a group in the regex capture. Each rule should start with group 0. Include groups 1 and higher if needed to represent parts of the expression that are contained in parentheses. The class_name is a label that describes the type of data the rule will find. For example, for a rule that finds hyphenated words 0:Hyphens.
     • Description: Optional field that describes the rule.
     • Regular Expression: Required field that specifies the regular expression to use for finding matching entities.

5. When you have finished configuring the annotator, click Save. Graph Studio adds the annotator to the pipeline and returns to the Annotators screen. For example:

   

6. If you want to change the annotator configuration, click the Edit icon () for the annotator and modify the settings as needed (see Annotator Settings Reference for information about settings). If you want to add another annotator to the pipeline, repeat the steps above.

7. When you have finished adding annotators to the pipeline, proceed to Run the Pipeline.