Creating a Copilot Prompt Configuration

Once you have an LLM connection set up, a Copilot prompt must be configured.

Creating the Prompt

1. In the Graphmarts page, select a Graphmart to create a Copilot prompt.
2. In the explorer pane, select Copilot and then click Create Prompt Configuration.

The Create Prompt dialog displays.

6. Provide the following details:

• Title - A name for the prompt.
• Description - A description for the prompt.
• LLM Connection - The LLM connection to use for the prompt.

8. Click Add to save your prompt.

The new prompt is saved, and its properties are displayed on screen.

Copilot Prompt Properties

Let's take a moment to examine the different properties of a Copilot prompt.

The properties of a Copilot prompt are grouped into four tabs: Settings, Fragments, Templates, and JSON Mapping.

Settings

The Settings tab provides general information on the Copilot prompt, such as its title, description, and other details included in the LLM you supplied for the prompt. In many cases, you will not need to modify most of these settings, although you may want to make changes to settings such as the Copilot Name and Disclaimer.

• Title - The title you provided when the prompt was created.
• Description - The description you provided when the prompt was created.
• Use Embedded Copilot Workflow - When enabled, specifies that the configuration should use the copilot workflow embedded in the LLM. If this option is disabled, you can specify a Rapid Miner Scoring Agent connection and copilot workflow.
• LLM Connection - The LLM connection you provided when the prompt was created.
• Maximum Tokens - The maximum number of tokens the LLM can generate in a single response.
• Maximum Previous Messages - The maximum number of prior conversation messages included as context when sending a request to the LLM. Older messages beyond this limit are excluded from the prompt.
• Copilot Name - The display name used to identify the copilot.
• Disclaimer - An optional, customizable shown to users that communicates usage limitations, legal notices, or responsibility statements related to the copilot’s output.
• Information Text - Static informational text displayed in the user interface to provide guidance, instructions, or contextual details about the copilot or its behavior.
• Query Limit - The maximum number of LLM requests a user or session is allowed to make within a defined time period.
• Max Result Tokens - The maximum number of tokens allowed in the LLM’s generated output after post-processing or filtering.
• Max Retries - The maximum number of times the system will automatically retry an LLM request after a failure, such as a timeout or transient service error.
• LLM Temperature - A parameter that controls the randomness of the LLM’s output. Lower values produce more deterministic responses, while higher values increase variability and creativity.
• LLM TopP - A parameter that limits token selection to the smallest set of tokens whose cumulative probability meets the specified threshold.
• LLM Frequency Penalty - A parameter that reduces the likelihood of tokens being repeated based on how often they have already appeared in the generated text.
• LLM Random Seed - An optional value used to initialize the model’s random number generator.
• LLM Request Timeout - The maximum amount of time the system will wait for a response from the LLM before aborting the request and treating it as a failure.

Fragments

The Fragments tab displays all the fragments used by Copilot to form the full LLM prompt. Fragments are predefined snippets of prompt content than can specify instructions, context, constraints, or examples/formatting rules. Different fragments can be created and added to your prompt depending on your needs.

You can view the contents of a fragment by clicking the dropdown button beside it.

An example AI guideline might read:

Templates

A template is a predefined structure that organize fragments into a complete prompt. All of the fragments configured for your prompt are saved to a template.

JSON Mapping

The JSON mapping defines the format of the LLM's response to a Copilot prompt. The mapping specifies the expected JSON fields and value types, thereby allowing the Copilot system to correctly parse, validate, and use the model’s output programmatically.