LLM : Parameters associated with LLM

# llmBehaviour

Behaviour for the LLM when retriving new visualization
Type: string
Default: append
Options: append, update

# llmFrequencyPenalty

The frequency penalty applies a penalty on the next token proportional to how many times that token already appeared in the response and prompt. The higher the frequency penalty, the less likely a word will appear again. This setting reduces the repetition of words in the model response by giving tokens that appear more a higher penalty
Type: float
Default:

# llmIconSize


# llmIconTransparency


# llmMaxNewTokens

The number of tokens the model generates by adjusting the max length. Specifying a max length helps you prevent long or irrelevant responses and control costs
Type: float
Default: 1250

# llmModel

Large language models are models that use deep learning algorithms to process large amounts of text. They are designed to understand the structure of natural language and to pick out meanings and relationships between words. These models are capable of understanding context, identifying and extracting information from text, and making predictions about a text’s content
Type: string
Default: gemini-1.5-flash
Options: gemini-1.5-flash, gemini-1.5-pro, gpt-4, gpt-4-32k, gpt-35-turbo-16k

# llmPresencePenalty

The presence penalty also applies a penalty on repeated tokens but, unlike the frequency penalty, the penalty is the same for all repeated tokens. A token that appears twice and a token that appears 10 times are penalized the same. This setting prevents the model from repeating phrases too often in its response. If you want the model to generate diverse or creative text, you might want to use a higher presence penalty. Or, if you need the model to stay focused, try using a lower presence penalty
Type: float
Default:

# llmServiceURL

URL for LLM service
Type: url
Default: https://www.canvasxpress.org/cgi-bin/services.pl

# llmTemperature

The lower the temperature, the more deterministic the results in the sense that the highest probable next token is always picked. Increasing temperature could lead to more randomness, which encourages more diverse or creative outputs. You are essentially increasing the weights of the other possible tokens. In terms of application, you might want to use a lower temperature value for tasks like fact-based QA to encourage more factual and concise responses. For poem generation or other creative tasks, it might be beneficial to increase the temperature value
Type: float
Default:

# llmTopp

A sampling technique with temperature, called nucleus sampling, where you can control how deterministic the model is. If you are looking for exact and factual answers keep this low. If you are looking for more diverse responses, increase to a higher value. If you use Top P it means that only the tokens comprising the top_p probability mass are considered for responses, so a low top_p value selects the most confident responses. This means that a high top_p value will enable the model to look at more possible words, including less likely ones, leading to more diverse outputs
Type: float
Default: 1