Configure the serving endpoint if required and publish the processes.
1To publish the Get Content from File process, click Actions in the row that contains the process and select Publish.
2Open the Chat with File process.
3On the Temp Fields tab of the Start step, in the ServingEndpointName field, enter the name of the serving endpoint in Databricks Mosaic AI. The default serving endpoint is databricks-meta-llama-3-1-8b-instruct. You can optionally edit the serving endpoint.
4Optionally, in the Prompt Configuration step, enter the prompt instructions in the Assignments field by updating the Prompt_Config field using the Expression Editor, as shown in the following sample code:
For the Prompt_Config field, enter values for the following properties:
Property
Description
Temperature
Controls the randomness of the model's output. A lower value close to 0 makes the output more deterministic, while a higher value close to 1 increases randomness and creativity. For example, if temperature is set to 0.5, the model balances between deterministic and creative outputs.
TopP
Determines the cumulative probability threshold for token selection. The model considers the smallest set of tokens whose cumulative probability meets or exceeds topP. For example, if topP is set to 0.1, the model considers only the top 10% most probable tokens at each step.
MaxTokens
Defines the token count of your prompt. The value can't exceed the model's context length. Most of the models have a context length of 2048 tokens.