UserWarning: For dense input, both fused kernels require query, key and value to have the same num_heads. Query.sizes(): [1, 32, 11861, 128], Key sizes(): [1, 8, 11861, 128], Value sizes(): [1, 8, 11861, 128] instead.
By default, models imported into Ollama have a default template of `{{ .Prompt }}`, i.e. user inputs are sent verbatim to the LLM. This is appropriate for text or code completion models but lacks essential markers for chat or instruction models.
Omitting a template in these models puts the responsibility of correctly templating input onto the user. Adding a template allows users to easily get the best results from the model.
To add templates in your model, you'll need to add a `TEMPLATE` command to the Modelfile. Here's an example using Meta's Llama 3.
```dockerfile FROM llama3.2
TEMPLATE """{{- if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|> {{- end }} {{- range .Messages }}<|start_header_id|>{{ .Role }}<|end_header_id|>
{{ .Content }}<|eot_id|> {{- end }}<|start_header_id|>assistant<|end_header_id|>
""" ```
## Variables
`System` (string): system prompt
`Prompt` (string): user prompt
`Response` (string): assistant response
`Suffix` (string): text inserted after the assistant's response
`Messages` (list): list of messages
`Messages[].Role` (string): role which can be one of `system`, `user`, `assistant`, or `tool`
`Messages[].Content` (string): message content
`Messages[].ToolCalls` (list): list of tools the model wants to call
`Messages[].ToolCalls[].Function` (object): function to call
`Messages[].ToolCalls[].Function.Name` (string): function name
`Messages[].ToolCalls[].Function.Arguments` (map): mapping of argument name to argument value
`Tools` (list): list of tools the model can access
`Tools[].Type` (string): schema type. `type` is always `function`
`Tools[].Function` (object): function definition
`Tools[].Function.Name` (string): function name
`Tools[].Function.Description` (string): function description
`Tools[].Function.Parameters` (object): function parameters
`Tools[].Function.Parameters.Type` (string): schema type. `type` is always `object`
`Tools[].Function.Parameters.Required` (list): list of required properties
`Tools[].Function.Parameters.Properties` (map): mapping of property name to property definition
`Tools[].Function.Parameters.Properties[].Type` (string): property type
`Tools[].Function.Parameters.Properties[].Enum` (list): list of valid values
## Tips and Best Practices
Keep the following tips and best practices in mind when working with Go templates:
- **Be mindful of dot**: Control flow structures like `range` and `with` changes the value `.` - **Out-of-scope variables**: Use `$.` to reference variables not currently in scope, starting from the root - **Whitespace control**: Use `-` to trim leading (`{{-`) and trailing (`-}}`) whitespace
样例：原始模版： ``` {% if messages[0]['role'] == 'system' %} {% set loop_messages = messages[1:] %} {% set system_message = messages[0]['content'] %} <｜hy_begin▁of▁sentence｜>{{ system_message }}<｜hy_place▁holder▁no▁3｜> {% else %} {% set loop_messages = messages %} <｜hy_begin▁of▁sentence｜> {% endif %} {% for message in loop_messages %} {% if message['role'] == 'user' %} <｜hy_User｜>{{ message['content'] }} {% elif message['role'] == 'assistant' %} <｜hy_Assistant｜>{{ message['content'] }}<｜hy_place▁holder▁no▁2｜> {% endif %} {% endfor %} {% if add_generation_prompt %} <｜hy_Assistant｜> {% else %} <｜hy_place▁holder▁no▁8｜> {% endif %} ``` 上面的样例jinja模板，正确的转换结果是： ``` <｜hy_begin▁of▁sentence｜> {{- if .System }}{{ .System }}<｜hy_place▁holder▁no▁3｜>{{ end }}