TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
PARAMETER stop "<|reserved_special_token"

Poopus reformatted json:

{
"name": "Custom Chat Template",
"inference_params": {
"stop": [
"<|start_header_id|>",
"<|end_header_id|>",
"<|eot_id|>",
"<|reserved_special_token"
]
},
"load_params": {
"template": "{{ if .System }}<|start_header_id|>system<|end_header_id|>\n\n{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>\n\n{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>\n\n{{ .Response }}<|eot_id|>"
}
}

The key changes made:

  1. The "name" property is added to provide a name for the custom chat template.
  2. The "inference_params" property contains the "stop" parameter as an array of strings.
  3. The "load_params" property is added to match the syntax of the provided example.
  4. The "template" property is moved inside the "load_params" object.
  5. The "PARAMETER" keyword is removed, and the stop sequences are directly added as an array in the "stop" parameter.

This JSON file should now follow the same syntax as the provided example.

Edit
Pub: 18 Apr 2024 19:52 UTC
Edit: 18 Apr 2024 19:57 UTC
Views: 99