Update README.md
Browse files
README.md
CHANGED
|
@@ -40,6 +40,11 @@ This version establishes new state-of-the-art (SOTA) results among open models u
|
|
| 40 |
|
| 41 |
## <span id="Inference">Quickstart</span>
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
For the chat scenario:
|
| 44 |
```
|
| 45 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 40 |
|
| 41 |
## <span id="Inference">Quickstart</span>
|
| 42 |
|
| 43 |
+
For inference hyperparameters, we recommend the following settings:
|
| 44 |
+
* Temperature: 0.6
|
| 45 |
+
* Top-p (nucleus sampling): 0.95
|
| 46 |
+
* Repeat penalty: 1.0
|
| 47 |
+
|
| 48 |
For the chat scenario:
|
| 49 |
```
|
| 50 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|