finished readme for stage1
Browse files
README.md
CHANGED
|
@@ -1,3 +1,8 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
|
| 5 |
+
| Stage | Batch size | Steps | HF path | Data mix | Comments |
|
| 6 |
+
+--------+-------------+-------+---------+-------------------------------+----------+
|
| 7 |
+
| stage1 | 262,144 tok | 37852 | [subfolder="stage1"](https://huggingface.co/danish-foundation-models/munin-7b-open-pt/tree/main/stage1)
|
| 8 |
+
| 2/3 [DynaWord](https://huggingface.co/datasets/danish-foundation-models/danish-dynaword/tree/9e230b35e31a510e5ab909112ad5bfc9463b2c23/), 1/3 [Common-Pile] (common-pile/comma_v0.1_training_dataset/5afc546db324e7f39f297ba757c9a60547151e7c/) | excludes depbank, jvj, nordjyllandnews, synne for DynaWord; uses subsets and weighting from [Comma-v0.1-2T](https://huggingface.co/common-pile/comma-v0.1-2t) cooldown phase for Common-Pile |
|