Update README.md
Browse files
README.md
CHANGED
|
@@ -53,10 +53,6 @@ The optimizer used is AdaFactor with inverse square root learning rate schedule
|
|
| 53 |
This model was then fine-tuned on a single TPU Pod V2-8 for 2000 steps in total, using sequence length 512 (batch size 256), using only the dataset only containing php code.
|
| 54 |
|
| 55 |
|
| 56 |
-
## Evaluation results
|
| 57 |
-
|
| 58 |
-
For the code documentation tasks, different models achieves the following results on different programming languages (in BLEU score):
|
| 59 |
-
|
| 60 |
Test results :
|
| 61 |
|
| 62 |
| Language / Model | Python | Java | Go | Php | Ruby | JavaScript |
|
|
@@ -69,9 +65,9 @@ Test results :
|
|
| 69 |
| MT-Small | 19.64 | 19.00 | 19.15 | 24.68 | 14.91 | 15.26 |
|
| 70 |
| MT-Base | **20.39** | **21.22** | 19.43 | **26.23** | **15.26** | 16.11 |
|
| 71 |
| MT-Large | XX | XX | XX | XX | XX | XX |
|
| 72 |
-
| MT-
|
| 73 |
-
|
|
| 74 |
-
| MT-
|
| 75 |
| State of the art | 19.06 | 17.65 | 18.07 | 25.16 | 12.16 | 14.90 |
|
| 76 |
|
| 77 |
|
|
|
|
| 53 |
This model was then fine-tuned on a single TPU Pod V2-8 for 2000 steps in total, using sequence length 512 (batch size 256), using only the dataset only containing php code.
|
| 54 |
|
| 55 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
Test results :
|
| 57 |
|
| 58 |
| Language / Model | Python | Java | Go | Php | Ruby | JavaScript |
|
|
|
|
| 65 |
| MT-Small | 19.64 | 19.00 | 19.15 | 24.68 | 14.91 | 15.26 |
|
| 66 |
| MT-Base | **20.39** | **21.22** | 19.43 | **26.23** | **15.26** | 16.11 |
|
| 67 |
| MT-Large | XX | XX | XX | XX | XX | XX |
|
| 68 |
+
| MT-FT-Small | 19.77 | 20.04 | 19.36 | 25.55 | 13.70 | 17.24 |
|
| 69 |
+
| **MT-FT-Base**| 19.77 | 21.12 | 18.86 | 25.79 | 14.24 | **18.62** |
|
| 70 |
+
| MT-FT-Large | XX | XX | XX | XX | XX | XX |
|
| 71 |
| State of the art | 19.06 | 17.65 | 18.07 | 25.16 | 12.16 | 14.90 |
|
| 72 |
|
| 73 |
|