TokSuite

community

AI & ML interests

Tokenization, Robustness, LLMs

Recent Activity

gsaltintas updated a Space about 12 hours ago

toksuite/leaderboard

gsaltintas published a Space 1 day ago

toksuite/leaderboard

gsaltintas updated a collection 5 days ago

Initializations

View all activity

Papers

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

View all Papers

toksuite 's models 22

toksuite/meta-llama-Llama-3.2-7B

Text Generation • 8B • Updated Apr 3 • 3

toksuite/meta-llama-Llama-3.2-300M

Text Generation • 0.6B • Updated Apr 3 • 8

toksuite/meta-llama-Llama-3.2-1B-seed_777_model_seed_222

Text Generation • 2B • Updated Apr 3 • 4

toksuite/meta-llama-Llama-3.2-1B-seed_777_model_seed_888

Text Generation • 2B • Updated Apr 3 • 2

toksuite/google-gemma-2-2b

Text Generation • 2B • Updated Dec 25, 2025 • 11

toksuite/meta-llama-Llama-3.2-1B

Text Generation • 2B • Updated Dec 25, 2025 • 8

toksuite/CohereLabs-aya-expanse-8b

Text Generation • 2B • Updated Dec 25, 2025 • 167

toksuite/tiktoken-gpt-4o

Text Generation • 2B • Updated Dec 25, 2025 • 11

toksuite/common-pile-comma-v0.1

Text Generation • 2B • Updated Dec 25, 2025 • 4

toksuite/microsoft-Phi-3-mini-4k-instruct

Text Generation • 1B • Updated Dec 25, 2025 • 6

toksuite/google-bert-bert-base-multilingual-cased

Text Generation • 2B • Updated Dec 25, 2025 • 11

toksuite/Qwen-Qwen3-8B

Text Generation • 2B • Updated Dec 25, 2025 • 41

toksuite/tokenmonster-englishcode-32000-consistent-v1

Text Generation • 1B • Updated Dec 25, 2025 • 4

toksuite/mistralai-tekken

Text Generation • 2B • Updated Dec 25, 2025 • 4

toksuite/gpt2

Text Generation • 1B • Updated Dec 25, 2025 • 172

toksuite/google-byt5-small

Text Generation • 1B • Updated Dec 25, 2025 • 144

toksuite/bigscience-bloom

Text Generation • 2B • Updated Dec 25, 2025 • 6

toksuite/facebook-xglm-564M

Text Generation • 2B • Updated Dec 25, 2025 • 155

toksuite/google-gemma-2-2b-textmatched

2B • Updated Dec 20, 2025 • 3

toksuite/common-pile-comma-v0.1-textmatched

2B • Updated Dec 20, 2025 • 1

toksuite/Qwen-Qwen3-8B-textmatched

2B • Updated Dec 20, 2025

toksuite/meta-llama-Llama-3.2-1B-textmatched

2B • Updated Dec 20, 2025 • 2