open-sci/open-sci-ref-v0.01-1.7b-pile-300B-4096
2B • Updated • 4
AI & ML interests
Researching and building foundation models with improved generalization and reasoning. LAION & friends spin-off for open-sourcing foundation models with strong generalization and reasoning , including datasets necessary for their creation, to serve as common open, reproducible grounds for further research experiments.
Recent Activity
View all activity
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-300B-8192
2B • Updated • 3
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-300B-4096
2B • Updated • 42
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-300B-16384
2B • Updated • 4
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-8192
2B • Updated • 1
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096-rope_theta-100k
2B • Updated • 2
open-sci/open-sci-ref-v0.01-1.7b-hplt-2.0-300B-4096
2B • Updated • 1
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-300B-4096
2B • Updated • 4
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-1T-4096
2B • Updated • 3
open-sci/open-sci-ref-v0.01-1.7b-dclm-300B-4096
2B • Updated • 3
open-sci/open-sci-ref-v0.01-1.7b-dclm-1T-4096
2B • Updated • 4
open-sci/open-sci-ref-v0.01-1.7b-commoncorpus-300B-4096
2B • Updated • 4 • 2
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup2000-lr0.006-2
1B • Updated • 2
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup2000
1B • Updated • 2
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup1500-lr0.005-2
1B • Updated • 3
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup1000
1B • Updated • 3
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup1000-lr0.004-4
1B • Updated • 1
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup1000-lr0.004-5
1B • Updated • 2
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup1000-lr0.004-6
1B • Updated • 4
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup1000-lr0.004-2
1B • Updated • 2
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup1000-lr0.004-3
1B • Updated • 1
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup1500
1B • Updated • 1
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup10000-lr0.004-3
1B • Updated • 1
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup10000-lr0.003-2
1B • Updated • 1
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-2048-warmup10000
1B • Updated • 1
open-sci/open-sci-ref-v0.01-1.3b-c4-50B-4096-warmup1000
Updated • 1
open-sci/open-sci-ref-v0.01-1.3b-c4-300B-2048-warmup30000-lr0.005-2
1B • Updated
open-sci/open-sci-ref-v0.01-1.3b-c4-300B-2048-warmup30000
1B • Updated • 5
open-sci/open-sci-ref-v0.01-1.3b-c4-300B-2048-warmup25000
1B • Updated • 1
open-sci/open-sci-ref-v0.01-1.3b-c4-300B-2048-warmup20000
1B • Updated • 1