Hugging Face's logo

FasterDFlash
/
Hanrui

Model card Files
xet
 Community
Hanrui
File size: 48,460 Bytes
62dca4c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
nohup: ignoring input

*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
/workspace/hanrui/specforge/lib/python3.11/site-packages/tvm_ffi/_optional_torch_c_dlpack.py:174: UserWarning: Failed to JIT torch c dlpack extension, EnvTensorAllocator will not be enabled.
We recommend installing via `pip install torch-c-dlpack-ext`
  warnings.warn(
Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/junquan/SpecForge/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/tvm_ffi/_optional_torch_c_dlpack.py:174: UserWarning: Failed to JIT torch c dlpack extension, EnvTensorAllocator will not be enabled.
We recommend installing via `pip install torch-c-dlpack-ext`
  warnings.warn(
Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/junquan/SpecForge/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/tvm_ffi/_optional_torch_c_dlpack.py:174: UserWarning: Failed to JIT torch c dlpack extension, EnvTensorAllocator will not be enabled.
We recommend installing via `pip install torch-c-dlpack-ext`
  warnings.warn(
Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/junquan/SpecForge/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/tvm_ffi/_optional_torch_c_dlpack.py:174: UserWarning: Failed to JIT torch c dlpack extension, EnvTensorAllocator will not be enabled.
We recommend installing via `pip install torch-c-dlpack-ext`
  warnings.warn(
Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/junquan/SpecForge/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
  warnings.warn(
`torch_dtype` is deprecated! Use `dtype` instead!

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 138.75it/s]
/workspace/hanrui/specforge/lib/python3.11/site-packages/tvm_ffi/_optional_torch_c_dlpack.py:174: UserWarning: Failed to JIT torch c dlpack extension, EnvTensorAllocator will not be enabled.
We recommend installing via `pip install torch-c-dlpack-ext`
  warnings.warn(
Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/junquan/SpecForge/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
  warnings.warn(
`torch_dtype` is deprecated! Use `dtype` instead!

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 140.46it/s]
/workspace/hanrui/specforge/lib/python3.11/site-packages/tvm_ffi/_optional_torch_c_dlpack.py:174: UserWarning: Failed to JIT torch c dlpack extension, EnvTensorAllocator will not be enabled.
We recommend installing via `pip install torch-c-dlpack-ext`
  warnings.warn(
Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/junquan/SpecForge/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
  warnings.warn(
`torch_dtype` is deprecated! Use `dtype` instead!
/workspace/hanrui/specforge/lib/python3.11/site-packages/tvm_ffi/_optional_torch_c_dlpack.py:174: UserWarning: Failed to JIT torch c dlpack extension, EnvTensorAllocator will not be enabled.
We recommend installing via `pip install torch-c-dlpack-ext`
  warnings.warn(

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 138.05it/s]
Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/junquan/SpecForge/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
  warnings.warn(
`torch_dtype` is deprecated! Use `dtype` instead!
/workspace/hanrui/specforge/lib/python3.11/site-packages/tvm_ffi/_optional_torch_c_dlpack.py:174: UserWarning: Failed to JIT torch c dlpack extension, EnvTensorAllocator will not be enabled.
We recommend installing via `pip install torch-c-dlpack-ext`
  warnings.warn(
Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/junquan/SpecForge/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
  warnings.warn(

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 136.70it/s]
`torch_dtype` is deprecated! Use `dtype` instead!
`torch_dtype` is deprecated! Use `dtype` instead!

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 138.90it/s]

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 138.64it/s]
`torch_dtype` is deprecated! Use `dtype` instead!

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 140.02it/s]
`torch_dtype` is deprecated! Use `dtype` instead!

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 139.71it/s]
trainable params: 15,335,424 || all params: 8,206,070,784 || trainable%: 0.1869
trainable params: 15,335,424 || all params: 8,206,070,784 || trainable%: 0.1869
trainable params: 15,335,424 || all params: 8,206,070,784 || trainable%: 0.1869
trainable params: 15,335,424 || all params: 8,206,070,784 || trainable%: 0.1869
trainable params: 15,335,424 || all params: 8,206,070,784 || trainable%: 0.1869
trainable params: 15,335,424 || all params: 8,206,070,784 || trainable%: 0.1869/workspace/hanrui/specforge/lib/python3.11/site-packages/peft/tuners/tuners_utils.py:285: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(

trainable params: 15,335,424 || all params: 8,206,070,784 || trainable%: 0.1869
trainable params: 15,335,424 || all params: 8,206,070,784 || trainable%: 0.1869
/workspace/hanrui/specforge/lib/python3.11/site-packages/peft/tuners/tuners_utils.py:285: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/peft/tuners/tuners_utils.py:285: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/peft/tuners/tuners_utils.py:285: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/peft/tuners/tuners_utils.py:285: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/peft/tuners/tuners_utils.py:285: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/peft/tuners/tuners_utils.py:285: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(
/workspace/hanrui/specforge/lib/python3.11/site-packages/peft/tuners/tuners_utils.py:285: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(
dataset is cached at /workspace/hanrui/junquan/SpecForge/cache/processed_dataset/b2b7bdc9eb8a4170c0d33f03d2bf640b.pkl
/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py:4876: UserWarning: barrier(): using the device under current context. You can specify `device_id` in `init_process_group` to mute this warning.
  warnings.warn(  # warn only once
[rank0]:[W306 19:39:31.915495172 ProcessGroupNCCL.cpp:5072] Guessing device ID based on global rank. This can cause a hang if rank to GPU mapping is heterogeneous. You can specify device_id in init_process_group()
dataset is cached at /workspace/hanrui/junquan/SpecForge/cache/processed_dataset/b2b7bdc9eb8a4170c0d33f03d2bf640b.pkl
dataset is cached at /workspace/hanrui/junquan/SpecForge/cache/processed_dataset/b2b7bdc9eb8a4170c0d33f03d2bf640b.pkl
dataset is cached at /workspace/hanrui/junquan/SpecForge/cache/processed_dataset/b2b7bdc9eb8a4170c0d33f03d2bf640b.pkl
dataset is cached at /workspace/hanrui/junquan/SpecForge/cache/processed_dataset/b2b7bdc9eb8a4170c0d33f03d2bf640b.pkl
dataset is cached at /workspace/hanrui/junquan/SpecForge/cache/processed_dataset/b2b7bdc9eb8a4170c0d33f03d2bf640b.pkl
dataset is cached at /workspace/hanrui/junquan/SpecForge/cache/processed_dataset/b2b7bdc9eb8a4170c0d33f03d2bf640b.pkl
dataset is cached at /workspace/hanrui/junquan/SpecForge/cache/processed_dataset/b2b7bdc9eb8a4170c0d33f03d2bf640b.pkl

Filter:   0%|          | 0/818627 [00:00<?, ? examples/s]
Filter:   0%|          | 1000/818627 [00:00<01:43, 7929.03 examples/s]
Filter:   0%|          | 3000/818627 [00:00<01:11, 11395.09 examples/s]
Filter:   1%|          | 5000/818627 [00:00<01:08, 11908.47 examples/s]
Filter:   1%|          | 7000/818627 [00:00<01:09, 11671.45 examples/s]
Filter:   1%|          | 9000/818627 [00:00<01:07, 11907.42 examples/s]
Filter:   1%|▏         | 11000/818627 [00:00<01:06, 12124.05 examples/s]/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/nn/attention/flex_attention.py:1622: FutureWarning: return_lse is deprecated and will be removed in v2.10. Please use return_aux=AuxRequest(lse=True) instead.
  _warn_once(

Filter:   2%|▏         | 13000/818627 [00:01<01:04, 12423.94 examples/s]
Filter:   2%|▏         | 15000/818627 [00:01<01:03, 12682.25 examples/s]/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/nn/attention/flex_attention.py:1622: FutureWarning: return_lse is deprecated and will be removed in v2.10. Please use return_aux=AuxRequest(lse=True) instead.
  _warn_once(
/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/nn/attention/flex_attention.py:1622: FutureWarning: return_lse is deprecated and will be removed in v2.10. Please use return_aux=AuxRequest(lse=True) instead.
  _warn_once(
/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/nn/attention/flex_attention.py:1622: FutureWarning: return_lse is deprecated and will be removed in v2.10. Please use return_aux=AuxRequest(lse=True) instead.
  _warn_once(
/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/nn/attention/flex_attention.py:1622: FutureWarning: return_lse is deprecated and will be removed in v2.10. Please use return_aux=AuxRequest(lse=True) instead.
  _warn_once(

Filter:   2%|▏         | 17000/818627 [00:01<01:01, 13085.01 examples/s]/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/nn/attention/flex_attention.py:1622: FutureWarning: return_lse is deprecated and will be removed in v2.10. Please use return_aux=AuxRequest(lse=True) instead.
  _warn_once(
/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/nn/attention/flex_attention.py:1622: FutureWarning: return_lse is deprecated and will be removed in v2.10. Please use return_aux=AuxRequest(lse=True) instead.
  _warn_once(

Filter:   2%|▏         | 19000/818627 [00:01<01:00, 13264.91 examples/s]
Filter:   3%|▎         | 21000/818627 [00:01<01:00, 13188.96 examples/s]
Filter:   3%|▎         | 23000/818627 [00:01<01:00, 13236.51 examples/s]
Filter:   3%|▎         | 25000/818627 [00:02<01:04, 12383.93 examples/s]
Filter:   3%|▎         | 27000/818627 [00:02<01:17, 10174.32 examples/s]
Filter:   4%|▎         | 29000/818627 [00:02<01:23, 9456.58 examples/s] 
Filter:   4%|▍         | 31000/818627 [00:02<01:30, 8664.72 examples/s]
Filter:   4%|▍         | 32000/818627 [00:02<01:29, 8804.59 examples/s]
Filter:   4%|▍         | 33000/818627 [00:03<01:29, 8744.34 examples/s]
Filter:   4%|▍         | 34000/818627 [00:03<01:38, 7972.84 examples/s]
Filter:   4%|▍         | 35000/818627 [00:03<01:36, 8138.42 examples/s]
Filter:   4%|▍         | 36000/818627 [00:03<01:34, 8316.12 examples/s]
Filter:   5%|▍         | 37000/818627 [00:03<01:42, 7642.55 examples/s]
Filter:   5%|▍         | 38000/818627 [00:03<01:39, 7824.23 examples/s]
Filter:   5%|▍         | 39000/818627 [00:03<01:38, 7895.37 examples/s]
Filter:   5%|▍         | 40000/818627 [00:03<01:45, 7389.06 examples/s]
Filter:   5%|▌         | 41000/818627 [00:04<01:40, 7776.27 examples/s]
Filter:   5%|▌         | 42000/818627 [00:04<01:35, 8093.35 examples/s]
Filter:   5%|▌         | 43000/818627 [00:04<01:34, 8178.01 examples/s]
Filter:   5%|▌         | 44000/818627 [00:04<01:44, 7441.33 examples/s]
Filter:   5%|▌         | 45000/818627 [00:04<01:40, 7696.22 examples/s]
Filter:   6%|▌         | 46000/818627 [00:04<01:38, 7878.92 examples/s]
Filter:   6%|▌         | 47000/818627 [00:04<01:47, 7183.17 examples/s]
Filter:   6%|▌         | 48000/818627 [00:05<01:42, 7545.44 examples/s]
Filter:   6%|▌         | 49000/818627 [00:05<01:50, 6994.22 examples/s]
Filter:   6%|▌         | 50000/818627 [00:05<01:43, 7422.09 examples/s]
Filter:   6%|▌         | 51000/818627 [00:05<01:38, 7755.87 examples/s]
Filter:   6%|▋         | 52000/818627 [00:05<01:48, 7044.42 examples/s]
Filter:   6%|▋         | 53000/818627 [00:05<01:43, 7386.35 examples/s]
Filter:   7%|▋         | 54000/818627 [00:05<01:37, 7805.36 examples/s]
Filter:   7%|▋         | 55000/818627 [00:05<01:34, 8055.82 examples/s]
Filter:   7%|▋         | 56000/818627 [00:06<01:43, 7386.85 examples/s]
Filter:   7%|▋         | 57000/818627 [00:06<01:38, 7766.29 examples/s]
Filter:   7%|▋         | 58000/818627 [00:06<01:33, 8091.96 examples/s]
Filter:   7%|▋         | 60000/818627 [00:06<01:21, 9359.80 examples/s]
Filter:   8%|▊         | 62000/818627 [00:06<01:11, 10599.48 examples/s]
Filter:   8%|▊         | 64000/818627 [00:06<01:08, 11097.41 examples/s]
Filter:   8%|▊         | 66000/818627 [00:07<01:09, 10902.11 examples/s]
Filter:   8%|▊         | 68000/818627 [00:07<01:10, 10639.12 examples/s]
Filter:   9%|▊         | 70000/818627 [00:07<01:10, 10609.94 examples/s]
Filter:   9%|▉         | 72000/818627 [00:07<01:08, 10958.72 examples/s]
Filter:   9%|▉         | 74000/818627 [00:07<01:07, 10978.51 examples/s]
Filter:   9%|▉         | 76000/818627 [00:07<01:08, 10859.87 examples/s]
Filter:  10%|▉         | 78000/818627 [00:08<01:08, 10822.40 examples/s]
Filter:  10%|▉         | 80000/818627 [00:08<01:09, 10613.14 examples/s]
Filter:  10%|█         | 82000/818627 [00:08<01:09, 10595.09 examples/s]
Filter:  10%|█         | 84000/818627 [00:08<01:05, 11154.83 examples/s]
Filter:  11%|█         | 86000/818627 [00:08<01:12, 10145.30 examples/s]
Filter:  11%|█         | 88000/818627 [00:09<01:14, 9747.22 examples/s] 
Filter:  11%|█         | 89000/818627 [00:09<01:16, 9563.96 examples/s]
Filter:  11%|█         | 90000/818627 [00:09<01:22, 8795.54 examples/s]
Filter:  11%|█         | 91000/818627 [00:09<01:22, 8802.68 examples/s]
Filter:  11%|█         | 92000/818627 [00:09<01:21, 8885.65 examples/s]
Filter:  11%|█▏        | 93000/818627 [00:09<01:20, 8963.82 examples/s]
Filter:  11%|█▏        | 94000/818627 [00:09<01:20, 8951.39 examples/s]
Filter:  12%|█▏        | 96000/818627 [00:09<01:12, 10026.80 examples/s]
Filter:  12%|█▏        | 98000/818627 [00:10<01:03, 11298.05 examples/s]
Filter:  12%|█▏        | 100000/818627 [00:10<00:59, 12176.32 examples/s]
Filter:  12%|█▏        | 102000/818627 [00:10<00:55, 12814.59 examples/s]
Filter:  13%|█▎        | 104000/818627 [00:10<00:55, 12903.35 examples/s]
Filter:  13%|█▎        | 106000/818627 [00:10<00:54, 13069.92 examples/s]
Filter:  13%|█▎        | 108000/818627 [00:10<00:53, 13271.05 examples/s]
Filter:  13%|█▎        | 110000/818627 [00:11<00:52, 13406.88 examples/s]
Filter:  14%|█▎        | 112000/818627 [00:11<00:52, 13566.53 examples/s]
Filter:  14%|█▍        | 114000/818627 [00:11<00:51, 13752.05 examples/s]
Filter:  14%|█▍        | 116000/818627 [00:11<00:50, 13917.43 examples/s]
Filter:  14%|█▍        | 118000/818627 [00:11<00:49, 14023.13 examples/s]
Filter:  15%|█▍        | 120000/818627 [00:11<00:49, 14094.23 examples/s]
Filter:  15%|█▍        | 122000/818627 [00:11<00:49, 14040.25 examples/s]
Filter:  15%|█▌        | 124000/818627 [00:12<00:50, 13826.38 examples/s]
Filter:  15%|█▌        | 126000/818627 [00:12<00:50, 13601.69 examples/s]
Filter:  16%|█▌        | 128000/818627 [00:12<00:50, 13550.78 examples/s]
Filter:  16%|█▌        | 130000/818627 [00:12<00:52, 13156.00 examples/s]
Filter:  16%|█▌        | 132000/818627 [00:12<00:51, 13407.38 examples/s]
Filter:  16%|█▋        | 134000/818627 [00:13<01:22, 8295.09 examples/s] 
Filter:  17%|█▋        | 136000/818627 [00:13<01:13, 9229.81 examples/s]
Filter:  17%|█▋        | 138000/818627 [00:13<01:06, 10199.94 examples/s]
Filter:  17%|█▋        | 140000/818627 [00:13<01:01, 11059.28 examples/s]
Filter:  17%|█▋        | 142000/818627 [00:13<00:57, 11810.61 examples/s]
Filter:  18%|█▊        | 144000/818627 [00:13<00:54, 12405.32 examples/s]
Filter:  18%|█▊        | 146000/818627 [00:13<00:52, 12865.01 examples/s]
Filter:  18%|█▊        | 148000/818627 [00:14<00:51, 13148.11 examples/s]
Filter:  18%|█▊        | 150000/818627 [00:14<00:49, 13434.71 examples/s]
Filter:  19%|█▊        | 152000/818627 [00:14<00:48, 13676.73 examples/s]
Filter:  19%|█▉        | 154000/818627 [00:14<00:49, 13331.82 examples/s]
Filter:  19%|█▉        | 156000/818627 [00:14<00:49, 13516.02 examples/s]
Filter:  19%|█▉        | 158000/818627 [00:14<00:48, 13712.91 examples/s]
Filter:  20%|█▉        | 160000/818627 [00:14<00:47, 13835.07 examples/s]
Filter:  20%|█▉        | 162000/818627 [00:15<00:47, 13843.12 examples/s]
Filter:  20%|██        | 164000/818627 [00:15<00:47, 13825.37 examples/s]
Filter:  20%|██        | 166000/818627 [00:15<00:46, 13924.20 examples/s]
Filter:  21%|██        | 168000/818627 [00:15<00:46, 13993.04 examples/s]
Filter:  21%|██        | 170000/818627 [00:15<00:46, 14029.22 examples/s]
Filter:  21%|██        | 172000/818627 [00:15<00:47, 13675.97 examples/s]
Filter:  21%|██▏       | 174000/818627 [00:15<00:47, 13686.96 examples/s]
Filter:  21%|██▏       | 176000/818627 [00:16<00:46, 13796.40 examples/s]
Filter:  22%|██▏       | 178000/818627 [00:16<00:46, 13788.26 examples/s]
Filter:  22%|██▏       | 180000/818627 [00:16<00:46, 13740.92 examples/s]
Filter:  22%|██▏       | 182000/818627 [00:16<00:45, 13865.43 examples/s]
Filter:  22%|██▏       | 184000/818627 [00:16<00:45, 13965.15 examples/s]
Filter:  23%|██▎       | 186000/818627 [00:16<00:45, 14043.33 examples/s]
Filter:  23%|██▎       | 188000/818627 [00:16<00:45, 13993.04 examples/s]
Filter:  23%|██▎       | 190000/818627 [00:17<00:45, 13962.73 examples/s]
Filter:  23%|██▎       | 192000/818627 [00:17<00:44, 14068.34 examples/s]
Filter:  24%|██▎       | 194000/818627 [00:17<00:44, 13994.94 examples/s]
Filter:  24%|██▍       | 196000/818627 [00:17<00:44, 13971.52 examples/s]
Filter:  24%|██▍       | 198000/818627 [00:17<00:44, 13918.42 examples/s]
Filter:  24%|██▍       | 200000/818627 [00:17<00:44, 14009.15 examples/s]
Filter:  25%|██▍       | 202000/818627 [00:17<00:44, 14012.16 examples/s]
Filter:  25%|██▍       | 204000/818627 [00:18<00:43, 14134.16 examples/s]
Filter:  25%|██▌       | 206000/818627 [00:18<00:44, 13785.35 examples/s]
Filter:  25%|██▌       | 208000/818627 [00:18<00:44, 13876.19 examples/s]
Filter:  26%|██▌       | 210000/818627 [00:18<00:43, 13848.44 examples/s]
Filter:  26%|██▌       | 212000/818627 [00:18<00:43, 13913.30 examples/s]
Filter:  26%|██▌       | 214000/818627 [00:18<00:43, 14045.22 examples/s]
Filter:  26%|██▋       | 216000/818627 [00:18<00:42, 14132.00 examples/s]
Filter:  27%|██▋       | 218000/818627 [00:19<00:43, 13954.10 examples/s]
Filter:  27%|██▋       | 220000/818627 [00:19<00:42, 13963.74 examples/s]
Filter:  27%|██▋       | 222000/818627 [00:19<00:43, 13860.41 examples/s]
Filter:  27%|██▋       | 224000/818627 [00:19<00:42, 13942.05 examples/s]
Filter:  28%|██▊       | 226000/818627 [00:19<00:42, 13892.31 examples/s]
Filter:  28%|██▊       | 228000/818627 [00:19<00:42, 13818.41 examples/s]
Filter:  28%|██▊       | 230000/818627 [00:19<00:42, 13851.48 examples/s]
Filter:  28%|██▊       | 232000/818627 [00:20<00:41, 13997.87 examples/s]
Filter:  29%|██▊       | 234000/818627 [00:20<00:41, 14090.25 examples/s]
Filter:  29%|██▉       | 236000/818627 [00:20<00:41, 14124.31 examples/s]
Filter:  29%|██▉       | 238000/818627 [00:20<00:41, 14040.80 examples/s]
Filter:  29%|██▉       | 240000/818627 [00:20<00:41, 13891.67 examples/s]
Filter:  30%|██▉       | 242000/818627 [00:20<00:41, 13910.34 examples/s]
Filter:  30%|██▉       | 244000/818627 [00:20<00:41, 13973.74 examples/s]
Filter:  30%|███       | 246000/818627 [00:21<00:40, 14043.45 examples/s]
Filter:  30%|███       | 248000/818627 [00:21<00:40, 14088.04 examples/s]
Filter:  31%|███       | 250000/818627 [00:21<00:40, 13967.12 examples/s]
Filter:  31%|███       | 252000/818627 [00:21<00:40, 13902.33 examples/s]
Filter:  31%|███       | 254000/818627 [00:21<00:41, 13682.22 examples/s]
Filter:  31%|███▏      | 256000/818627 [00:21<00:42, 13393.12 examples/s]
Filter:  32%|███▏      | 258000/818627 [00:22<00:42, 13184.95 examples/s]
Filter:  32%|███▏      | 260000/818627 [00:22<00:42, 13225.77 examples/s]
Filter:  32%|███▏      | 262000/818627 [00:22<00:42, 13212.66 examples/s]
Filter:  32%|███▏      | 264000/818627 [00:22<00:41, 13394.06 examples/s]
Filter:  32%|███▏      | 266000/818627 [00:22<00:40, 13505.02 examples/s]
Filter:  33%|███▎      | 268000/818627 [00:22<00:40, 13666.81 examples/s]
Filter:  33%|███▎      | 270000/818627 [00:22<00:40, 13392.95 examples/s]
Filter:  33%|███▎      | 272000/818627 [00:23<00:40, 13547.05 examples/s]
Filter:  33%|███▎      | 274000/818627 [00:23<00:40, 13334.91 examples/s]
Filter:  34%|███▎      | 276000/818627 [00:23<00:40, 13278.36 examples/s]
Filter:  34%|███▍      | 278000/818627 [00:23<00:40, 13209.08 examples/s]
Filter:  34%|███▍      | 280000/818627 [00:23<00:40, 13261.24 examples/s]
Filter:  34%|███▍      | 282000/818627 [00:23<00:39, 13527.33 examples/s]
Filter:  35%|███▍      | 284000/818627 [00:23<00:38, 13718.09 examples/s]
Filter:  35%|███▍      | 286000/818627 [00:24<00:39, 13533.93 examples/s]
Filter:  35%|███▌      | 288000/818627 [00:24<00:39, 13315.90 examples/s]
Filter:  35%|███▌      | 290000/818627 [00:24<00:39, 13550.68 examples/s]
Filter:  36%|███▌      | 292000/818627 [00:24<00:39, 13318.50 examples/s]
Filter:  36%|███▌      | 294000/818627 [00:24<00:39, 13434.72 examples/s]
Filter:  36%|███▌      | 296000/818627 [00:24<00:39, 13348.90 examples/s]
Filter:  36%|███▋      | 298000/818627 [00:24<00:38, 13540.74 examples/s]
Filter:  37%|███▋      | 300000/818627 [00:25<00:37, 13715.10 examples/s]
Filter:  37%|███▋      | 302000/818627 [00:25<00:37, 13802.55 examples/s]
Filter:  37%|███▋      | 304000/818627 [00:25<00:37, 13893.94 examples/s]
Filter:  37%|███▋      | 306000/818627 [00:25<00:36, 13974.89 examples/s]
Filter:  38%|███▊      | 308000/818627 [00:25<00:37, 13642.94 examples/s]
Filter:  38%|███▊      | 310000/818627 [00:25<00:37, 13503.36 examples/s]
Filter:  38%|███▊      | 312000/818627 [00:26<00:37, 13529.64 examples/s]
Filter:  38%|███▊      | 314000/818627 [00:26<00:37, 13428.82 examples/s]
Filter:  39%|███▊      | 316000/818627 [00:26<00:37, 13339.00 examples/s]
Filter:  39%|███▉      | 318000/818627 [00:26<00:37, 13446.00 examples/s]
Filter:  39%|███▉      | 320000/818627 [00:26<00:37, 13323.50 examples/s]
Filter:  39%|███▉      | 322000/818627 [00:26<00:37, 13166.88 examples/s]
Filter:  40%|███▉      | 324000/818627 [00:26<00:36, 13443.31 examples/s]
Filter:  40%|███▉      | 326000/818627 [00:27<00:36, 13613.87 examples/s]
Filter:  40%|████      | 328000/818627 [00:27<00:36, 13361.13 examples/s]
Filter:  40%|████      | 330000/818627 [00:27<00:36, 13298.44 examples/s]
Filter:  41%|████      | 332000/818627 [00:27<00:36, 13239.35 examples/s]
Filter:  41%|████      | 334000/818627 [00:27<00:36, 13187.96 examples/s]
Filter:  41%|████      | 336000/818627 [00:27<00:35, 13408.76 examples/s]
Filter:  41%|████▏     | 338000/818627 [00:27<00:36, 13320.01 examples/s]
Filter:  42%|████▏     | 340000/818627 [00:28<00:36, 13075.55 examples/s]
Filter:  42%|████▏     | 342000/818627 [00:28<00:36, 13094.01 examples/s]
Filter:  42%|████▏     | 344000/818627 [00:28<00:35, 13332.33 examples/s]
Filter:  42%|████▏     | 346000/818627 [00:28<00:34, 13521.05 examples/s]
Filter:  43%|████▎     | 348000/818627 [00:28<00:34, 13666.47 examples/s]
Filter:  43%|████▎     | 350000/818627 [00:28<00:33, 13786.97 examples/s]
Filter:  43%|████▎     | 352000/818627 [00:28<00:33, 13791.48 examples/s]
Filter:  43%|████▎     | 354000/818627 [00:29<00:34, 13586.14 examples/s]
Filter:  43%|████▎     | 356000/818627 [00:29<00:55, 8333.33 examples/s] 
Filter:  44%|████▎     | 358000/818627 [00:29<00:49, 9219.25 examples/s]
Filter:  44%|████▍     | 360000/818627 [00:29<00:45, 9992.63 examples/s]
Filter:  44%|████▍     | 362000/818627 [00:30<00:42, 10848.37 examples/s]
Filter:  44%|████▍     | 364000/818627 [00:30<00:39, 11587.14 examples/s]
Filter:  45%|████▍     | 366000/818627 [00:30<00:37, 12185.04 examples/s]
Filter:  45%|████▍     | 368000/818627 [00:30<00:36, 12474.26 examples/s]
Filter:  45%|████▌     | 370000/818627 [00:30<00:35, 12660.60 examples/s]
Filter:  45%|████▌     | 372000/818627 [00:30<00:34, 12827.17 examples/s]
Filter:  46%|████▌     | 374000/818627 [00:30<00:34, 12935.99 examples/s]
Filter:  46%|████▌     | 376000/818627 [00:31<00:34, 12868.88 examples/s]
Filter:  46%|████▌     | 378000/818627 [00:31<00:33, 13120.88 examples/s]
Filter:  46%|████▋     | 380000/818627 [00:31<00:32, 13414.45 examples/s]
Filter:  47%|████▋     | 382000/818627 [00:31<00:32, 13329.68 examples/s]
Filter:  47%|████▋     | 384000/818627 [00:31<00:32, 13301.99 examples/s]
Filter:  47%|████▋     | 386000/818627 [00:31<00:32, 13403.41 examples/s]
Filter:  47%|████▋     | 388000/818627 [00:32<00:31, 13506.49 examples/s]
Filter:  48%|████▊     | 390000/818627 [00:32<00:32, 13305.39 examples/s]
Filter:  48%|████▊     | 392000/818627 [00:32<00:32, 13034.34 examples/s]
Filter:  48%|████▊     | 394000/818627 [00:32<00:32, 13253.54 examples/s]
Filter:  48%|████▊     | 396000/818627 [00:32<00:32, 13081.41 examples/s]
Filter:  49%|████▊     | 398000/818627 [00:32<00:32, 13069.13 examples/s]
Filter:  49%|████▉     | 400000/818627 [00:32<00:31, 13362.69 examples/s]
Filter:  49%|████▉     | 402000/818627 [00:33<00:31, 13163.35 examples/s]
Filter:  49%|████▉     | 404000/818627 [00:33<00:31, 13091.49 examples/s]
Filter:  50%|████▉     | 406000/818627 [00:33<00:30, 13326.98 examples/s]
Filter:  50%|████▉     | 408000/818627 [00:33<00:30, 13442.99 examples/s]
Filter:  50%|█████     | 410000/818627 [00:33<00:30, 13392.52 examples/s]
Filter:  50%|█████     | 412000/818627 [00:33<00:31, 13091.07 examples/s]
Filter:  51%|█████     | 414000/818627 [00:33<00:30, 13083.50 examples/s]
Filter:  51%|█████     | 416000/818627 [00:34<00:30, 13352.05 examples/s]
Filter:  51%|█████     | 418000/818627 [00:34<00:31, 12764.36 examples/s]
Filter:  51%|█████▏    | 420000/818627 [00:34<00:30, 13103.13 examples/s]
Filter:  52%|█████▏    | 422000/818627 [00:34<00:30, 13075.26 examples/s]
Filter:  52%|█████▏    | 424000/818627 [00:34<00:30, 13152.93 examples/s]
Filter:  52%|█████▏    | 426000/818627 [00:34<00:29, 13303.09 examples/s]
Filter:  52%|█████▏    | 428000/818627 [00:35<00:29, 13431.45 examples/s]
Filter:  53%|█████▎    | 430000/818627 [00:35<00:28, 13530.53 examples/s]
Filter:  53%|█████▎    | 432000/818627 [00:35<00:28, 13462.94 examples/s]
Filter:  53%|█████▎    | 434000/818627 [00:35<00:28, 13645.48 examples/s]
Filter:  53%|█████▎    | 436000/818627 [00:35<00:28, 13512.75 examples/s]
Filter:  54%|█████▎    | 438000/818627 [00:35<00:27, 13661.21 examples/s]
Filter:  54%|█████▎    | 440000/818627 [00:35<00:27, 13757.62 examples/s]
Filter:  54%|█████▍    | 442000/818627 [00:36<00:27, 13857.16 examples/s]
Filter:  54%|█████▍    | 444000/818627 [00:36<00:26, 13958.48 examples/s]
Filter:  54%|█████▍    | 446000/818627 [00:36<00:26, 14030.82 examples/s]
Filter:  55%|█████▍    | 448000/818627 [00:36<00:27, 13479.06 examples/s]
Filter:  55%|█████▍    | 450000/818627 [00:36<00:26, 13703.42 examples/s]
Filter:  55%|█████▌    | 452000/818627 [00:36<00:27, 13298.91 examples/s]
Filter:  55%|█████▌    | 454000/818627 [00:36<00:27, 13480.65 examples/s]
Filter:  56%|█████▌    | 456000/818627 [00:37<00:26, 13520.69 examples/s]
Filter:  56%|█████▌    | 458000/818627 [00:37<00:26, 13604.83 examples/s]
Filter:  56%|█████▌    | 460000/818627 [00:37<00:26, 13650.21 examples/s]
Filter:  56%|█████▋    | 462000/818627 [00:37<00:26, 13541.43 examples/s]
Filter:  57%|█████▋    | 464000/818627 [00:37<00:26, 13216.71 examples/s]
Filter:  57%|█████▋    | 466000/818627 [00:37<00:26, 13422.91 examples/s]
Filter:  57%|█████▋    | 468000/818627 [00:37<00:25, 13628.84 examples/s]
Filter:  57%|█████▋    | 470000/818627 [00:38<00:25, 13766.22 examples/s]
Filter:  58%|█████▊    | 472000/818627 [00:38<00:25, 13555.31 examples/s]
Filter:  58%|█████▊    | 474000/818627 [00:38<00:25, 13474.30 examples/s]
Filter:  58%|█████▊    | 476000/818627 [00:38<00:25, 13305.22 examples/s]
Filter:  58%|█████▊    | 478000/818627 [00:38<00:25, 13525.31 examples/s]
Filter:  59%|█████▊    | 480000/818627 [00:38<00:24, 13607.56 examples/s]
Filter:  59%|█████▉    | 482000/818627 [00:39<00:24, 13566.08 examples/s]
Filter:  59%|█████▉    | 484000/818627 [00:39<00:24, 13732.39 examples/s]
Filter:  59%|█████▉    | 486000/818627 [00:39<00:24, 13760.00 examples/s]
Filter:  60%|█████▉    | 488000/818627 [00:39<00:23, 13824.91 examples/s]
Filter:  60%|█████▉    | 490000/818627 [00:39<00:23, 13821.70 examples/s]
Filter:  60%|██████    | 492000/818627 [00:39<00:24, 13599.42 examples/s]
Filter:  60%|██████    | 494000/818627 [00:39<00:24, 13410.89 examples/s]
Filter:  61%|██████    | 496000/818627 [00:40<00:23, 13515.01 examples/s]
Filter:  61%|██████    | 498000/818627 [00:40<00:23, 13447.12 examples/s]
Filter:  61%|██████    | 500000/818627 [00:40<00:23, 13354.94 examples/s]
Filter:  61%|██████▏   | 502000/818627 [00:40<00:23, 13450.83 examples/s]
Filter:  62%|██████▏   | 504000/818627 [00:40<00:23, 13382.98 examples/s]
Filter:  62%|██████▏   | 506000/818627 [00:40<00:23, 13480.57 examples/s]
Filter:  62%|██████▏   | 508000/818627 [00:40<00:23, 13169.79 examples/s]
Filter:  62%|██████▏   | 510000/818627 [00:41<00:23, 13343.58 examples/s]
Filter:  63%|██████▎   | 512000/818627 [00:41<00:23, 13210.50 examples/s]
Filter:  63%|██████▎   | 514000/818627 [00:41<00:23, 13216.57 examples/s]
Filter:  63%|██████▎   | 516000/818627 [00:41<00:23, 13127.38 examples/s]
Filter:  63%|██████▎   | 518000/818627 [00:41<00:22, 13166.38 examples/s]
Filter:  64%|██████▎   | 520000/818627 [00:41<00:22, 13183.54 examples/s]
Filter:  64%|██████▍   | 522000/818627 [00:42<00:22, 13445.64 examples/s]
Filter:  64%|██████▍   | 524000/818627 [00:42<00:21, 13659.17 examples/s]
Filter:  64%|██████▍   | 526000/818627 [00:42<00:21, 13373.72 examples/s]
Filter:  64%|██████▍   | 528000/818627 [00:42<00:21, 13581.82 examples/s]
Filter:  65%|██████▍   | 530000/818627 [00:42<00:21, 13705.69 examples/s]
Filter:  65%|██████▍   | 532000/818627 [00:42<00:21, 13563.12 examples/s]
Filter:  65%|██████▌   | 534000/818627 [00:42<00:21, 13443.41 examples/s]
Filter:  65%|██████▌   | 536000/818627 [00:43<00:21, 13430.49 examples/s]
Filter:  66%|██████▌   | 538000/818627 [00:43<00:21, 13047.31 examples/s]
Filter:  66%|██████▌   | 540000/818627 [00:43<00:21, 12985.63 examples/s]
Filter:  66%|██████▌   | 542000/818627 [00:43<00:20, 13200.35 examples/s]
Filter:  66%|██████▋   | 544000/818627 [00:43<00:20, 13355.05 examples/s]
Filter:  67%|██████▋   | 546000/818627 [00:43<00:20, 13339.69 examples/s]
Filter:  67%|██████▋   | 548000/818627 [00:43<00:19, 13545.04 examples/s]
Filter:  67%|██████▋   | 550000/818627 [00:44<00:19, 13576.86 examples/s]
Filter:  67%|██████▋   | 552000/818627 [00:44<00:19, 13633.56 examples/s]
Filter:  68%|██████▊   | 554000/818627 [00:44<00:19, 13414.69 examples/s]
Filter:  68%|██████▊   | 556000/818627 [00:44<00:19, 13486.83 examples/s]
Filter:  68%|██████▊   | 558000/818627 [00:44<00:19, 13424.80 examples/s]
Filter:  68%|██████▊   | 560000/818627 [00:44<00:19, 13367.34 examples/s]
Filter:  69%|██████▊   | 562000/818627 [00:44<00:18, 13554.43 examples/s]
Filter:  69%|██████▉   | 564000/818627 [00:45<00:19, 13400.61 examples/s]
Filter:  69%|██████▉   | 566000/818627 [00:45<00:18, 13596.74 examples/s]
Filter:  69%|██████▉   | 568000/818627 [00:45<00:18, 13638.76 examples/s]
Filter:  70%|██████▉   | 570000/818627 [00:45<00:18, 13519.67 examples/s]
Filter:  70%|██████▉   | 572000/818627 [00:45<00:18, 13333.07 examples/s]
Filter:  70%|███████   | 574000/818627 [00:45<00:18, 13420.31 examples/s]
Filter:  70%|███████   | 576000/818627 [00:46<00:29, 8360.48 examples/s] 
Filter:  71%|███████   | 578000/818627 [00:46<00:26, 9248.77 examples/s]
Filter:  71%|███████   | 580000/818627 [00:46<00:23, 10285.73 examples/s]
Filter:  71%|███████   | 582000/818627 [00:46<00:21, 10934.22 examples/s]
Filter:  71%|███████▏  | 584000/818627 [00:46<00:20, 11712.71 examples/s]
Filter:  72%|███████▏  | 586000/818627 [00:47<00:19, 12230.28 examples/s]
Filter:  72%|███████▏  | 588000/818627 [00:47<00:18, 12683.53 examples/s]
Filter:  72%|███████▏  | 590000/818627 [00:47<00:17, 12914.55 examples/s]
Filter:  72%|███████▏  | 592000/818627 [00:47<00:17, 13146.56 examples/s]
Filter:  73%|███████▎  | 594000/818627 [00:47<00:16, 13430.79 examples/s]
Filter:  73%|███████▎  | 596000/818627 [00:47<00:16, 13367.39 examples/s]
Filter:  73%|███████▎  | 598000/818627 [00:47<00:16, 13564.99 examples/s]
Filter:  73%|███████▎  | 600000/818627 [00:48<00:15, 13700.29 examples/s]
Filter:  74%|███████▎  | 602000/818627 [00:48<00:15, 13815.87 examples/s]
Filter:  74%|███████▍  | 604000/818627 [00:48<00:15, 13429.57 examples/s]
Filter:  74%|███████▍  | 606000/818627 [00:48<00:15, 13603.62 examples/s]
Filter:  74%|███████▍  | 608000/818627 [00:48<00:15, 13744.25 examples/s]
Filter:  75%|███████▍  | 610000/818627 [00:48<00:15, 13553.89 examples/s]
Filter:  75%|███████▍  | 612000/818627 [00:48<00:15, 13707.32 examples/s]
Filter:  75%|███████▌  | 614000/818627 [00:49<00:15, 13276.70 examples/s]
Filter:  75%|███████▌  | 616000/818627 [00:49<00:15, 13498.51 examples/s]
Filter:  75%|███████▌  | 618000/818627 [00:49<00:14, 13514.31 examples/s]
Filter:  76%|███████▌  | 620000/818627 [00:49<00:14, 13660.60 examples/s]
Filter:  76%|███████▌  | 622000/818627 [00:49<00:14, 13690.41 examples/s]
Filter:  76%|███████▌  | 624000/818627 [00:49<00:14, 13835.95 examples/s]
Filter:  76%|███████▋  | 626000/818627 [00:50<00:14, 13280.09 examples/s]
Filter:  77%|███████▋  | 628000/818627 [00:50<00:14, 13510.71 examples/s]
Filter:  77%|███████▋  | 630000/818627 [00:50<00:14, 13251.65 examples/s]
Filter:  77%|███████▋  | 632000/818627 [00:50<00:14, 13135.35 examples/s]
Filter:  77%|███████▋  | 634000/818627 [00:50<00:13, 13412.91 examples/s]
Filter:  78%|███████▊  | 636000/818627 [00:50<00:13, 13619.69 examples/s]
Filter:  78%|███████▊  | 638000/818627 [00:50<00:13, 13458.01 examples/s]
Filter:  78%|███████▊  | 640000/818627 [00:51<00:13, 13420.21 examples/s]
Filter:  78%|███████▊  | 642000/818627 [00:51<00:12, 13644.93 examples/s]
Filter:  79%|███████▊  | 644000/818627 [00:51<00:13, 13311.68 examples/s]
Filter:  79%|███████▉  | 646000/818627 [00:51<00:12, 13374.86 examples/s]
Filter:  79%|███████▉  | 648000/818627 [00:51<00:12, 13580.88 examples/s]
Filter:  79%|███████▉  | 650000/818627 [00:51<00:12, 13330.76 examples/s]
Filter:  80%|███████▉  | 652000/818627 [00:51<00:12, 13298.47 examples/s]
Filter:  80%|███████▉  | 654000/818627 [00:52<00:12, 13213.39 examples/s]
Filter:  80%|████████  | 656000/818627 [00:52<00:12, 13217.17 examples/s]
Filter:  80%|████████  | 658000/818627 [00:52<00:11, 13492.97 examples/s]
Filter:  81%|████████  | 660000/818627 [00:52<00:11, 13319.74 examples/s]
Filter:  81%|████████  | 662000/818627 [00:52<00:11, 13174.92 examples/s]
Filter:  81%|████████  | 664000/818627 [00:52<00:11, 13432.74 examples/s]
Filter:  81%|████████▏ | 666000/818627 [00:53<00:11, 13316.26 examples/s]
Filter:  82%|████████▏ | 668000/818627 [00:53<00:11, 13300.99 examples/s]
Filter:  82%|████████▏ | 670000/818627 [00:53<00:11, 13260.44 examples/s]
Filter:  82%|████████▏ | 672000/818627 [00:53<00:11, 13285.87 examples/s]
Filter:  82%|████████▏ | 674000/818627 [00:53<00:10, 13245.08 examples/s]
Filter:  83%|████████▎ | 676000/818627 [00:53<00:10, 13488.31 examples/s]
Filter:  83%|████████▎ | 678000/818627 [00:53<00:10, 13537.32 examples/s]
Filter:  83%|████████▎ | 680000/818627 [00:54<00:10, 13575.13 examples/s]
Filter:  83%|████████▎ | 682000/818627 [00:54<00:10, 13371.46 examples/s]
Filter:  84%|████████▎ | 684000/818627 [00:54<00:09, 13561.07 examples/s]
Filter:  84%|████████▍ | 686000/818627 [00:54<00:09, 13677.83 examples/s]
Filter:  84%|████████▍ | 688000/818627 [00:54<00:09, 13765.73 examples/s]
Filter:  84%|████████▍ | 690000/818627 [00:54<00:09, 13691.07 examples/s]
Filter:  85%|████████▍ | 692000/818627 [00:54<00:09, 13726.99 examples/s]
Filter:  85%|████████▍ | 694000/818627 [00:55<00:09, 13749.72 examples/s]
Filter:  85%|████████▌ | 696000/818627 [00:55<00:08, 13767.01 examples/s]
Filter:  85%|████████▌ | 698000/818627 [00:55<00:08, 13896.16 examples/s]
Filter:  86%|████████▌ | 700000/818627 [00:55<00:08, 13839.96 examples/s]
Filter:  86%|████████▌ | 702000/818627 [00:55<00:08, 13787.39 examples/s]
Filter:  86%|████████▌ | 704000/818627 [00:55<00:08, 13839.65 examples/s]
Filter:  86%|████████▌ | 706000/818627 [00:55<00:08, 13900.36 examples/s]
Filter:  86%|████████▋ | 708000/818627 [00:56<00:07, 13954.41 examples/s]
Filter:  87%|████████▋ | 710000/818627 [00:56<00:07, 13736.15 examples/s]
Filter:  87%|████████▋ | 712000/818627 [00:56<00:07, 13402.83 examples/s]
Filter:  87%|████████▋ | 714000/818627 [00:56<00:07, 13624.00 examples/s]
Filter:  87%|████████▋ | 716000/818627 [00:56<00:07, 13482.43 examples/s]
Filter:  88%|████████▊ | 718000/818627 [00:56<00:07, 13347.95 examples/s]
Filter:  88%|████████▊ | 720000/818627 [00:56<00:07, 13092.35 examples/s]
Filter:  88%|████████▊ | 722000/818627 [00:57<00:07, 12942.08 examples/s]
Filter:  88%|████████▊ | 724000/818627 [00:57<00:07, 13020.36 examples/s]
Filter:  89%|████████▊ | 726000/818627 [00:57<00:06, 13334.84 examples/s]
Filter:  89%|████████▉ | 728000/818627 [00:57<00:06, 13457.75 examples/s]
Filter:  89%|████████▉ | 730000/818627 [00:57<00:06, 13363.35 examples/s]
Filter:  89%|████████▉ | 732000/818627 [00:57<00:06, 13538.65 examples/s]
Filter:  90%|████████▉ | 734000/818627 [00:58<00:06, 13573.01 examples/s]
Filter:  90%|████████▉ | 736000/818627 [00:58<00:06, 13689.24 examples/s]
Filter:  90%|█████████ | 738000/818627 [00:58<00:05, 13749.21 examples/s]
Filter:  90%|█████████ | 740000/818627 [00:58<00:05, 13419.49 examples/s]
Filter:  91%|█████████ | 742000/818627 [00:58<00:05, 13175.03 examples/s]
Filter:  91%|█████████ | 744000/818627 [00:58<00:05, 13324.74 examples/s]
Filter:  91%|█████████ | 746000/818627 [00:58<00:05, 13369.12 examples/s]
Filter:  91%|█████████▏| 748000/818627 [00:59<00:05, 13527.63 examples/s]
Filter:  92%|█████████▏| 750000/818627 [00:59<00:05, 13572.45 examples/s]
Filter:  92%|█████████▏| 752000/818627 [00:59<00:04, 13734.64 examples/s]
Filter:  92%|█████████▏| 754000/818627 [00:59<00:04, 13547.34 examples/s]
Filter:  92%|█████████▏| 756000/818627 [00:59<00:04, 13430.24 examples/s]
Filter:  93%|█████████▎| 758000/818627 [00:59<00:04, 13405.11 examples/s]
Filter:  93%|█████████▎| 760000/818627 [00:59<00:04, 13425.10 examples/s]
Filter:  93%|█████████▎| 762000/818627 [01:00<00:04, 13558.86 examples/s]
Filter:  93%|█████████▎| 764000/818627 [01:00<00:03, 13666.83 examples/s]
Filter:  94%|█████████▎| 766000/818627 [01:00<00:03, 13798.84 examples/s]
Filter:  94%|█████████▍| 768000/818627 [01:00<00:03, 13567.11 examples/s]
Filter:  94%|█████████▍| 770000/818627 [01:00<00:03, 13694.15 examples/s]
Filter:  94%|█████████▍| 772000/818627 [01:00<00:03, 13758.38 examples/s]
Filter:  95%|█████████▍| 774000/818627 [01:00<00:03, 13880.77 examples/s]
Filter:  95%|█████████▍| 776000/818627 [01:01<00:03, 13357.79 examples/s]
Filter:  95%|█████████▌| 778000/818627 [01:01<00:03, 13409.48 examples/s]
Filter:  95%|█████████▌| 780000/818627 [01:01<00:02, 13462.23 examples/s]
Filter:  96%|█████████▌| 782000/818627 [01:01<00:02, 13553.29 examples/s]
Filter:  96%|█████████▌| 784000/818627 [01:01<00:02, 13150.28 examples/s]
Filter:  96%|█████████▌| 786000/818627 [01:01<00:02, 13410.33 examples/s]
Filter:  96%|█████████▋| 788000/818627 [01:02<00:02, 13603.07 examples/s]
Filter:  97%|█████████▋| 790000/818627 [01:02<00:02, 13249.56 examples/s]
Filter:  97%|█████████▋| 792000/818627 [01:02<00:02, 13212.24 examples/s]
Filter:  97%|█████████▋| 794000/818627 [01:02<00:01, 13172.57 examples/s]
Filter:  97%|█████████▋| 796000/818627 [01:02<00:01, 13058.86 examples/s]
Filter:  97%|█████████▋| 798000/818627 [01:03<00:02, 8008.14 examples/s] 
Filter:  98%|█████████▊| 800000/818627 [01:03<00:02, 9087.15 examples/s]
Filter:  98%|█████████▊| 802000/818627 [01:03<00:01, 10121.96 examples/s]
Filter:  98%|█████████▊| 804000/818627 [01:03<00:01, 11055.26 examples/s]
Filter:  98%|█████████▊| 806000/818627 [01:03<00:01, 11831.60 examples/s]
Filter:  99%|█████████▊| 808000/818627 [01:03<00:00, 12305.50 examples/s]
Filter:  99%|█████████▉| 810000/818627 [01:03<00:00, 12675.10 examples/s]
Filter:  99%|█████████▉| 812000/818627 [01:04<00:00, 13077.19 examples/s]
Filter:  99%|█████████▉| 814000/818627 [01:04<00:00, 13369.73 examples/s]
Filter: 100%|█████████▉| 816000/818627 [01:04<00:00, 13623.88 examples/s]
Filter: 100%|█████████▉| 818000/818627 [01:04<00:00, 13786.52 examples/s]
Filter: 100%|██████████| 818627/818627 [01:04<00:00, 12669.00 examples/s]
/workspace/hanrui/specforge/lib/python3.11/site-packages/torch/nn/attention/flex_attention.py:1622: FutureWarning: return_lse is deprecated and will be removed in v2.10. Please use return_aux=AuxRequest(lse=True) instead.
  _warn_once(