compositional_test / transformers /utils /test_module /custom_tokenization.py
chendl's picture
add requirements
a1d409e
raw
history blame contribute delete
88 Bytes
from transformers import BertTokenizer
class CustomTokenizer(BertTokenizer):
pass