Instructions to use kulia-moon/Text2GPT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kulia-moon/Text2GPT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kulia-moon/Text2GPT")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kulia-moon/Text2GPT")
model = AutoModelForCausalLM.from_pretrained("kulia-moon/Text2GPT")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use kulia-moon/Text2GPT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kulia-moon/Text2GPT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kulia-moon/Text2GPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/kulia-moon/Text2GPT

SGLang

How to use kulia-moon/Text2GPT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kulia-moon/Text2GPT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kulia-moon/Text2GPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kulia-moon/Text2GPT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kulia-moon/Text2GPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use kulia-moon/Text2GPT with Docker Model Runner:
```
docker model run hf.co/kulia-moon/Text2GPT
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Text2GPT 🤖

Text2GPT is a lightweight text generation model fine-tuned from DistilGPT2, with 81.9M parameters, designed for efficient and coherent text generation. It leverages the power of transformers and supports Safetensors for secure model loading. Ideal for creative writing, text completion, and more! 🚀

Features ✨

Generates human-like text with minimal input 📝
Supports Safetensors for safe and efficient loading 🔒
Fine-tuned for low-resource environments ⚡
Compatible with Hugging Face transformers and vLLM 🚀

Installation 🛠️

Install the required dependencies:

pip install transformers torch safetensors

Usage 🎮

Loading the Model with Transformers

Use the Hugging Face transformers library to load and generate text:

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model and tokenizer
model_name = "kulia-moon/Text2GPT"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Generate text
input_text = "Once upon a time"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Using Pipeline for Simplicity

For quick text generation:

from transformers import pipeline

pipe = pipeline("text-generation", model="kulia-moon/Text2GPT")
print(pipe("My name is Julien and I like to", max_length=30, do_sample=True)[0]["generated_text"])

vLLM Deployment for Scalability

Deploy with vLLM for high-throughput inference:

docker run --runtime nvidia --gpus all -v ~/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model kulia-moon/Text2GPT

Then, serve the model:

docker exec -it my_vllm_container bash -c "vllm serve kulia-moon/Text2GPT"

Widget Examples 🖱️

Try these prompts on the model page:

"Once upon a time" ⏳
"My name is Julien and I like to" 😊
"Paris is an amazing place to visit," 🗼
"I like traveling by train because" 🚂

Example Output:

Input: "Once upon a time"
Output: "Once upon a time, a curious AI roamed the digital realm, crafting tales of wonder."

Model Details 📊

Architecture: DistilGPT2-based, 6 layers, 81.9M parameters
Base Model: distilbert/distilgpt2
Safetensors: Supported, 81,912,576 parameters (non-sharded, non-quantized)
Intended Use: Text generation, creative writing, dialogue completion
Limitations: May produce biased or repetitive outputs; not optimized for sensitive tasks

Evaluation Report 📈

Evaluation metrics are under development. Preliminary tests suggest performance comparable to DistilGPT2 (perplexity ~21.1 on WikiText-103). Contributions for detailed metrics are welcome via discussions! 🙌