kiro-1.0-7B-XCode

kiro-1.0-7B-XCode — a code-focused language model fine-tuned on top of Qwen2.5-Coder-7B, trained on a mixed dataset of real-world code and instruction pairs.

---

📖 Overview

kiro-1.0-7B-XCode is the first model in the kiro series by constructai.

This model is specialized for writing, analyzing, and explaining code in Python and JavaScript. It is trained to follow instructions in the ### Instruction → ### Response format, making it suitable for IDE plugins, coding assistants, and code review tools.

---

🏋️ Training

Parameter	Value
Base model	Qwen/Qwen2.5-Coder-7B
Method	QLoRA (4-bit, NF4) + LoRA merge
LoRA rank	16
LoRA alpha	32
Epochs	1
Learning rate	2e-4
Scheduler	Cosine
Hardware	NVIDIA RTX A5000 24GB

Dataset

The model was trained on ~58,000 samples from a mixed dataset:

Source	Samples	Description
bigcode/the-stack (Python)	20,000	Real-world Python code from GitHub
bigcode/the-stack (JavaScript)	20,000	Real-world JavaScript code from GitHub
iamtarun/python_code_instructions_18k_alpaca	18,000	Python instruction-response pairs

---

🚀 Quick Start

Installation

pip install transformers torch accelerate

Inference

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "constructai/kiro-1.0-7B-XCode"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

prompt = "### Instruction:\nWrite a Python function that checks if a number is prime.\n\n### Response:\n"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=False,
    repetition_penalty=1.3,
    pad_token_id=tokenizer.eos_token_id,
)
response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True
)
print(response)

Prompt Format

### Instruction:
{your request}

### Response:

With additional context:

### Instruction:
{your request}

### Input:
{additional context or code}

### Response:

---

📊 Example

Prompt:

### Instruction:
Write a Python function that checks if a number is prime.

### Response:

kiro-1.0 output:

def is_prime(num):
    for i in range(2, num):
        if (num % i) == 0:
            return False
    return True

---

⚠️ Limitations

• Trained for 1 epoch — may produce repetitions in long outputs (use repetition_penalty=1.3)
• Optimized for Python and JavaScript — other languages have limited support
• This is v1.0 — quality will improve in future releases

---

📜 License

This model is released under the Apache 2.0 license, inherited from the base model Qwen2.5-Coder-7B.

---

🙏 Acknowledgements

• Qwen Team for the excellent base model
• BigCode for The Stack dataset
• Hugging Face for the infrastructure

---

---

Made with ❤️ by constructai