scripts/README.md

# Scripts

This folder contains helper scripts for working with the Redis-backed chat history used by the server.

## 1) `smoke_redis_history.py`

Purpose: quick smoke test that writes two messages to a session and verifies they are readable from a fresh `HistoryStore` instance.

Usage (from repo root):

```bash
# ensure redis is running (Homebrew example)
brew services start redis

# run the smoke script (make `server/` importable via PYTHONPATH)
PYTHONPATH=server venv/bin/python scripts/smoke_redis_history.py
```

You can pass a session id argument to use a different session key:

```bash
PYTHONPATH=server venv/bin/python scripts/smoke_redis_history.py my_session_id
```

Notes: the script sets `config.HISTORY_BACKEND = 'redis'` at runtime; it does not delete the session key after running.

## 2) `inspect_redis_history.py`

Purpose: inspect chat history keys in Redis, pretty-print messages, tail a session, or delete keys. Useful for debugging and manual inspection.

Requirements:
- A Python environment with the `redis` package installed (install with `pip install redis` into the project's `venv`).
- A running Redis server reachable at `redis://localhost:6379/0` (default). The script will attempt to read `llm_system.config.REDIS_URL` if `server/` is on `PYTHONPATH`.

Basic usage (show all `chat_history:*` keys):

```bash
PYTHONPATH=server venv/bin/python scripts/inspect_redis_history.py
```

Show a single session (pretty output):

```bash
PYTHONPATH=server venv/bin/python scripts/inspect_redis_history.py --session smoke_test_session_01
```

Useful flags:
- `--session, -s`: show one session (e.g. `smoke_test_session_01`) instead of listing all keys.
- `--pattern, -p`: key pattern to match (default: `chat_history:*`).
- `--redis-url, -r`: override the Redis URL for this run.
- `--format, -f`: output format, `pretty` (default) or `json` (JSON-lines).
- `--truncate`: truncate message content to this many characters (default: `200`; `0` = no truncation).
- `--limit`: show only the last N messages (default: `0` = all).
- `--tail, -t`: poll a single session and print updates (press Ctrl+C to stop).
- `--delete, -d`: delete the inspected key(s) after printing (use with caution).
- `--no-color`: disable ANSI color output.

Examples:

Pretty, truncated output:

```bash
PYTHONPATH=server venv/bin/python scripts/inspect_redis_history.py --session smoke_test_session_01 --format pretty --truncate 120 --limit 10
```

JSON-lines output (suitable for piping to files or jq):

```bash
PYTHONPATH=server venv/bin/python scripts/inspect_redis_history.py --session smoke_test_session_01 --format json --limit 50 > session.jsonl
```

Tail a session (poll every second):

```bash
PYTHONPATH=server venv/bin/python scripts/inspect_redis_history.py --session smoke_test_session_01 --tail
```

Delete after printing:

```bash
PYTHONPATH=server venv/bin/python scripts/inspect_redis_history.py --session smoke_test_session_01 --delete
```

Notes & safety:
- The script is intentionally lightweight and intended for local/dev use. `--delete` will remove Redis keys immediately and without recovery — use carefully.
- If the `llm_system` package is not importable from the current working directory, the script falls back to `redis://localhost:6379/0`.

## 3) `redis-memory.py`

Purpose: inspect Redis memory usage, set memory limits, and configure eviction policies to prevent unbounded memory growth.

Basic usage (show current memory info):

```bash
PYTHONPATH=server venv/bin/python scripts/redis-memory.py --info
```

Show recommendations for chat history setup:

```bash
PYTHONPATH=server venv/bin/python scripts/redis-memory.py --recommend
```

Useful commands:

Set memory limit to 512 MB:

```bash
PYTHONPATH=server venv/bin/python scripts/redis-memory.py --set-maxmemory 512mb
```

Set eviction policy to `volatile-lru` (recommended for chat history with TTL):

```bash
PYTHONPATH=server venv/bin/python scripts/redis-memory.py --set-policy volatile-lru
```

**Memory Management for Chat History:**

1. **Set session TTL** in `server/llm_system/config.py`:
   ```python
   REDIS_HISTORY_TTL_SECONDS = 2592000  # 30 days; 0 = no expiry
   ```
   Sessions will auto-expire after this duration, freeing memory.

2. **Set maxmemory limit** to prevent unbounded growth:
   ```bash
   PYTHONPATH=server venv/bin/python scripts/redis-memory.py --set-maxmemory 512mb
   ```

3. **Set eviction policy** to remove least-recently-used sessions:
   ```bash
   PYTHONPATH=server venv/bin/python scripts/redis-memory.py --set-policy volatile-lru
   ```
   This removes keys with TTL set, prioritizing LRU.

4. **Monitor memory usage** regularly:
   ```bash
   PYTHONPATH=server venv/bin/python scripts/redis-memory.py --info
   ```

**Why these settings?**
- **TTL**: Automatically expires old sessions, preventing memory bloat over time.
- **maxmemory**: Prevents Redis from consuming all system RAM and crashing.
- **volatile-lru**: Removes least-recently-used sessions (with TTL set) first, keeping active sessions in memory.

Example setup (30-day session expiry, 512 MB limit):

```bash
# Set config
cat >> server/llm_system/config.py << 'EOF'
REDIS_HISTORY_TTL_SECONDS = 2592000  # 30 days
EOF

# Set Redis limits
PYTHONPATH=server venv/bin/python scripts/redis-memory.py --set-maxmemory 512mb
PYTHONPATH=server venv/bin/python scripts/redis-memory.py --set-policy volatile-lru

# Verify
PYTHONPATH=server venv/bin/python scripts/redis-memory.py --info
```

## 4) `redis-commander.sh`

Purpose: start/stop/status the redis-commander web UI for visual inspection of Redis data.

Usage (from repo root):

```bash
# Start redis-commander (listening on port 8081)
./scripts/redis-commander.sh start

# Check status
./scripts/redis-commander.sh status

# Stop
./scripts/redis-commander.sh stop
```

Then open http://127.0.0.1:8081 in your browser to explore Redis keys and values visually.

Notes:
- Logs: `/tmp/redis-commander.log`
- PID file: `/tmp/redis-commander.pid`
- Requires `redis-commander` to be installed globally: `npm install -g redis-commander`

---

Want more?
- I can add `--output-file` to `inspect_redis_history.py` to write JSONL to a file, `--cleanup-all` (with confirmation) to remove all `chat_history:*` keys, or a `Makefile` target for convenience. Tell me which and I'll add it.