README.md

BEX Engine v6 — WASM Plugin Engine

A production-grade WASM/Wasmtime Component Model plugin engine built in Rust. BEX enables sandboxed, deterministic plugin execution using WebAssembly components with WIT interface definitions. Designed for media streaming, metadata, and content provider applications with a Pure C ABI integration layer — no cxx dependency.

What's New in v6

This version incorporates all fixes from the QuickJS Integration Plan v3 review:

Critical Bug Fixes

• eval-js now accepts input parameter — user data is safely injected as a global variable instead of being concatenated into JS source code, eliminating code injection vulnerabilities
• call-js-fn now accepts fn-source parameter — functions are registered and auto-re-registered when source changes, eliminating the broken track_functions text parser
• TextEncoder/TextDecoder are now spec-correct UTF-8 — properly handles CJK characters, emoji, and all Unicode code points
• crypto.getRandomValues uses Rust-backed CSPRNG — rand::thread_rng() instead of Math.random()
• crypto.subtle is now fully implemented — SHA-1/256/384/512, AES-CBC encrypt/decrypt, HMAC-SHA256/512, PBKDF2, all in pure JS with immediate-resolved Promises
• args_json is passed as a string, not eval'd — eliminates JS injection attack vector
• Pool dispatch is non-blocking — uses try_send instead of blocking send, returns PoolBusy instead of hanging Wasmtime threads
• Pool shutdown uses SeqCst ordering — fixes race condition on ARM/Apple Silicon

Missing Features Now Implemented

• console.log routes to Rust tracing — no longer silently dropped
• setTimeout/setInterval call callbacks synchronously — no longer silently skipped
• clear-js-fn WIT function — allows unregistering JS functions when cipher rotates
• JsPoolConfig wired into EngineConfig — JS pool settings are configurable from engine config

Design Improvements

• Idle context eviction throttled to 30-second intervals — reduces overhead from checking on every loop iteration
• apply_fn helper removed — func.call((args_json,)) used directly
• max_stack_bytes configurable via JsPoolConfig — default 512KB
• All compiler warnings fixed — unused imports, dead code, unused variables
• atob/btoa are Rust-backed — correct Latin-1 handling via base64 crate
• Additional polyfills — URL, URLSearchParams, performance.now(), structuredClone, navigator, location, queueMicrotask

Architecture

┌──────────────────────────────────────────────────────────────┐
│                    C++ Application                           │
│                  (via Pure C ABI)                             │
│  ┌──────────────────────────────────────────────────────┐   │
│  │  BexResultCallback (function pointer)                │   │
│  │  bex_submit_*() → request_id                         │   │
│  │  callback(user_data, req_id, success, payload, len)  │   │
│  │  bex_cancel_request(request_id)                       │   │
│  └──────────────────────┬───────────────────────────────┘   │
├─────────────────────────┼───────────────────────────────────┤
│                BexRuntime (async callback-driven)             │
│  ┌─────────────┐  ┌──────────────┐  ┌───────────────────┐  │
│  │  Scheduler   │  │  Cancellation│  │  Tokio Runtime    │  │
│  │  (3 lanes)   │  │  Tokens      │  │  (async tasks)    │  │
│  └──────┬──────┘  └──────┬───────┘  └─────────┬─────────┘  │
├─────────┼────────────────┼─────────────────────┼────────────┤
│         │     BEX Engine (Wasmtime)            │            │
│  ┌──────▼──────┐  ┌────▼─────┐  ┌─────────────▼──────────┐ │
│  │  Wasmtime    │  │  Host    │  │  Plugin Registry       │ │
│  │  Component   │  │  APIs    │  │  (circuit breaker)     │ │
│  │  Model       │  │          │  │                        │ │
│  └──────┬──────┘  └────┬─────┘  └────────────────────────┘ │
│         │              │                                     │
│  ┌──────▼──────┐  ┌────▼─────┐  ┌─────────────┐            │
│  │  WASM        │  │  HTTP    │  │  Redb DB     │            │
│  │  Components  │  │  (reqwest)│  │  (storage)   │            │
│  └──────────────┘  └──────────┘  └─────────────┘            │
│                                                               │
│  FlatBuffers (wire format)  │  JSON (CLI/debug)              │
└──────────────────────────────┴────────────────────────────────┘

Core Design Principles

1. WASM-Only: All plugins run as WebAssembly components via Wasmtime — no native plugins, no dual-mode engine
2. Component Model: Uses Wasmtime's Component Model with WIT interface definitions for type-safe host-guest communication
3. Callback-Driven: C++ backend submits requests via bex_submit_*(), receives results via a C function pointer callback (BexResultCallback) invoked from a background Tokio thread. No polling, no event queue, no drain pattern.
4. Self-Describing IDs: The engine treats IDs as opaque strings — it does not know or care what they mean. Each plugin defines its own ID format and parses IDs internally. For example, get_servers takes a single id parameter; the plugin parses slug$ep=1$sub=1$dub=0 because it knows its own encoding scheme. There is no episode_id parameter anywhere in the engine.
5. Sandboxed Execution: Fuel-based metering, stack limits, epoch-based timeouts, and capability-based host APIs
6. Lane-Based Scheduling: Three concurrency lanes — Control (1), User (4), Background (2) — with semaphores, plus global WASM (4) and HTTP (8) permit limits
7. Cancellation: Each request gets a CancellationToken; cancel via bex_cancel_request()
8. Pure C ABI: The Rust engine exports extern "C" functions matching bex_engine.h. No cxx, no bridge codegen, no special build steps. Link the Rust static/shared library natively.

Workspace Structure

bex-engine/
├── crates/
│   ├── bex-types/              # Shared types (Manifest, Capabilities, BexError, PluginInfo, etc.)
│   ├── bex-pkg/                # BEX package format (pack/unpack/verify with zstd+CRC32+SHA256)
│   ├── bex-db/                 # Redb-backed storage (KV, secrets, plugins, WASM blobs)
│   ├── bex-wire/               # FlatBuffer wire-format types + builders
│   ├── bex-core/               # Core engine (Wasmtime, host APIs, linker, compile cache)
│   ├── bex-runtime/            # Async runtime (scheduler, cancellation, Pure C FFI via ffi.rs)
│   └── bex-cli/                # Rust CLI tool (clap-based)
├── plugins/
│   ├── bex-gogoanime/          # GogoAnime/Anitaku streaming plugin
│   ├── bex-kaianime/           # KaiAnime streaming plugin
│   ├── bex-hianime/            # HiAnime streaming plugin
│   ├── bex-imdb/               # IMDb metadata plugin
│   └── bex-kisskh/             # KissKH streaming plugin
├── cpp-cli/
│   ├── bex_engine.h            # Pure C ABI header (the FFI boundary)
│   ├── bexcli.cpp              # C++ CLI tool (uses promise/future + C callbacks)
│   ├── CMakeLists.txt          # CMake build configuration (links libbex_runtime natively)
│   └── wire_gen/               # Generated FlatBuffer C++ headers
├── wit/
│   └── plugin.wit              # Master WIT interface definitions
├── dist/                       # Built WASM components and manifests
├── build-plugins.sh            # Build, convert, and pack all plugins
└── Cargo.toml                  # Workspace root

Pure C ABI

The Rust engine exposes a Pure C ABI through bex_engine.h. No cxx, no code generation, no bridge crate. The Rust library compiles as both cdylib and staticlib, and CMake links it natively.

How It Works

1. C++ calls bex_submit_search(engine, plugin_id, query, callback, user_data)
2. Rust spawns a Tokio task that does the work
3. On completion, Rust invokes callback(user_data, request_id, success, payload, len) from the Tokio background thread
4. C++ receives the result in the callback and can parse/copy the payload before the callback returns

C++ Integration Example

#include "bex_engine.h"

// 1. Define a callback handler
extern "C" void on_result(void* user_data, uint64_t req_id,
                          bool success, const uint8_t* payload, size_t len) {
    auto* promise = static_cast<std::promise<std::string>*>(user_data);
    if (success) {
        promise->set_value(std::string(reinterpret_cast<const char*>(payload), len));
    } else {
        promise->set_exception(std::make_exception_ptr(
            std::runtime_error(std::string(reinterpret_cast<const char*>(payload), len))));
    }
}

// 2. Create engine
BexEngine* engine = bex_engine_new("/path/to/data");

// 3. Submit async requests — returns request_id immediately
std::promise<std::string> promise;
uint64_t req1 = bex_submit_home(engine, "bex.gogoanime", on_result, &promise);
uint64_t req2 = bex_submit_search(engine, "bex.gogoanime", "one piece", on_result, &promise);
uint64_t req3 = bex_submit_info(engine, "bex.gogoanime", "one-piece", on_result, &promise);
uint64_t req4 = bex_submit_servers(engine, "bex.gogoanime", "one-piece$ep=1$sub=1$dub=0",
                                    on_result, &promise);

// 4. Wait for results (or use the callback in your event loop)
std::string result = promise.get_future().get();

// 5. Cancel a request
bex_cancel_request(engine, req2);

// 6. Plugin management (synchronous)
bex_engine_install(engine, "/path/to/plugin.bex");
bex_engine_uninstall(engine, "bex.gogoanime");
BexPluginInfoList plugins = bex_engine_list_plugins(engine);
bex_plugin_info_list_free(plugins);

// 7. API key management (synchronous)
bex_engine_secret_set(engine, "bex.imdb", "api-key", "your-key");

// 8. Shutdown
bex_engine_free(engine);

Full C ABI Function Reference

Lifecycle

Function	Description
bex_engine_new(data_dir)	Create engine → BexEngine*
bex_engine_free(engine)	Graceful shutdown and free

Plugin Management (synchronous)

Function	Description
bex_engine_install(engine, path)	Install .bex plugin package → int
bex_engine_uninstall(engine, id)	Uninstall plugin by ID → int
bex_engine_list_plugins(engine)	List installed plugins → BexPluginInfoList
bex_engine_plugin_info(engine, id, out)	Get detailed plugin info → int
bex_engine_enable(engine, id)	Enable plugin → int
bex_engine_disable(engine, id)	Disable plugin → int
bex_plugin_info_list_free(list)	Free plugin list
bex_plugin_info_free(info)	Free plugin info struct

Secret / API Key Management (synchronous)

Function	Description
bex_engine_secret_set(engine, plugin_id, key, value)	Store a secret → int
bex_engine_secret_get(engine, plugin_id, key, out_buf, out_buf_len)	Retrieve a secret value → int
bex_engine_secret_delete(engine, plugin_id, key)	Delete a secret → int
bex_engine_secret_keys(engine, plugin_id)	List key names (comma-separated) → char*
bex_string_free(s)	Free a string returned by the engine

Async Operations (callback-driven)

Function	Description
bex_submit_home(engine, plugin_id, callback, user_data)	Submit home request → uint64_t request_id
bex_submit_search(engine, plugin_id, query, callback, user_data)	Submit search request → uint64_t request_id
bex_submit_info(engine, plugin_id, media_id, callback, user_data)	Submit info request → uint64_t request_id
bex_submit_servers(engine, plugin_id, id, callback, user_data)	Submit servers request → uint64_t request_id
bex_submit_stream(engine, plugin_id, server_json, callback, user_data)	Submit stream resolve → uint64_t request_id

Cancellation & Stats

Function	Description
bex_cancel_request(engine, request_id)	Cancel a pending request → bool
bex_engine_stats(engine)	Get engine stats (JSON) → char*
bex_engine_last_error(engine)	Get last error message → char*

Quick Start

Prerequisites

• Rust toolchain (stable)
• wasm-tools CLI: cargo install wasm-tools-cli
• C++17 compiler (for C++ CLI / integration)
• CMake 3.16+ (for C++ CLI / integration)

Build the Engine

# Build the Rust engine and CLI
cargo build --release

# Build and pack all plugins (compile → component convert → pack)
bash build-plugins.sh

# Build the C++ CLI with CMake
cd cpp-cli && mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

Build a Single Plugin (Manual)

Plugins are built in three steps: compile to WASM, convert to a component, then pack.

# Step 1: Compile to wasm32-wasip1
cargo build -p bex-gogoanime --target wasm32-wasip1 --release

# Step 2: Convert to a WASM Component (requires wasm-tools + WASI adapter)
wasm-tools component new \
    target/wasm32-wasip1/release/bex_gogoanime.wasm \
    -o target/components/bex-gogoanime.component.wasm \
    --adapt ~/.cargo/registry/src/index.crates.io-*/wasi-preview1-component-adapter-provider-*/artefacts/wasi_snapshot_preview1.reactor.wasm

# Step 3: Pack into a .bex package
cargo run -p bex-cli --release -- pack \
    dist/bex-gogoanime.yaml \
    target/components/bex-gogoanime.component.wasm \
    dist/bex-gogoanime.bex

Install and Use

# Install a plugin
cargo run -p bex-cli --release -- install dist/bex-gogoanime.bex

# List installed plugins
cargo run -p bex-cli --release -- list

# Get detailed plugin info
cargo run -p bex-cli --release -- plugin-info bex.gogoanime

Plugin Management

Rust CLI

# Install / uninstall
bex install dist/bex-gogoanime.bex
bex uninstall bex.gogoanime

# List installed plugins
bex list

# Show detailed plugin info (capabilities, enabled state, etc.)
bex plugin-info bex.gogoanime

# Enable / disable
bex enable bex.gogoanime
bex disable bex.gogoanime

# Inspect a .bex package without installing
bex inspect dist/bex-gogoanime.bex

# Engine stats
bex stats

C++ CLI

# Install / uninstall
./bexcli install dist/bex-gogoanime.bex
./bexcli uninstall bex.gogoanime

# List plugins (with capabilities column)
./bexcli list

# Detailed plugin info (includes API keys list)
./bexcli info-plugin bex.gogoanime

# Enable / disable
./bexcli enable bex.gogoanime
./bexcli disable bex.gogoanime

# Engine stats
./bexcli stats

API Key / Secret Management

The engine provides per-plugin secret storage backed by Redb. Secrets are scoped to a plugin ID and are accessible to the plugin at runtime via the secrets WIT interface. This is the mechanism for storing API keys, tokens, and other credentials that plugins need.

Rust CLI

# Set an API key
bex set-key bex.imdb api-key "your-api-key-here"

# Get an API key value
bex get-key bex.imdb api-key

# Delete an API key
bex delete-key bex.imdb api-key

# List all keys for a plugin
bex list-keys bex.imdb

C++ CLI

# Set an API key
./bexcli set-key bex.imdb api-key "your-api-key-here"

# Get an API key value
./bexcli get-key bex.imdb api-key

# Delete an API key
./bexcli delete-key bex.imdb api-key

# List all keys for a plugin
./bexcli list-keys bex.imdb

C API

// Store a secret
bex_engine_secret_set(engine, "bex.imdb", "api-key", "your-key");

// Retrieve a secret
char buf[4096];
size_t buf_len = sizeof(buf);
bex_engine_secret_get(engine, "bex.imdb", "api-key", buf, &buf_len);

// Delete a secret
bex_engine_secret_delete(engine, "bex.imdb", "api-key");

// List all secret key names (comma-separated, caller frees with bex_string_free)
char* keys = bex_engine_secret_keys(engine, "bex.imdb");
bex_string_free(keys);

Plugin Access (WIT)

Inside a plugin, secrets are accessed read-only through the secrets host interface:

use bindings::bex::plugin::secrets;

fn get_api_key() -> Option<String> {
    secrets::get("api-key")
}

Secrets that a plugin expects are declared in the manifest:

secrets:
  - api-key
  - tmdb-token

Self-Describing IDs

Self-describing IDs are the core design pattern for how the BEX engine handles typed identifiers. The engine itself treats all IDs as opaque strings — it does not parse, validate, or interpret them. Only the plugin knows what its IDs mean and how to decode them.

This means:

• get_servers takes a single id parameter — there is no separate episode_id parameter
• The engine never parses IDs — it passes them straight through to the plugin
• Each plugin defines its own ID encoding — different plugins can use entirely different schemes
• IDs are portable — they can be stored, serialized, and passed between systems without the engine needing to understand them

Example: GogoAnime Episode IDs

The GogoAnime plugin encodes episode context directly in the ID:

{slug}$ep={episode_number}$sub={0|1}$dub={0|1}

ID	Meaning
one-piece$ep=1$sub=1$dub=0	One Piece episode 1, subbed
jujutsu-kaisen-tv$ep=24$sub=0$dub=1	Jujutsu Kaisen episode 24, dubbed

When get_servers is called with this ID, the GogoAnime plugin splits on $ and parses each key-value pair to determine the slug, episode number, and sub/dub flags. The engine never does this parsing — it just passes the string through.

Example: IMDb Media IDs

The IMDb plugin might use a different scheme entirely (e.g., tt1234567), and the engine works equally well because it does not interpret the ID.

Usage

# The ID is self-describing — pass it directly
bex servers bex.gogoanime 'one-piece$ep=1$sub=1$dub=0'

# C++ CLI works the same way
./bexcli servers bex.gogoanime 'one-piece$ep=1$sub=1$dub=0'

WIT Interface Definitions

Host-Provided APIs (imports — plugins call these)

Interface	Functions	Description
http	send-request	HTTP client with caching, redirect control, size limits
kv	set, get, remove, keys	Scoped key-value storage
secrets	get	Read-only secret/API key access
log	write	Structured logging through host
clock	now-ms, monotonic	Time access
rng	bytes	Secure random bytes
js	eval-js, eval-js-opts, call-js-fn, clear-js-fn	QuickJS sandbox — safe JS eval, function call, and cleanup

Plugin-Provided APIs (exports — host calls these)

Function	Description
get-home	Get home page sections
get-category	Browse by category with pagination
search	Search media content
get-info	Get detailed media info with episodes
get-servers	Get streaming servers for an episode
resolve-stream	Resolve stream source from server
search-subtitles	Search for subtitles
download-subtitle	Download subtitle file
get-articles	Get article sections
search-articles	Search articles

Writing a Plugin

1. Create a plugin project

cargo init --lib my-plugin
cd my-plugin

Add to Cargo.toml:

[package]
name = "my-plugin"
version = "0.1.0"
edition = "2021"

[dependencies]
wit-bindgen = { version = "0.57.1", features = ["bitflags"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "bex:my-plugin"

[package.metadata.component.dependencies]

2. Copy the WIT definitions

Copy wit/plugin.wit from the engine repository into your plugin's wit/ directory.

3. Generate bindings

Run cargo build --target wasm32-wasip1 --release once to generate src/bindings.rs, then implement the Guest trait:

#[allow(warnings)]
mod bindings;

use bindings::bex::plugin::common::*;
use bindings::bex::plugin::http;
use bindings::exports::api::Guest;

struct Component;

impl Guest for Component {
    fn get_home(_ctx: RequestContext) -> Result<Vec<HomeSection>, PluginError> {
        Ok(vec![HomeSection {
            id: "home".to_string(),
            title: "My Plugin".to_string(),
            subtitle: None,
            items: vec![],
            next_page: None,
            layout: CardLayout::Grid,
            show_rank: false,
            categories: vec![],
            extra: vec![],
        }])
    }

    fn search(_ctx: RequestContext, query: String, _filters: SearchFilters) -> Result<PagedResult, PluginError> {
        let response = http::send_request(&http::Request {
            method: http::Method::Get,
            url: format!("https://api.example.com/search?q={}", query),
            headers: vec![],
            body: None,
            timeout_ms: Some(10000),
            follow_redirects: true,
            cache_mode: http::CacheMode::Normal,
            max_bytes: Some(1024 * 1024),
        }).map_err(|e| PluginError::Network(format!("{:?}", e)))?;

        Ok(PagedResult { items: vec![], categories: vec![], next_page: None })
    }

    fn get_servers(_ctx: RequestContext, id: String) -> Result<Vec<Server>, PluginError> {
        // The ID is self-describing — parse it however your plugin needs
        // The engine does not interpret the ID, only your plugin does
        let parts: Vec<&str> = id.split('$').collect();
        // ... parse and fetch servers ...
        Ok(vec![])
    }

    // ... implement other methods ...
}

bindings::export!(Component with_types_in bindings);

4. Build, convert, and pack

# Step 1: Compile to WASM
cargo build --target wasm32-wasip1 --release

# Step 2: Convert to a WASM Component
wasm-tools component new \
    target/wasm32-wasip1/release/my_plugin.wasm \
    -o target/components/my-plugin.component.wasm \
    --adapt /path/to/wasi_snapshot_preview1.reactor.wasm

# Step 3: Pack into a .bex package
bex pack manifest.yaml target/components/my-plugin.component.wasm my-plugin.bex

# Step 4: Install
bex install my-plugin.bex

Plugin Manifest

schema: 1
id: bex.my-plugin
name: My Plugin
version: 1.0.0
authors:
  - Your Name
abi: ">=1.0.0,<2.0.0"
provides:
  home: true
  search: true
  info: true
  servers: true
  stream: true
network:
  hosts:
    - "api.example.com"
  concurrent: 4
storage: true
secrets:
  - api-key
display:
  description: My awesome plugin
  tags:
    - streaming
  priority: 100

Capability Bits

Bit	Name	Methods
0	HOME	get_home
1	CATEGORY	get_category
2	SEARCH	search
3	INFO	get_info
4	SERVERS	get_servers
5	STREAM	resolve_stream
6	SUBTITLES	search_subtitles, download_subtitle
7	ARTICLES	get_articles, search_articles

CMake Integration

The C++ CLI's CMakeLists.txt is designed to be self-contained and reusable. You can integrate the BEX engine into any C++ project with minimal setup. The only requirement is the bex_engine.h header and the Rust static/shared library — no cxx bridge, no code generation, no special include paths.

Quick Integration

1. Build the Rust library:

cargo build -p bex-runtime --release

2. Copy the cpp-cli/ directory into your project (or reference it via BEX_ENGINE_ROOT).

3. In your CMakeLists.txt:

cmake_minimum_required(VERSION 3.16)
project(myapp LANGUAGES C CXX)

set(CMAKE_CXX_STANDARD 17)

# Point to the bex-engine root (required if not inside the repo)
set(BEX_ENGINE_ROOT "/path/to/bex-engine")

# Add the bex engine subdirectory
add_subdirectory(${BEX_ENGINE_ROOT}/cpp-cli bex_engine_build)

# Link against the imported bex::engine target
add_executable(myapp main.cpp)
target_link_libraries(myapp PRIVATE bex::engine)

BEX_ENGINE_ROOT Variable

The CMake build uses BEX_ENGINE_ROOT to locate the Rust library and the C header:

• Default: If not set, it walks up from CMAKE_SOURCE_DIR to find a directory containing Cargo.toml
• Override: Set -DBEX_ENGINE_ROOT=/path/to/bex-engine when running cmake

Imported Target

The CMakeLists.txt creates an INTERFACE library target bex::engine with all include paths and link libraries configured:

target_link_libraries(myapp PRIVATE bex::engine)

Helper Target

A rustlib custom target is provided to build the Rust library from CMake:

make rustlib

Required Files

File	Purpose
cpp-cli/bex_engine.h	Pure C ABI header (the FFI boundary)
target/release/libbex_runtime.a	Rust static library (Pure C ABI exports)

That's it. No generated headers, no bridge codegen, no extra include paths.

Error Code Reference

Code	Meaning
ABI_MISMATCH	Plugin ABI version incompatible
INVALID_MANIFEST	Manifest validation failed
HASH_MISMATCH	Package integrity check failed
NOT_FOUND	Plugin or resource not found
DISABLED	Plugin is disabled
UNSUPPORTED	Operation not supported by plugin
NETWORK_BLOCKED	Host not in plugin's allowed list
TIMEOUT	Request timed out
FUEL_EXHAUSTED	WASM fuel limit exceeded
CANCELLED	Request was cancelled
PLUGIN_FAULT	Plugin panicked or crashed
PLUGIN_ERROR	Plugin returned an error
NETWORK	Network error
STORAGE	Storage error
NOT_READY	Engine not ready
INTERNAL	Internal engine error

Technologies

Component	Technology	Version
WASM Runtime	Wasmtime	30
Interface Types	WIT (Component Model)	-
WASI	WASI Preview 1/2	-
Bindings	wit-bindgen	0.57.1
Database	Redb	2
HTTP	reqwest (rustls)	0.12
C++ FFI	Pure C ABI (extern "C")	-
Wire Format	FlatBuffers	-
Compression	zstd	0.13
Async Runtime	tokio	1
Cancellation	tokio-util (CancellationToken)	-
Package Format	Custom (BEX v1)	-

License

MIT