Wasmtime Production Hardening: Fuel, Memory, Epoch Interrupts, and WASI Capability Allowlists
Problem
Wasmtime is the most widely-deployed standalone WebAssembly runtime — used inside Spin, wasmCloud, Fastly Compute, Shopify’s Function Runner, and many self-hosted serverless platforms. Its sandbox guarantees are strong: linear memory isolation, no ambient authority, deterministic semantics. They are also defeated by misconfiguration.
A Module::new + Linker::module + Instance::new flow with default Config and a permissive WASI context will:
- Execute indefinitely. A
loop {}in the module hangs the host thread. - Allocate up to 4 GB of linear memory per instance with no rejection.
- Mmap arbitrary table and stack sizes within Wasmtime’s defaults.
- Inherit the host process’s filesystem view through
WasiCtxBuilder::inherit_stdio/preopened_dirif the embedder calls them carelessly. - Make outbound network calls if
wasi:socketsis wired in. - Use floating-point and SIMD operations that may differ across hosts (a problem for deterministic execution, but not strictly a security issue).
In production, the runtime is the boundary between trusted host code and untrusted user code (WASM modules uploaded by tenants, dynamic plugins, edge-runtime customer code). Every default is a decision the embedder needs to make explicitly.
The specific gaps in a default Wasmtime embedding:
- No CPU bound. A malicious or buggy module can consume a thread for arbitrary time.
- No memory ceiling beyond linear-memory’s 4 GB cap. Many small allocations (tables, instances, stack) accumulate without per-tenant accounting.
- No I/O resource accounting. A module reading or writing through preopened FDs has no quota.
- WASI capability surface defaulted to inherit. Embedders who want “give the module access to the cwd” sometimes hand over the entire host filesystem.
- No epoch deadline support enabled. The cooperative-deadline mechanism that lets the host reliably interrupt long-running modules is opt-in.
This article covers Config knobs for fuel, memory, and epoch interrupts; the Store::limiter API for per-instance caps; explicit WASI capability allowlists; and operational telemetry for tracking abusive modules.
Target systems: Wasmtime 22+ (Rust API and C API, with similar shapes in language bindings). Examples are in Rust; the same knobs exist in wasmtime-py, wasmtime-go, @bytecodealliance/wasmtime, and the wasmtime C API.
Threat Model
- Adversary 1 — Untrusted module author: an attacker who uploads or supplies a
.wasmmodule to a multi-tenant platform. They want to exhaust resources to cause denial of service for other tenants, or escape the sandbox to access the host filesystem, network, or other tenants’ data. - Adversary 2 — Compromised module via supply chain: a previously-trusted module containing malicious code introduced through a dependency compromise or builder error.
- Adversary 3 — Resource accounting bypass: an attacker who knows the embedder’s resource limits and crafts a module that operates just below them indefinitely, denying capacity to legitimate workloads.
- Access level: Adversary 1 has WASM module-upload capability and module input control. Adversary 2 has the same plus prior trust. Adversary 3 has long-term observation of resource limits.
- Objective: CPU exhaustion, memory exhaustion, filesystem read or write outside the intended directory, network exfiltration via WASI sockets, persistence within the runtime, host process compromise.
- Blast radius: Without hardening: a single tenant module hangs a thread or consumes memory until OOM, affecting all other tenants on the same Wasmtime process. With hardening: per-instance bounds enforce fairness; capability denial blocks unauthorized I/O entirely.
Configuration
Step 1: Configure CPU Bounds with Fuel and Epoch Interrupts
Wasmtime offers two CPU-limiting mechanisms. Use them together for production.
Fuel counts execution units. Every WASM operation consumes a fixed amount; when fuel is exhausted, the runtime traps. Fuel is precise but expensive (every operation pays the accounting cost).
Epoch interrupts are a cooperative-deadline mechanism. The host increments an epoch counter (typically from a timer thread); the runtime compares epoch values at safe points. When the deadline epoch passes, the next safe point traps. Cheaper than fuel but coarser-grained.
// wasmtime_config.rs
use wasmtime::*;
use std::time::Duration;
pub fn make_config() -> Config {
let mut config = Config::new();
// Enable both fuel and epoch tracking.
config.consume_fuel(true);
config.epoch_interruption(true);
// Reject modules using features we do not allow.
config.wasm_simd(true); // safe; allow
config.wasm_threads(false); // disallow shared memory across instances
config.wasm_multi_memory(false); // disallow multi-memory until policy is decided
config.wasm_reference_types(true);
config.wasm_bulk_memory(true);
// Compilation cache for expensive modules.
config.cache_config_load_default().ok();
// Static memory reservations sized to the worst-case instance.
config.static_memory_maximum_size(64 * 1024 * 1024); // 64 MiB
config.static_memory_guard_size(2 * 1024 * 1024 * 1024);
config.dynamic_memory_guard_size(64 * 1024);
config
}
pub fn run_module(wasm: &[u8], input: &str) -> anyhow::Result<String> {
let engine = Engine::new(&make_config())?;
let mut store = Store::new(&engine, ());
// Per-invocation resource grant.
store.set_fuel(10_000_000)?; // ~10M ops max
store.set_epoch_deadline(1); // trap on next epoch tick after deadline
// Increment-epoch thread (started once per Engine, not per Store).
let engine_clone = engine.clone();
std::thread::spawn(move || loop {
std::thread::sleep(Duration::from_millis(50));
engine_clone.increment_epoch();
});
let module = Module::new(&engine, wasm)?;
let instance = Instance::new(&mut store, &module, &[])?;
let func = instance.get_typed_func::<(), i32>(&mut store, "run")?;
let _ = func.call(&mut store, ())?;
Ok(format!("fuel remaining: {}", store.get_fuel().unwrap_or(0)))
}
The 50ms epoch tick combined with set_epoch_deadline(1) produces a hard 50–100ms wall-clock budget per call, enforced regardless of what the module does. Fuel provides finer-grained accounting for billing or quota purposes — the host can observe how much was consumed before the trap, even on legitimate completion.
Step 2: Per-Instance Memory and Resource Limiter
The Store::limiter callback controls every memory growth, table growth, and instance creation. Reject when limits are exceeded.
use wasmtime::ResourceLimiter;
struct Limits {
max_memory: usize,
max_tables: u32,
max_instances: u32,
current_instances: u32,
}
impl ResourceLimiter for Limits {
fn memory_growing(&mut self, current: usize, desired: usize, _max: Option<usize>)
-> anyhow::Result<bool> {
Ok(desired <= self.max_memory)
}
fn table_growing(&mut self, _current: u32, desired: u32, _max: Option<u32>)
-> anyhow::Result<bool> {
Ok(desired <= self.max_tables)
}
fn instances(&self) -> usize { self.max_instances as usize }
fn tables(&self) -> usize { 100 }
fn memories(&self) -> usize { 1 }
}
let mut store = Store::new(&engine, Limits {
max_memory: 32 * 1024 * 1024, // 32 MiB
max_tables: 1024,
max_instances: 1,
current_instances: 0,
});
store.limiter(|state| state);
The limiter is consulted on every growth attempt. A module that requests 1 GB of memory traps cleanly with MemoryGrowError rather than allocating and starving the host.
Step 3: WASI Capability Allowlist
WASI is the syscall surface for WASM. Default WasiCtxBuilder::inherit_* calls give the module a copy of the host’s I/O. Always start from empty and add only what the module needs.
use wasmtime_wasi::preview2::{WasiCtxBuilder, WasiCtx};
use wasmtime_wasi::DirPerms;
use wasmtime_wasi::FilePerms;
fn build_wasi_ctx(tenant_id: &str) -> anyhow::Result<WasiCtx> {
let workdir = format!("/var/lib/wasm-tenants/{tenant_id}/workdir");
let readonly_assets = "/usr/share/wasm-platform/assets";
let dir = cap_std::fs::Dir::open_ambient_dir(
&workdir, cap_std::ambient_authority())?;
let assets = cap_std::fs::Dir::open_ambient_dir(
readonly_assets, cap_std::ambient_authority())?;
let mut wasi = WasiCtxBuilder::new();
wasi.preopened_dir(dir, DirPerms::all(), FilePerms::all(), "/work")?;
wasi.preopened_dir(assets, DirPerms::READ, FilePerms::READ, "/assets")?;
// No stdin. Wrap stdout/stderr in size-limited buffers.
wasi.stdin(Box::new(wasmtime_wasi::preview2::pipe::ClosedInputStream));
wasi.stdout(Box::new(LimitedWriter::new(64 * 1024)));
wasi.stderr(Box::new(LimitedWriter::new(64 * 1024)));
// No environment variables; no command-line arguments.
// No clocks beyond monotonic.
wasi.allow_blocking_current_thread(false);
Ok(wasi.build())
}
The two preopened directories are cap-std handles, not paths. The module cannot use .. traversal to escape — cap-std refuses paths that resolve outside the directory handle’s view. Symlinks within the directory are followed; symlinks pointing outside are rejected at open time.
For network access, use the WASI Preview 2 sockets API explicitly:
use wasmtime_wasi::preview2::SocketAddrCheck;
let mut wasi = WasiCtxBuilder::new();
wasi.socket_addr_check(|addr, _| {
// Allowlist: only specific external endpoints.
matches!(addr.ip().to_string().as_str(),
"192.168.10.5" | "192.168.10.6")
});
wasi.allow_tcp(true);
wasi.allow_udp(false);
Block by default: do not call allow_tcp(true) unless the workload genuinely needs outbound network. Most WASM workloads do not.
Step 4: Disable Risky Features
Many proposals (threads, multi-memory, GC, exceptions) add complexity to the runtime and broaden the attack surface. Disable what you do not use.
config.wasm_threads(false); // shared linear memory
config.wasm_multi_memory(false);
config.wasm_memory64(false); // 64-bit linear memory; rare in production
config.wasm_function_references(true); // safe
config.wasm_gc(false); // disable until policy is set
config.wasm_exceptions(false);
config.wasm_relaxed_simd(false); // less audited than core SIMD
config.wasm_tail_call(true); // safe; ergonomic for some compilers
Each disabled feature reduces both runtime and verifier surface. Re-enable individually after auditing the runtime version’s stability for that feature.
Step 5: Module Validation and Cache Hygiene
Validate every module before instantiation. Wasmtime validates by default during Module::new, but explicit validation surfaces errors before the engine commits to caching.
let validator_config = wasmparser::WasmFeatures {
threads: false,
multi_memory: false,
relaxed_simd: false,
..wasmparser::WasmFeatures::default()
};
let mut validator = wasmparser::Validator::new_with_features(validator_config);
validator.validate_all(wasm_bytes)?;
let module = Module::new(&engine, wasm_bytes)?;
The validator runs at compile time; the engine then caches the compiled artifact. If a module compiles cleanly once, subsequent invocations skip the cost. Pin the cache directory to a tenant-scoped path so cache poisoning across tenants is impossible:
# /etc/wasmtime/cache.toml
[cache]
enabled = true
directory = "/var/cache/wasmtime/tenant-{tenant_id}"
cleanup-interval = "1d"
files-total-size-soft-limit = "1Gi"
Step 6: Operational Telemetry
Track per-tenant and per-module behavior:
wasmtime_module_executions_total{tenant, module} counter
wasmtime_fuel_consumed{tenant, module} histogram
wasmtime_traps_total{tenant, module, kind} counter
wasmtime_memory_growth_rejections_total{tenant} counter
wasmtime_instance_lifetime_seconds{tenant, module} histogram
wasmtime_wasi_calls_total{tenant, capability, op} counter
Alert on spikes in wasmtime_traps_total{kind="fuel_exhausted"} or wasmtime_memory_growth_rejections_total — these usually indicate either an abusive module or a misconfigured limit.
Expected Behaviour
| Signal | Default Wasmtime | Hardened |
|---|---|---|
loop {} in a module |
Hangs the host thread indefinitely | Traps within ~50–100 ms (epoch deadline) |
| Module attempts to grow memory to 1 GB | Allocates if host memory allows | Trap; memory_growing returns Ok(false) |
Module tries open("/etc/passwd") |
Succeeds if WASI inherits stdio paths | ENOENT (cap-std refuses) |
| Module connects to external IP | Succeeds with default WASI sockets | Refused unless on allowlist |
Module uses wasm_threads |
Compiles | Compile-time error: feature disabled |
| Per-call accounting | None | Fuel + epoch + memory metrics per Store |
| Cache poisoning across tenants | Possible if cache shared | Per-tenant cache directories |
Verification:
# Confirm CPU limit fires.
cat <<'EOF' > /tmp/loop.wat
(module (func (export "run") (result i32)
(loop $l (br $l))
(i32.const 0)))
EOF
wasmtime compile /tmp/loop.wat -o /tmp/loop.cwasm
timeout 3 wasm-runner /tmp/loop.cwasm
# Expected: trap "epoch deadline reached" within 100ms.
# Confirm filesystem capability check.
cat <<'EOF' > /tmp/escape.wat
(module
(import "wasi_snapshot_preview1" "path_open" (func ...))
(func (export "run") ...))
EOF
# Should receive errno=44 (ENOENT) for paths outside the preopened dir.
Trade-offs
| Aspect | Benefit | Cost | Mitigation |
|---|---|---|---|
| Fuel + epoch combo | Hard CPU bound regardless of module behavior | Fuel adds ~5-15% per-op cost; epoch incrementer thread costs negligible CPU | Use epoch alone for performance-critical workloads where billing-grade accuracy is not needed; fuel for billing/quota workloads. |
| Per-instance memory limiter | Prevents memory exhaustion across tenants | Module rejection when limit hit; needs graceful error path | Surface the limit in the module ABI so well-behaved modules can degrade rather than crash. |
| Empty WASI context | Smallest attack surface | Modules that need any I/O must declare it explicitly | Provide a tenant-config layer mapping declared capabilities to validated allowlists. |
| Disabled features | Reduces runtime surface | Some compilers emit threads/multi-memory by default | Document feature requirements per platform; reject modules at upload time that use disabled features. |
| Cache per tenant | Eliminates cross-tenant cache poisoning | Disk usage scales with tenant count and module count | Cap per-tenant cache size; expire idle tenants’ caches. |
| Telemetry | Detect abuse early | Metric cardinality grows with tenants × modules | Aggregate at high cardinality (per-tenant) and use exemplars for per-module drill-down. |
Failure Modes
| Failure | Symptom | Detection | Recovery |
|---|---|---|---|
| Epoch incrementer thread dies | Modules run beyond their deadline | wasmtime_traps_total{kind="epoch_deadline"} drops to zero while module count stays nonzero |
Restart the incrementer; supervise with a watchdog. |
| Fuel exhaustion misconfigured | All modules trap on first instruction | High rate of fuel_exhausted traps; first-call latency near zero |
Increase the per-call fuel grant. Determine empirically by measuring fuel consumption of representative workloads. |
| Module bypasses WASI by using compiled-in syscalls | A WASM module ostensibly without WASI imports still does I/O | Should be impossible — WASM has no host calls without imports | Audit the module’s import section. Rebuild the toolchain if the build inadvertently compiled host calls in. |
| Cap-std symlink check fails | Module reads file through a stale symlink that pointed inside but now points outside | File-access logs show an unexpected target | Cap-std resolves at open time; rare but possible if a directory is shared with non-WASI tenants. Lock the workdir to root-owned, immutable directory entries. |
| Memory limit too low for legitimate use | Module fails to grow memory; legitimate workload broken | wasmtime_memory_growth_rejections_total rises with no obvious abuse pattern |
Raise the limit. Track per-module memory peaks and set limit at 1.5x the observed peak. |
| Compilation cache corruption | Modules fail to load with cache errors | Wasmtime logs cache miss or compilation errors at load time |
Clear the cache directory; rebuild. Investigate file-system or filesystem-driver corruption. |
| Long-running compilation blocks request | First-time module load takes seconds | High p99 latency on cold-start; CPU time spent in Module::new |
Pre-compile modules at upload; serve precompiled .cwasm artifacts from the cache. |