Run code in a sandbox

The types are in tnsai-quality, package com.tnsai.quality.sandbox. The Python and Node.js code tools run through it.

Run a command

static String run(String script) {
    SandboxSpec spec = SandboxSpec.builder()
        .image("python:3.12-slim")
        .networkPolicy(NetPolicy.denyAll())
        .resourceLimits(ResourceLimits.standard())
        .build();

    try (Sandbox sandbox = SandboxFactory.byId("container").create(spec)) {
        SandboxResult result = sandbox.execute(Command.of("python", "-c", script));
        return result.success() ? result.stdoutString() : result.stderrString();
    }
}

A Sandbox has four operations:

Command.of(...) takes the program and its arguments. Command.shell(script) runs the script with /bin/sh -c. Command.withStdin(argv, text) also writes text to standard input.

Read the result

Only wallClockMs is always measured. The process backend adds CPU time where the platform reports it.

Pick a backend

SandboxFactory.byId(id) returns a backend by name. SandboxFactory.preferred() returns the available backend with the highest priority, and throws SandboxException when none is available. Backends are found through ServiceLoader, so you can add your own by implementing SandboxFactory.

container

Each execute starts a new container with <runtime> run --rm, so nothing carries over between calls except files in read-write mounts. The spec needs an image, or create throws SandboxException.

• The root filesystem is read-only. /tmp is a tmpfs of diskMb megabytes; with diskMb 0 there is no writable /tmp.
• CPU, memory and process count map to --cpus, --memory and --pids-limit.
• The container sees only the spec's env. The container CLI keeps only the host variables it needs to reach the runtime: PATH, HOME, USER, XDG_RUNTIME_DIR, DOCKER_HOST, DOCKER_TLS_VERIFY, DOCKER_CERT_PATH and PODMAN_HOST.
• The timeout counts from the CLI call and allows 2 extra seconds for container start-up.

process

The command runs as a child process of the JVM, as the same user.

• The command starts in a new temporary folder, or in workingDir resolved inside it. The folder is deleted on terminate().
• The command sees only the spec's env; the host's environment is not passed on.
• The timeout and the output cap are enforced.
• The command can still read and write any host file the JVM user can, and it uses the host network.
• CPU, memory, disk and process limits are not enforced.
• A spec with mounts is rejected with SandboxException. A network policy other than inherit is logged as a warning and not enforced.

Use process only for code you would also run outside a sandbox.

Configure the sandbox

static SandboxSpec withFolders(Path inputs, Path outputs) {
    return SandboxSpec.builder()
        .image("python:3.12-slim")
        .fsMount(FsMount.readOnly(inputs, Path.of("/data")))
        .fsMount(FsMount.readWrite(outputs, Path.of("/work")))
        .workingDir("/work")
        .env(Map.of("PYTHONUNBUFFERED", "1"))
        .resourceLimits(new ResourceLimits(
            0.5,                         // CPU cores
            512,                         // memory, MB
            64,                          // /tmp size, MB
            Duration.ofSeconds(60),      // timeout per execute
            32))                         // max processes, 0 for no limit
        .build();
}

Resource limits

The constructor takes the same six values, or five with the default 4 MB output cap. It rejects a CPU or memory value of zero or less, a negative disk size or process count, a timeout that is not positive, and an output cap of zero or less.

Network and mounts

No backend enforces a host allow list.

Reuse sandboxes

static SandboxPool pool() {
    SandboxSpec spec = SandboxSpec.builder().warmable(true).build();
    return new SandboxPool(SandboxFactory.byId("process"), spec, 8, Duration.ofSeconds(2));
}

static String runPooled(SandboxPool pool, String file) {
    try (SandboxPool.Lease lease = pool.borrow()) {    // close() returns the sandbox
        return lease.execute(Command.of("python3", file)).stdoutString();
    }
}

SandboxPool(factory, spec, maxSize, acquireTimeout) keeps up to maxSize sandboxes. borrow() hands out an idle one, creates one while the pool is below maxSize, or waits up to acquireTimeout. After that wait it creates an extra sandbox outside the pool, which is terminated when its lease closes, so borrow() never blocks for longer. pool.close() terminates the idle sandboxes; leases still out terminate when they close.

A pooled process sandbox keeps its working folder between leases, so one borrower can see files another left behind. A container sandbox starts a fresh container on every execute, so pooling it saves little.

Observe executions

static SandboxResult observed(SandboxFactory factory, SandboxSpec spec, Command command) {
    SandboxExecutionListener listener = event -> System.out.println(
        event.backend() + " exit=" + event.exitCode() + " timedOut=" + event.timedOut()
            + " wallMs=" + event.resourceUsage().wallClockMs());

    try (Sandbox sandbox = new ObservedSandbox(factory.create(spec), listener, factory.id())) {
        return sandbox.execute(command);
    }
}

ObservedSandbox sends a SandboxExecutionEvent to the listener after each execute: event id, time, a sandbox id such as container-3, backend, image, the command's arguments, exit code, timeout flag, resource usage and the network policy name. An exception from the listener is logged and the result is still returned. RecordingSandboxExecutionListener keeps events in memory for tests.

Related

• Code tools
• Enforce permissions and content rules
• Approve risky actions
• Detect prompt injection