---
name: headless-browser-automation
title: Browser Automation on Headless Servers
category: devops
description: Running real browser automation (Playwright, Selenium) on headless Linux systems that lack desktop libraries, package managers, or writable system paths — common on ZimaOS, CasaOS, NAS appliances, and restricted containers. Covers Docker-based Chromium, system-library fallbacks, and anti-detection setup.
tags: [browser, automation, playwright, docker, headless, zimaos, casaos]
---

# Browser Automation on Headless Servers

## When to use
The system has no `apt`, `pacman`, or `apk`, and Playwright's bundled Chromium fails with `libatk-1.0.so.0: cannot open shared object file` or similar. Or `docker` is available but the user is not in the `docker` group.

## 1. Check system reality first
```bash
# Which package manager exists?
which apt-get || which pacman || which apk || which dnf || echo 'NONE'

# Are desktop libs present?
ldconfig -p | grep libatk
ldconfig -p | grep libgtk

# Docker access?
docker ps 2>&1 | head -1
```

**If a package manager exists:** Install missing libs with it and use Playwright normally.

**If no package manager exists:** Go to Docker approach (section 2).

## 2. Dockerized Chromium for ZimaOS/CasaOS (recommended)

When the host OS has **no package manager** (no apt/pacman/apk) and a **read-only root filesystem** (docker build fails), the robust approach is to start a minimal Debian container, install Chromium inside it, and expose the browser as an HTTP API.

### The "Debian Container Build-Up" method (works when upstream images fail)
```bash
# 1. Start a Debian container that stays alive
sudo docker run -d --name browser-build debian:bookworm-slim sleep 3600

# 2. Update and install Chromium + Python inside the container
sudo docker exec browser-build apt-get update -qq
sudo docker exec browser-build apt-get install -y -qq --no-install-recommends chromium python3 curl

# 3. Verify Chromium works inside the container
sudo docker exec browser-build chromium --version

# 4. Commit the container to a local image
sudo docker stop browser-build
sudo docker commit browser-build kiwi-browser:v1
sudo docker rm browser-build

# 5. Start the committed image with port mapping + auto-restart
sudo docker run -d \
  --name kiwi-browser \
  --restart unless-stopped \
  -p 0.0.0.0:30000:3000 \
  kiwi-browser:v1 \
  bash -c 'cd /app && python3 server.py'
```

**Why this works on ZimaOS:** It avoids `docker build` (needs writable `/root/.docker`) and avoids pulling large upstream images that may timeout or have unavailable tags. Debian slim is ~30MB and pulls reliably. Everything happens inside the container.

### Upgraded: Playwright + real interaction (v2) — True mouse clicks, typing, scrolling
For automating forms, clicking buttons, or uploading files, the CLI approach above is insufficient. Instead, install **Playwright inside the Debian container** and expose it as an async HTTP API.

```bash
# Install Playwright + aiohttp inside the same Debian container
sudo docker exec browser-build pip3 install --break-system-packages playwright aiohttp
sudo docker exec browser-build playwright install chromium
```

> **Pitfall:** Debian Bookworm marks pip as "externally managed" (PEP 668). You **must** pass `--break-system-packages` or the install will fail.

Then copy the v2 server script (see `scripts/kiwi-browser-server-v2.py`), commit, and run:

```bash
sudo docker stop browser-build
sudo docker commit browser-build kiwi-browser:v2
sudo docker rm browser-build
sudo docker run -d \
  --name kiwi-browser \
  --restart unless-stopped \
  -p 0.0.0.0:30000:3000 \
  kiwi-browser:v2 \
  bash -c 'cd /app && python3 server-v2.py'
```

**v2 endpoints (real interaction):**
- `POST /click` — click by CSS selector or (x,y) coordinates
- `POST /type` — type into input, optional submit with Enter
- `POST /scroll` — scroll directions or scroll-to selector
- `POST /evaluate` — execute arbitrary JavaScript in page context

See `references/zimaos-playwright-v2-recipe.md` for full reproduction.

### Quick alternative: Pull upstream image (if network is fast)
```bash
# Use the official Playwright image
docker run -d \
  --name playwright-browser \
  --network host \
  -p 9222:9222 \
  -e DISPLAY=:99 \
  mcr.microsoft.com/playwright:v1.59.0-jammy \
  bash -c "cd /ms-playwright && npx playwright-core launch-server chromium --port=9222 --ws-path=/"
```

### Connect from Python / Agent
```python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    # Connect to remote browser in Docker
    browser = p.chromium.connect_over_cdp("http://localhost:9222")
    page = browser.new_page()
    page.goto("https://example.com")
    page.screenshot(path="screenshot.png")
    browser.close()
```

### Alternative: CDP connection (raw)
```python
import requests

# Start Chrome/CDP in container with --remote-debugging-port=9222
# Then connect via websocket or HTTP:
r = requests.get("http://localhost:9222/json/version")
print(r.json())
```

## 3. Anti-detection setup
Regardless of where Chromium runs, mask automation flags:

```python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(
        headless=True,
        args=[
            '--no-sandbox',
            '--disable-gpu',
            '--disable-dev-shm-usage',
            '--disable-blink-features=AutomationControlled',
            '--disable-web-security',
            '--window-size=1920,1080',
        ]
    )
    context = browser.new_context(
        viewport={'width': 1920, 'height': 1080},
        user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120.0.0.0 Safari/537.36',
        locale='de-DE',
    )
    page = context.new_page()
    page.add_init_script('''
        Object.defineProperty(navigator, 'webdriver', {get: () => undefined});
        Object.defineProperty(navigator, 'plugins', {get: () => [1,2,3,4,5]});
        window.chrome = { runtime: {} };
    ''')
```

## 4. User-permission prerequisites
The agent user must either:
- Be in the `docker` group: `sudo usermod -aG docker az-a` (then re-login)
- Or have explicit `NOPASSWD` sudo for `docker`: add `/usr/bin/docker` to sudoers

## Pitfalls
- **Debian 12+ requires `--break-system-packages`.** When installing Python packages inside a Debian Bookworm container with pip, you will hit *"externally-managed"* (PEP 668). Always pass `--break-system-packages` for Playwright, aiohttp, Selenium, or similar.
- **ZimaOS has no package manager.** `apt-get`, `pacman`, `apk` are all absent. Do not waste time trying to install system libraries on the host.
- **ZimaOS rootfs is read-only.** `docker build` fails because `/root/.docker` cannot be written. Work around this by running a Debian container, installing packages inside it, and committing it: `docker run -d debian:bookworm-slim sleep 3600`, then `docker exec <id> apt-get install chromium`, then `docker commit`.
- **Playwright's bundled Chromium expects Ubuntu-like libs.** On stripped-down NAS OS, it will always fail with missing `.so` files.
- **Docker group membership requires re-login.** After `usermod -aG docker`, the user must open a new shell session.
- **Browserless services may need memory limits.** Chromium in Docker uses 200-400 MB baseline; restrict with `--memory=512m` if the NAS is tight.
- **Screenshots accumulate fast.** Always write to `/DATA/AppData/hermes/cache/` or similar, and clean up periodically.
- **Amazon DE aggressively blocks headless Chromium** even inside a container. Expect "Page Not Found" on product pages. Use the API for sites with lighter bot detection, or add proxy rotation (not covered here).
- **ZimaOS rootfs is read-only.** `docker build` fails because `/root/.docker` cannot be written. Work around this by running a Debian container, installing packages inside it, and committing it: `docker run -d debian:bookworm-slim sleep 3600`, then `docker exec <id> apt-get install chromium`, then `docker commit`.
- **Playwright's bundled Chromium expects Ubuntu-like libs.** On stripped-down NAS OS, it will always fail with missing `.so` files.
- **Docker group membership requires re-login.** After `usermod -aG docker`, the user must open a new shell session.
- **Browserless services may need memory limits.** Chromium in Docker uses 200-400 MB baseline; restrict with `--memory=512m` if the NAS is tight.
- **Screenshots accumulate fast.** Always write to `/DATA/AppData/hermes/cache/` or similar, and clean up periodically.
- **Amazon DE aggressively blocks headless Chromium** even inside a container. Expect "Page Not Found" on product pages. Use the API for sites with lighter bot detection, or add proxy rotation (not covered here).

## Verification checklist
- [ ] Docker accessible (`docker ps` works)
- [ ] Container starts without port conflict
- [ ] `curl http://localhost:30000/health` returns `{"status": "ok"}` (or equivalent for your chosen port)
- [ ] Screenshot or scrape from target site succeeds
- [ ] Anti-detection script injected (`navigator.webdriver` is undefined)

## Session-specific reference
- `references/kiwi-browser-api-recipe.md` — exact reproduction recipe for the Docker-based Chromium HTTP API built on ZimaOS during session 2026-05-07.
- `references/zimaos-playwright-v2-recipe.md` — upgraded v2 recipe with Playwright in Debian container for true mouse clicks, typing, scrolling, and JS evaluation (2026-05-09 session).
- `templates/kiwi-browser-server.py` — the Flask-style HTTP server (v1) that runs inside the container.
- `templates/kiwi-browser-server-v2.py` — Playwright async server (v2) with `/click`, `/type`, `/scroll`, `/evaluate` endpoints for real browser interaction.