eilandert/mailstrix repository overview

⁠Mailstrix — YARA malware scanning for mail, ICAP and clamd

Mailstrix is the owl that finds malware hiding in your mail. It takes hostile attachments apart — unwrapping OLE2/OOXML, VBA, RTF objects, PDFs, archives and nested carriers — until YARA detection rules can finally see the dangerous bits. It runs out-of-process behind Rspamd (async HTTP), SpamAssassin, an ICAP server, Dovecot Sieve, or standalone.

strixd is a small HTTP service that scans email for malware with YARA⁠. You hand it a message (or one attachment) on POST /scan; it runs compiled YARA rules over it and tells you which ones matched. It ships as a ready-to-run Docker image with an initial rules bundle — see Quick start⁠ below or pull it straight from Docker Hub⁠.

Why YARA, in one paragraph. YARA is the rule engine malware analysts use to recognise families of malicious files — booby-trapped Office docs, packed executables, phishing kits, script droppers. A plain string signature dies the moment the author edits one byte; a YARA rule matches the shape of a file (PE imports, section entropy, embedded magic) and survives the next variant. strixd compiles those rules — libyara modules and all — and runs them over your mail.

Six ways to plug it into a mail server, all shipped in this repo:

• rspamd — an async mailstrix.lua plugin (contrib/rspamd/⁠) POSTs each message/part to strixd at SMTP time and turns the hits into a spam-score symbol.
• SpamAssassin — the Mailstrix.pm⁠ plugin scans each message through the same central service and turns a YARA match into a spam-score hit (contrib/spamassassin/⁠).
• Dovecot / Sieve — the lean strix-scan⁠ client scans at delivery and a Sieve rule quarantines a match (contrib/sieve/⁠).
• Postfix / Sendmail (milter) — the lean strix-milter⁠ runs on the MTA host, POSTs each message to strixd and stamps the verdict as a header. It always accepts; your milter_header_checks decides what to do about it (contrib/postfix/⁠).
• ICAP — set MAILSTRIX_ICAP_ADDR and strixd also speaks ICAP (RFC 3507) so an ICAP-aware proxy or content-filter (Squid, c-icap) scans REQMOD/RESPMOD bodies through the same engine (ICAP mode⁠).
• clamd streams — opt-in Unix/TCP listeners in strixd accept INSTREAM from supported clamd clients (subset, limits and examples⁠).

These integrations use the same strixd scan engine and rules:

It runs out of process, never inside the MTA worker, because libyara is a C library (CGO): in an rspamd worker it would block the event loop and drag a heavy C dependency into the mail image. Separate, the caller stays async and strixd can be scaled, restarted, or reload its rules on its own. Same shape as the gozer⁠ DCC/Razor/Pyzor backend.

⁠Exactly what it does

• Scans mail with YARA — POST /scan raw message bytes (or one MIME part), get back the matched rules as JSON; the rspamd mailstrix.lua plugin (contrib/rspamd/⁠) wires the hits into the spam score, or the strix-scan client scans at delivery from Dovecot/Sieve (contrib/sieve/⁠).
• Loads a compiled rules bundle — the image includes a seed; the rolling rules-current publication can refresh the active bundle without changing the binary. The source set includes YARA-Forge, signature-base, ANY.RUN, Didier Stevens, bartblaze, InQuest, CAPEv2 and YARAify.
• Decompresses Office macros before matching — MS-OVBA VBA out of .docm/.xlsm/.doc/.xls, scans the cleartext (sets the VBA rule var).
• Cracks open containers — pulls the hidden payload out of: OLE2/OOXML, RTF \objdata, OLE Package (Ole10Native), MSI, Outlook .msg, TNEF (winmail.dat), OneNote .one, PDF (FlateDecode streams), .lnk shortcuts, VBE/JSE encoded scripts, and nested archives (zip/7z/rar/gz/tar.gz/cab, recursive) — then scans each. A part that is itself a message is walked as MIME, so attachments inside a forwarded or message/rfc822 carrier are unpacked too.
• Deobfuscates before matching — decodes long base64/hex runs, undoes StrReverse, and folds the olevba string set (Chr/ChrW concat, Replace(), Array() Xor k, Environ, Dridex DridexUrlDecode) to cleartext; a bounded recursive pass unwinds multi-stage (2+-layer) payloads, not just the first layer.
• Resolves Excel 4.0 (XLM) macros — detects hidden/very-hidden macrosheets (OOXML + legacy .xls BIFF), reassembles ptg-token formula strings (BIFF8/.xlsb/SLK), and runs a bounded XLM emulator (cell eval, GOTO, SET.VALUE) to resolve obfuscated cell references. BIFF8 macrosheets also resolve SHRFMLA shared formulas referenced by ptgExp, within parser limits.
• Triages PDFs — surfaces /OpenAction, /JS, /Launch, /EmbeddedFile, /JBIG2Decode and hex-name obfuscation as scoreable markers.
• Catches macro-less & exploit attacks — Equation Editor (CVE-2017-11882), remote-template injection (CVE-2017-0199 / T1221), DDE/DDEAUTO fields, MHTML/x-usc (CVE-2021-40444), Shell.Explorer CLSID, and OLE structural indicators (ObjectPool, embedded Flash, digital-signature, doc-security).
• Decrypts default-password documents — VelvetSweatshop XOR, BIFF8 RC4, and OOXML agile/standard AES, so an "encrypted" but default-keyed payload is unlocked and re-scanned (other encryption is flagged, not cracked).
• Analyses carved executables — PE/ELF structural checks on embedded/decoded binaries (section entropy / packing, overlay, .NET, anomalies), plus base64-PE carving that re-aligns a padded MZ header so the pe rules fire.
• Ships 100+ of its own rules — docker/local-rules/⁠ scores the structural evidence extraction surfaces, so a sample with no public signature still has something to hit: weight-of-evidence maldoc scoring, oleid/oletimes/pdfid parity indicators, XLM dropper shapes, macro-less execution carriers, HTML/SVG smuggling, PowerShell/VBS/JScript dropper families, polyglot and renamed-container evasion, and carved-PE structure — full inventory under What Mailstrix detects on its own⁠. Among them: an mraptor-style autoexec∧write∧execute rule, olevba suspicious-keyword / VBA-shellcode-API heuristics, LOLBin / WMI / PowerShell / anti-analysis intent rules, HTML-smuggling (data: URI, embedded SVG, javascript:/vbscript: and entity-hidden script URIs) detection, and a position-independent-shellcode GetEIP prologue rule.
• Scales effort under load — a single 1–10 effort dial (MAILSTRIX_EFFORT) scales decode depth, XLM/PDF clamps, feeds and scan timeout; auto sheds a level at a time as the admission gate fills and climbs back as it drains.
• Speaks ICAP too (optional) — MAILSTRIX_ICAP_ADDR adds an RFC 3507 REQMOD/RESPMOD listener for ICAP-aware proxies, sharing the same scan engine, cache and concurrency gate as /scan.
• Uses the attachment name — filename/extension YARA vars from the plugin's X-MAILSTRIX-Filename, so name-keyed (THOR/Loki) rules fire.
• Checks abuse.ch feeds (optional) — URLhaus malware-URL/host lookup (with URL defanging), MalwareBazaar attachment-SHA256 lookup, and ThreatFox URL/domain IOCs; all cached, fail-open.
• Drops/demotes noisy rules — MAILSTRIX_RULE_DENYLIST (suppress) and MAILSTRIX_RULE_ALLOWLIST (keep but score log-only) without patching upstream.
• Canary mode — MAILSTRIX_CANARY=1 returns hits as log-only metadata so integrations can observe rule/feed behaviour without scoring or blocking mail.
• Caches verdicts — SHA256(body) → matches (LRU+TTL), plus request coalescing and an optional shared Redis/Valkey L2 for a high-volume firehose. The L2 can be integrity-protected with MAILSTRIX_REDIS_MAC_KEY: an HMAC-SHA256 bound to the Redis key, so a tampered or un-MACed value, or a valid value lifted to a different key, is treated as a cache miss instead of a trusted verdict. The MAC carries no nonce or counter, so it does not make an entry fresh — a stale value restored under its own key still verifies.
• Fails open, always — a scan error, timeout, or libyara panic is reported as "no match"; a broken scanner never blocks mail. Bounded concurrency, per-scan timeout, body cap, graceful drain on SIGTERM.
• Updatable rules without a rebuild, origin-checked — strixd fetch-rules pulls a version-matched compiled bundle into a cache; SIGHUP reloads it. The remote manifest must carry a valid ed25519 signature from a trusted public key — one pinned into the binary (internal/rulespin), plus any the operator adds through MAILSTRIX_RULES_EXTRA_SIGNING_KEYS, which is additive and cannot remove a pinned key. Otherwise the update is refused, so a tampered, truncated or untrusted-key manifest never reaches the compiler. There is no switch to turn verification off (details⁠).
• CLI tools — strixd scan (local triage), strixd extract (dump what a container carves), strixd check-rules, strixd info; and strix-scan, a tiny CGO-free client for a Dovecot/Sieve box (contrib/sieve/⁠).
• Optional CAPE detonation adapter (report-only) — a separate, disabled-by-default HTTPS listener (MAILSTRIX_CAPE_CONFIG_FILE) can submit an attachment to a CAPEv2⁠ sandbox and poll its report. It is deliberately not in the delivery path: MAILSTRIX_CAPE_POLICY accepts only static-only, so mail always follows the static verdict and a pending detonation never holds, quarantines or tempfails a message. No automatic submission, no enforcement, no callback bridge (configuration⁠, operator guide⁠).
• Observable — /health, /ready, /version, Prometheus /metrics (scans, matches, cache, per-extractor counters, rule staleness).

⁠Install

Three ways, pick one:

Debian/Ubuntu package (.deb, amd64 + arm64) — attached to every release⁠:

# Resolve the latest version + your arch (release assets are version-pinned,
# e.g. strixd_1.1.0_amd64.deb — the bare latest/download/ path is not).
VER=$(curl -fsSL https://api.github.com/repos/myguard-labs/mailstrix/releases/latest \
        | grep -oP '"tag_name":\s*"v\K[^"]+')
ARCH=$(dpkg --print-architecture)   # amd64 or arm64
BASE=https://github.com/myguard-labs/mailstrix/releases/download/v${VER}

# strixd — the daemon (systemd unit + /etc/mailstrix/strixd.env config)
curl -fsSLO "${BASE}/strixd_${VER}_${ARCH}.deb"
sudo apt install "./strixd_${VER}_${ARCH}.deb"

# fetch the rolling compiled rule bundle into the cache dir, then start it
sudo -u strixd strixd fetch-rules -cache-dir /var/cache/mailstrix   # or drop your own .yar in /var/lib/mailstrix/rules
sudoedit /etc/mailstrix/strixd.env                                 # set MAILSTRIX_TOKEN, feeds, …
sudo systemctl enable --now strixd

# strix-scan — the lean CGO-free Sieve/LDA client (no daemon)
curl -fsSLO "${BASE}/strix-scan_${VER}_${ARCH}.deb"
sudo apt install "./strix-scan_${VER}_${ARCH}.deb"

# strix-milter — the lean CGO-free Postfix/Sendmail milter (systemd unit)
curl -fsSLO "${BASE}/strix-milter_${VER}_${ARCH}.deb"
sudo apt install "./strix-milter_${VER}_${ARCH}.deb"

The daemon package installs a hardened systemd unit (unprivileged strixd user, ProtectSystem=strict, NoNewPrivileges) and a documented /etc/mailstrix/strixd.env. State (rules) lives in /var/lib/mailstrix.

Static binaries — strixd-linux-{amd64,arm64}, strix-scan-linux-{amd64,arm64} and strix-milter-linux-{amd64,arm64} plus SHA256SUMS are on the same release page.

Docker — see Quick start⁠ below (rules baked in).

⁠Quick start

The image includes a compiled seed bundle, so a token is all you need:

docker run -d --name strixd \
    -e MAILSTRIX_TOKEN=changeme \
    -p 8079:8079 \
    eilandert/mailstrix

# ask it something:
printf 'hello' | curl -s -H 'X-MAILSTRIX-Token: changeme' \
    --data-binary @- http://127.0.0.1:8079/scan
# -> {"matches":[]}

To use your own rules instead of the baked bundle:

docker run -d --name strixd \
    -e MAILSTRIX_TOKEN=changeme \
    -e MAILSTRIX_RULES= \
    -e MAILSTRIX_RULES_DIR=/rules \
    -v "$PWD/myrules:/rules:ro" \
    -p 8079:8079 \
    eilandert/mailstrix

Send an attachment name so name-keyed rules fire (base64 — the name is attacker-controlled, encoding it stops header injection). To check that detection works end to end, send the harmless EICAR test string⁠:

printf '%s' 'X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*' |
    curl -s -H 'X-MAILSTRIX-Token: changeme' \
    -H "X-MAILSTRIX-Filename: $(printf 'eicar.com' | base64)" \
    --data-binary @- http://127.0.0.1:8079/scan
# -> {"matches":[{"rule":"SUSP_Just_EICAR",…},{"rule":"TRELLIX_ARC_Malw_Eicar",…}]}

A few placeholder bytes such as MZ... are not malware and correctly return {"matches":[]}; test with a real sample. Set MAILSTRIX_VERBOSE=1 to log one line per request.

The /scan reply names the rule and its source ruleset file:

{"matches":[{"rule":"Suspicious_Macro","namespace":"sigbase-gen_maldoc.yar","tags":["office"],"meta":{"author":"…"}}]}

The list is [] (never null) when nothing matched. namespace is the file the rule was compiled from, so a generic rule like http is traceable to the set that shipped it.

When strixd could not compute a complete verdict, the reply still answers 200 (fail-open) but adds "degraded" and an X-MAILSTRIX-Degraded header with the reason: incomplete (scan budget ran out, an extraction cap such as the stream or archive-member limit stopped the walk, an extracted stream failed to scan, or an extractor crashed), error (the scan failed) or busy (no scan slot freed up in time). A log-only MAILSTRIX_SCAN_INCOMPLETE or MAILSTRIX_SCAN_DEGRADED match names it too. Treat a degraded reply with no actionable match as unknown, not clean; any actionable match in it is still a real detection. Degraded results are never cached. ICAP answers 500 instead of 204 for them. For the hardened container setup (read-only rootfs, dropped caps, Docker secret, static IPv4) see docker/docker-compose.yml⁠.

⁠Scanning without a server

The same binary scans locally — no HTTP, no token — by compiling the rules in-process. For one-off triage and pipelines:

strixd scan suspicious.doc            # one file
strixd scan /var/mail/cur             # a maildir, recursed
cat msg.eml | strixd scan             # stdin
strixd scan -json /tmp/quarantine     # machine-readable

Exit codes: 0 clean, 1 ≥1 match, 2 usage/load/read error. Other helpers share the binary (strixd help):

strixd check-rules            # compile rules, print the count, non-zero on failure (CI gate)
strixd extract suspicious.doc # show what the extractor carves (no scan)
strixd fetch-rules            # update the cached rule bundle from the release
strixd info                   # build / libyara / loaded-bundle identity

⁠Updating rules without rebuilding (fetch-rules)

Rules move faster than image rebuilds — and outside Docker you'd otherwise need yarac + a matching libyara to compile them. strixd fetch-rules downloads a prebuilt, version-matched bundle into the cache instead:

strixd fetch-rules -cache-dir /var/cache/mailstrix

The tagged binary release and the rolling rules-current bundle have separate dates. The local nightly publisher replaces the rolling bundle's assets; it does not rebuild a tagged binary. A binary's release date therefore cannot tell you when its loaded rules were generated. Check the /version fields rules_manifest.generated and rules_update.loaded_version for the active bundle when present. The published and cached versions may differ from the successfully loaded one.

It reads a small manifest first and updates only when the published version is newer; it refuses a bundle built against a different libyara, verifies the sha256, and swaps atomically (keeping one .bak). On any error the current bundle is retained; rollback errors explicitly require cache recovery. Then SIGHUP (or restart) strixd to load it, or enable daemon polling below. The bundle is published by docker/generate-rules.sh (run from cron). Point -url / MAILSTRIX_RULES_URL at a mirror if not fetching from GitHub. The URL must be https; a plain-http mirror needs an explicit opt-in (-allow-http, serve -rules-allow-http, or MAILSTRIX_RULES_ALLOW_HTTP=1). Other schemes and URLs without a host are always refused, and a redirect from https to http is refused even with the opt-in. The base URL must not contain userinfo (credentials), query parameters, or fragments; these are refused to prevent credential leaks and broken asset URL construction.

⁠Rules-manifest signature

A remote manifest must carry a valid ed25519 signature or the update is refused and the running rules stay loaded. The publisher writes a detached compiled.yac.manifest.json.sig beside the manifest: base64 of the raw 64-byte signature over the exact manifest bytes. strixd verifies it against the public key(s) compiled into the binary before reading a single manifest field, so an unsigned, truncated, tampered or untrusted-key manifest never reaches the update logic. Verification uses the Go standard library only.

Two embedded key slots (current, then a pre-published successor) allow rotation without an unsigned window: ship the build that trusts the successor first, then switch publication to it. -require-trusted only proves the key matches the pin in the publisher's checkout; it cannot speak for binaries already deployed, so the trusting build must reach clients before publication switches keys. MAILSTRIX_RULES_EXTRA_SIGNING_KEYS can add further trust anchors for an operator who signs their own mirror; it can never remove an embedded key, and there is deliberately no switch to skip verification.

A manifest already installed in the cache is not signature-checked — it is trusted on its recorded checksum against the cached bytes, as before. An existing deployment that was seeded before signing existed therefore keeps serving its bundle; it simply cannot be updated until the publisher signs.

Publishing requires MAILSTRIX_RULES_SIGNING_KEY (an ed25519 PKCS#8 PEM key) in the environment of docker/generate-rules.sh, which signs through cmd/rulessign. The key is never passed on a command line, and it must never be exposed to a pull_request-triggered CI job. The nightly cron run has no such environment, so when MAILSTRIX_RULES_SIGNING_KEY is unset the publisher falls back to /etc/myguard-build-env, where the operator installs the same key as a single base64 line MAILSTRIX_RULES_SIGNING_KEY_B64=$(base64 -w0 key.pem) in that root-owned mode-600 file (the PEM is multi-line; the build-env reader is not). An exported MAILSTRIX_RULES_SIGNING_KEY always wins. The publisher resolves and validates the key before building, so a missing or malformed one fails in seconds instead of after the rules build. The publisher signs with -require-trusted, so a key whose public half is not pinned in the binary aborts the run before any asset is uploaded rather than publishing a bundle every client would refuse. rulessign -print-public prints the base64 public key to compare against the pin, which lives in internal/rulespin — a stdlib-only package, so the signer builds on a host without libyara.

A bundle is refused when the rules actually loaded from the verified download number zero, or fall below 50% of the installed bundle's rule count (exactly 50% is accepted; a first install is refused only at zero). The manifest rules field is never trusted for this. On a refusal, a daemon poll keeps the current rules, logs the error and counts it in count_drop_refusals of the rules-update state, while strixd fetch-rules keeps the current rules, prints the error and exits 2. To deliberately ship a much smaller ruleset, set MAILSTRIX_RULES_ALLOW_COUNT_DROP=1 (default off) or pass strixd fetch-rules -allow-count-drop.

Docker images, the reference Compose service, and Debian packages enable daemon polling daily (MAILSTRIX_RULES_POLL_INTERVAL=86400 seconds). Set it explicitly to 0 to disable polling for offline installations or custom local .yar directories. Bare binaries and library configuration keep the code default 0; opt in with MAILSTRIX_RULES_POLL_INTERVAL=900 or serve -rules-poll-interval=15m. Polling requires the compiled cache actually loaded by the scanner; a cache fallback cannot silently change to remote rules. If writable cache setup fails, strixd serves the baked/local fallback and logs that automatic polling was disabled. Network and cache-lock waits share a configurable five-minute default deadline. Native libyara validation and reload finish synchronously once entered. There is one check at startup, then one per interval plus 0–20% jitter, including after failure (no retry burst). Concurrent polls coalesce and shutdown cancels and joins the worker.

The manifest checksum detects corruption but is not an origin signature because the manifest and bundle share one channel. Use an HTTPS URL whose publisher you trust; reserve plaintext HTTP overrides for loopback or an equivalently trusted private transport. Detached manifest signing remains tracked separately.