logcrux / troubleshooting


troubleshooting

externally-managed-environment (pip)

error: externally-managed-environment

PEP 668, on Ubuntu, Debian, and Fedora: the system Python refuses a global pip install. Install into a venv instead (see install), or use uv tool install logcrux / pipx install logcrux, which avoid this automatically.

command not found: uv (after installing it)

The installer puts uv in ~/.local/bin. Add it to PATH for the current shell, or restart the shell:

export PATH="$HOME/.local/bin:$PATH"

uv installer fails with a missing tar/gzip

ERROR: need 'tar' (command not found)

The uv install script needs tar and gzip. Minimal amazonlinux:2023 and opensuse/leap base images don't ship them; install first (Fedora and the base RHEL images already include tar):

sudo dnf install -y tar gzip      # or: sudo zypper install -y tar gzip

no onnxruntime wheel / musl build (Alpine)

No matching distribution found for onnxruntime>=1.18       # pip
no wheels with a matching platform tag (musllinux_...)     # uv

You're on Alpine or another musl-based image, which onnxruntime doesn't publish wheels for at any Python version. Not fixable in place: switch to a glibc base image, or analyze the container's logs from outside it (docker logs <container> | logcrux). See install for details.

uv silently picked a musl build on a glibc distro

$ uv --version
uv 0.x.x (...-unknown-linux-musl)

Your glibc is older than 2.28 (RHEL 7, Amazon Linux 2, Ubuntu 18.04, and similar), so uv fell back to its musl binary, and no onnxruntime musl wheel exists (install may also fail trying to build onnxruntime from source with linker 'cc' not found). Check with ldd --version. Fix: use a host or image with glibc ≥2.28, or run logcrux from outside the old machine. See older & enterprise linux.

python3: command not found / Python too old

logcrux needs Python 3.11+. Rocky, AlmaLinux, and Amazon Linux 2023 default to 3.9; openSUSE Leap defaults to 3.6. Install python3.11 / python311 explicitly, or just use uv, which brings its own Python.

permission denied

Error: Permission denied: /var/log/syslog

Some logs (/var/log/secure, /var/log/messages) are root-only by default. logcrux runs with the caller's privileges and has no special powers of its own. Pick whichever fits your setup:

# Run as root for ad-hoc analysis
sudo logcrux /var/log/secure

# Least-privilege for the journal: join systemd-journal, pipe journalctl
sudo usermod -aG systemd-journal "$USER"     # then re-login
journalctl -o short-iso | logcrux

# Grant a read-only user access via ACLs
sudo setfacl -m u:loguser:r /var/log/secure

sudo logcrux ... only works if logcrux is on root's PATH. uv tool install and pipx put it in ~/.local/bin, which root doesn't have, so sudo logcrux ... fails with "command not found" even though the same command works fine unprivileged. Resolve your own logcrux and hand that path to sudo, or keep the elevation on the read instead of the tool:

sudo "$(command -v logcrux)" /var/log/secure   # resolve your logcrux, run it as root
sudo cat /var/log/secure | logcrux             # or elevate only the read

wrong parser detected

Parsed with: nginx-access (but it's actually apache)

Override it directly, and check the formats index to confirm the detection markers for your format:

logcrux /var/log/access.log --format apache-access

many lines skipped

Parsed: 100 events, Skipped: 5000 events

The detected parser isn't matching most lines, usually because the file is a mix of formats or an unusual variant. Check the raw content, then either force a format or let the built-in safety net take over (it already does this automatically if the detected parser covers under 60% of the file):

head -20 /var/log/mylog.log
logcrux /var/log/mylog.log --format syslog

If the source is a container log stream, check for a syslog <PRI> prefix or ANSI color codes first: both block detection entirely rather than just lowering coverage. See usage, feeding logs correctly.

no anomalies detected but you expected some

Output: CLEAN, no anomalies detected

Try, in order:

# Lower the classification confidence threshold
logcrux /var/log/syslog --threshold 0.2

# See what the analysis engines are actually finding
logcrux /var/log/syslog --verbose

# Skip baseline comparison if the state DB is new/thin
logcrux /var/log/syslog --no-baseline

Also confirm the source is actually logging at error/warning severity, not info. Some services log events like failed logins at info level by default (MySQL, MongoDB); logcrux deliberately ignores info-level noise so a clean result is meaningful, and won't cluster what it never sees as an error. Raise the source's log verbosity if you need those events caught.

You can also loosen the statistical thresholds in config:

analysis:
  burst_multiplier: 2.0   # more sensitive
  spike_factor: 2.0

too many false positives

Output: CRITICAL on what looks like normal operation
# Raise the confidence threshold
logcrux /var/log/syslog --threshold 0.7

Or tighten the statistical thresholds:

analysis:
  burst_multiplier: 5.0   # only alert on 5x baseline
  spike_factor: 5.0

If you're comparing against a baseline, let it run for 24+ hours first. A thin or fresh baseline treats everything as anomalous relative to almost nothing.

all incidents classified as unknown

Category: UNKNOWN (0.15 confidence)

Confidence is below the classification threshold. Lower it, or confirm the local models are actually present (a shallow or partial clone can leave them missing, in which case logcrux says so plainly and still runs statistical analysis):

logcrux /var/log/syslog --threshold 0.1
ls -lh logcrux/inference/models/*/model.onnx   # dev checkout only

baseline not updating

Baseline unchanged across multiple analyses

Usually --no-baseline was passed, or the state DB isn't writable:

ls -la ~/.local/share/logcrux/
chmod 700 ~/.local/share/logcrux/
chmod 600 ~/.local/share/logcrux/state.db

# Or just reset it
rm ~/.local/share/logcrux/state.db

database locked

Error: Database is locked

Another logcrux process is writing at the same time.

pgrep logcrux
killall logcrux

# Or sidestep the state DB entirely
logcrux /var/log/syslog --no-baseline

analysis is slow on a large file

logcrux /var/log/syslog --last 1h
tail -n 100000 /var/log/syslog | logcrux

Both narrow the input before analysis instead of loading and processing the whole file.

no output at all

$ logcrux /var/log/syslog
(nothing printed)
# Force unbuffered output and surface any hidden error
stdbuf -oL logcrux /var/log/syslog
logcrux /var/log/syslog 2>&1 | cat

unicode / decode errors

Error: UnicodeDecodeError: 'utf-8' codec can't decode byte...

The log has non-UTF-8 bytes (common on older Windows-exported logs). Convert it first:

iconv -f ISO-8859-1 -t UTF-8 /var/log/syslog > /tmp/syslog_utf8
logcrux /tmp/syslog_utf8

config file not found or not applied

logcrux checks, in order:

  1. --config PATH
  2. ~/.config/logcrux/logcrux.yaml
  3. /etc/logcrux/logcrux.yaml
  4. built-in defaults
# Confirm it's being read
logcrux /var/log/syslog --verbose 2>&1 | grep -i config

# Or force it explicitly
logcrux /var/log/syslog --config /path/to/custom.yaml

See config for the full field reference; every field is validated, so a typo'd key fails loudly at startup rather than silently keeping the default.

filing a bug report

Gather the basics and open an issue:

logcrux /path/to/log --verbose > /tmp/logcrux.log 2>&1
logcrux --version
python --version
uname -a

Include a small log sample (first 50 lines), the verbose output, and what you expected versus what happened. Open an issue on GitHub.