logcrux / troubleshooting


troubleshooting

externally-managed-environment (pip)

error: externally-managed-environment

PEP 668, on Ubuntu, Debian, and Fedora: the system Python refuses a global pip install. Install into a venv instead (see install), or use uv tool install logcrux / pipx install logcrux, which avoid this automatically.

command not found: uv (after installing it)

The installer puts uv in ~/.local/bin. Add it to PATH for the current shell, or restart the shell:

export PATH="$HOME/.local/bin:$PATH"

uv installer fails with a missing tar/gzip

ERROR: need 'tar' (command not found)

The uv install script needs tar and gzip. Minimal amazonlinux:2023 and opensuse/leap base images don't ship them; install first (Fedora and the base RHEL images already include tar):

sudo dnf install -y tar gzip      # or: sudo zypper install -y tar gzip

"You require CPython 3.15 (cp315)" or "AI classification unavailable"

hint: You require CPython 3.15 (`cp315`), but we only found wheels for `onnxruntime` ...
logcrux WARNING AI classification unavailable: ...

With no suitable Python on the host, uv installs the newest one, and the runtime behind logcrux's local classification step has no build for it yet. logcrux installs anyway and every detector works; only incident naming is less precise. For the full tool, pin an interpreter:

uv tool install --reinstall --python 3.12 logcrux
logcrux --doctor

no onnxruntime wheel / musl build (Alpine)

No matching distribution found for onnxruntime>=1.18       # pip, --python 3.12
no wheels with a matching platform tag (musllinux_...)     # uv, --python 3.12

You're on Alpine or another musl-based image, which onnxruntime doesn't publish wheels for. Plain uv tool install logcrux (without --python) installs logcrux without its classification step: detection works, naming is less precise. For the full tool, use a glibc base image, or analyze the container's logs from outside it (docker logs <container> | logcrux). See install for details.

uv silently picked a musl build on a glibc distro

$ uv --version
uv 0.x.x (...-unknown-linux-musl)

Your glibc is older than 2.28 (RHEL 7, Amazon Linux 2, Ubuntu 18.04, and similar), so uv fell back to its musl binary, and no onnxruntime musl wheel exists (install may also fail trying to build onnxruntime from source with linker 'cc' not found). Check with ldd --version. Fix: use a host or image with glibc ≥2.28, or run logcrux from outside the old machine. See older & enterprise linux.

python3: command not found / Python too old

logcrux needs Python 3.11+. Rocky, AlmaLinux, and Amazon Linux 2023 default to 3.9; openSUSE Leap defaults to 3.6. Install python3.11 / python311 explicitly, or just use uv, which brings its own Python.

permission denied

Error: Permission denied: /var/log/syslog. System logs are usually readable only by root: run it with sudo ...
Skipped: 8 file(s) not readable by this user were skipped (auth.log, syslog, ...). Run with sudo to include them.
This input is journalctl's permission notice, not the system journal ...

Some logs (/var/log/secure, /var/log/messages) are root-only by default. logcrux runs with the caller's privileges and has no special powers of its own. Pick whichever fits your setup:

# Run as root for ad-hoc analysis
sudo logcrux /var/log/secure

# Least-privilege for the journal: join systemd-journal, pipe journalctl
sudo usermod -aG systemd-journal "$USER"     # then re-login
journalctl -b | logcrux

# Grant a read-only user access via ACLs
sudo setfacl -m u:loguser:r /var/log/secure

sudo logcrux ... only works if logcrux is on root's PATH. uv tool install and pipx put it in ~/.local/bin, which root doesn't have, so sudo logcrux ... fails with "command not found" even though the same command works fine unprivileged. Resolve your own logcrux and hand that path to sudo, or keep the elevation on the read instead of the tool:

sudo "$(command -v logcrux)" /var/log/secure   # resolve your logcrux, run it as root
sudo cat /var/log/secure | logcrux             # or elevate only the read

wrong parser detected

Parsed with: nginx-access (but it's actually apache)

Override it directly, and check the formats index to confirm the detection markers for your format:

logcrux /var/log/access.log --format apache-access

many lines skipped

Checked 100 events as syslog  │  2% of 5,100 lines parsed  │ ...

The detected parser isn't matching most lines, usually because the file is an unusual variant. Mixed formats are handled automatically, and lines no parser understands are still analyzed as generic text, so a low figure here means less structure, not lost lines. Check the raw content, and force a format if you know it:

head -20 /var/log/mylog.log
logcrux /var/log/mylog.log --format syslog

Container log streams need no preparation: syslog <PRI> prefixes, ANSI colour codes and the docker logs --timestamps column are stripped automatically.

no anomalies detected but you expected some

Output: CLEAN, no anomalies detected

Try, in order:

# Lower the classification confidence threshold
logcrux /var/log/syslog --threshold 0.2

# See what the analysis engines are actually finding
logcrux /var/log/syslog --verbose

# Skip baseline comparison if the state DB is new/thin
logcrux /var/log/syslog --no-baseline

Check the Checked line under the report: if it says the input was empty, or that it was journalctl's permission notice, the problem is the input, not the analysis. Make sure you are reading the log the failure is written to (Tomcat servlet errors go to logs/localhost.*.log; an OOM-killed container leaves nothing in its own output, so use kubectl describe pod or docker inspect). If the fault really is in the log, that is a miss worth reporting as a wrong verdict.

Also confirm the source is actually logging at error/warning severity, not info. Some services log events like failed logins at info level by default (MySQL, MongoDB); logcrux deliberately ignores info-level noise so a clean result is meaningful, and won't cluster what it never sees as an error. Raise the source's log verbosity if you need those events caught.

You can also loosen the statistical thresholds in config:

analysis:
  burst_multiplier: 2.0   # more sensitive
  spike_factor: 2.0

too many false positives

Output: CRITICAL on what looks like normal operation
# Raise the confidence threshold
logcrux /var/log/syslog --threshold 0.7

Or tighten the statistical thresholds:

analysis:
  burst_multiplier: 5.0   # only alert on 5x baseline
  spike_factor: 5.0

If you're comparing against a baseline, let it run for 24+ hours first. A thin or fresh baseline treats everything as anomalous relative to almost nothing.

all incidents classified as unknown

Category: UNKNOWN (0.15 confidence)

Confidence is below the classification threshold. Lower it, or confirm the local models are actually present (a shallow or partial clone can leave them missing, in which case logcrux says so plainly and still runs statistical analysis):

logcrux /var/log/syslog --threshold 0.1
ls -lh logcrux/inference/models/*/model.onnx   # dev checkout only

baseline not updating

Baseline unchanged across multiple analyses

Usually --no-baseline was passed, or the state DB isn't writable:

ls -la ~/.local/share/logcrux/
chmod 700 ~/.local/share/logcrux/
chmod 600 ~/.local/share/logcrux/state.db

# Or just reset it
rm ~/.local/share/logcrux/state.db

database locked

Error: Database is locked

Another logcrux process is writing at the same time.

pgrep logcrux
killall logcrux

# Or sidestep the state DB entirely
logcrux /var/log/syslog --no-baseline

analysis is slow on a large file

tail -n 100000 /var/log/syslog | logcrux
journalctl --since "2 hours ago" | logcrux
logcrux /var/log/syslog --last 1h

The whole input is held in memory while it is analyzed (about 1.3 KB per line). Slicing the input with tail or journalctl --since keeps both time and memory down; --last filters after parsing, so it saves less.

no output at all

$ logcrux /var/log/syslog
(nothing printed)
# Force unbuffered output and surface any hidden error
stdbuf -oL logcrux /var/log/syslog
logcrux /var/log/syslog 2>&1 | cat

unicode / decode errors

Error: UnicodeDecodeError: 'utf-8' codec can't decode byte...

The log has non-UTF-8 bytes (common on older Windows-exported logs). Convert it first:

iconv -f ISO-8859-1 -t UTF-8 /var/log/syslog > /tmp/syslog_utf8
logcrux /tmp/syslog_utf8

config file not found or not applied

logcrux checks, in order:

  1. --config PATH
  2. ~/.config/logcrux/logcrux.yaml
  3. /etc/logcrux/logcrux.yaml
  4. built-in defaults
# Confirm it's being read
logcrux /var/log/syslog --verbose 2>&1 | grep -i config

# Or force it explicitly
logcrux /var/log/syslog --config /path/to/custom.yaml

See config for the full field reference; every field is validated, so a typo'd key fails loudly at startup rather than silently keeping the default.

filing a bug report

Gather the basics and open an issue:

logcrux /path/to/log --verbose > /tmp/logcrux.log 2>&1
logcrux --doctor
uname -a

Include a small, redacted log sample, the verbose output, and what you expected versus what happened. If the verdict itself was wrong (CLEAN when something failed, an alarm on a healthy log, the wrong incident), use the wrong verdict form; otherwise open an issue on GitHub.