exec: "ssh": executable file not found in $PATH

[ilium007]

Not sure what this one is about....

I have an autorestic command I am calling but the ssh binary is not found, presumably because of a $PATH issue:

root@pve01:~# cat /etc/proxmox_grapple.yml
production:
  job-end:
    script:
      - '/usr/local/bin/autorestic -c /root/.autorestic.yml --restic-bin /usr/local/bin/restic backup --verbose --all 2>&1 | tee -a /mnt/ext_usb_01/pve01.log'

Error in Proxmox log:

INFO:     > Executing: /usr/local/bin/restic backup --tag ar:location:ext_usb_01 /mnt/ext_usb_01/dump/
INFO:     Fatal: unable to open repository at sftp:honeyeater-autorestic:/mnt/storage0/winter/pve01/: exec: "ssh": executable file not found in $PATH

[lingfish]

Hrm, interesting, thanks for this one. I'll have to do a little digging; I don't recall my code clearing $PATH, but perhaps auto/restic does in some circumstances.

As a first test, can you just configure a grapple command to just run /usr/bin/env (or wherever it is on your machine) and paste the output back here? Feel free to wash it, only pasting PATH and perhaps anything non-sensitive and related to restic.

[ilium007]

Yeah. I've wasted hours on this. It runs independently but if called from vzdump hooks, either script or the python grapple code it fails.

[lingfish]

Are you new to restic/auto? Or been using it for ages? If so, how? Via cron? Another way to divorce the code from a shell (and hence PATH) would be a simple test cron or even at job.

[ilium007]

I've been using it for many many years! I'm very familiar with cron ENV requirements. I don't think this is the same.

[lingfish]

All I'm suggesting is another way to eliminate the shell at the start, because that'll work for a ton of different reasons.

As above, have you been using it for years then via cron?

All processes get a PATH, as set usually from PID 1... and usually, ssh would be in /usr/bin, like it is for you... very weird.

---
title: This works
---
graph LR;
  shell-->autorestic-->restic-->ssh

Loading

---
title: This doesn't
---
graph LR;
  vzdump-->autorestic-->restic --x ssh

Loading

---
title: This doesn't either
---
graph LR;
  vzdump-->proxmox-grapple-->autorestic-->restic --x ssh

Loading

[ilium007]

I've been running autorestic via systemd units and systemd timers on Debian without issue. I retired cron years ago like everyone should have done already! Its 2024!

[ilium007]

So this worked for the shell script called direct from vzdump.conf (no grapple) - re-defined $PATH and $HOME

#!/bin/bash

PVE_NODE="$(hostname)"
LOG_FILE="/mnt/ext_usb_01/$PVE_NODE.log"

export PATH=$PATH
export HOME="/root"

if [ "$1" == "job-start" ]; then
  printf "$(date) PVE hook script start ($PVE_NODE) - $(pveversion)\n" | tee $LOG_FILE
fi

if [ "$1" == "job-end" ]; then
  export
  /usr/local/bin/autorestic -c /root/.autorestic.yml --restic-bin /usr/local/bin/restic backup --verbose --all 2>&1 | tee -a $LOG_FILE
  printf "$(date) PVE hook script finish ($PVE_NODE)\n" | tee -a $LOG_FILE
fi

[lingfish]

Very odd that you have to re-export PATH.

Why are you teeing to logs, when the output of vzdump can/will be mailed? I know mail gets people fired up, but I just send mine to mailrise.

Not to mention stuff will goto the journal.

You could also just use exec lines at the top to redirect all descriptors, instead of pipes and tee.