How do you set up plotman if you don’t have any buffer drives or remotes?

[erikbaender]

Plotman 10:09:47 (refresh 8s/20)  |  <P>lotting: (active) max jobs (12) <A>rchival: (not configured)
Jobs (12): [1        2        3      .4!]
Prefixes:  tmp=/mnt/tmp/00  dst=/mnt/dst (remote)
  #    plot id    k   tmp   dst    wall   phase    tmp      pid   stat      mem   user    sys     io
  0   5034ef88   32     .    01    5:11     3:6   168G   183727    RUN     5.2G   5:51   0:16   0:04
  1   7617d6e2   32     .    00    5:42     4:0   109G   181386    SLP   258.5M   6:13   0:18   0:04
  2   fd8db84e   32     .    01    6:14     4:0   109G   178896    SLP   324.6M   6:24   0:18   0:04
...
  9   0de15687   32     .    00   10:33     4:0   109G   158974    SLP   258.5M   7:26   0:22   0:05
 10   08496e6d   32     .    00   11:03     4:0   109G   156649    SLP   258.5M   7:22   0:23   0:06
 11   3345c620   32     .    00   11:33     4:0   109G   153899    SLP   258.5M   7:30   0:22   0:06
tmp   ready                       phases                         tmp   ready                       phases                            dst   plots   GBfree         inbnd phases
  .      --   3:6 4:0 4:0 4:0 4:0 4:0 4:0 4:0 4:0 4:0 4:0 4:0      .      --   3:6 4:0 4:0 4:0 4:0 4:0 4:0 4:0 4:0 4:0 4:0 4:0       00    110     0        4:0 4:0 [+2] 4:0 4:0 4:0
                                                                                                                                     01    0       11997    3:6 4:0 4:0 4:0 4:0
Archive dirs free space
<archiving not configured>
Log: 0 (<up>/<down>/<end> to scroll)

I mounted a second drive some time before the last job that would fit on the first one started, and expected plotman to schedule all new jobs to that drive as soon as there was no more space on the first one. Apparently that's not the case though, and it just keeps randomly assigning jobs to one or the other, completely clogging up the queue. Is there anything I can do to divert job 11 and all the the other ones assigned to dst/00 to dst/01?

Am I completely misunderstanding how plotman works? I’ve read the instructions like 10 times now, but those only explain the setup for archiving/rsyncing to a remote as a final destination and using local hard drives as temporary buffer. I don’t have any buffer drives, I only have a bunch of external hard drives that are meant to be the final destination.

Would hugely appreciate if somebody could explain how to set up plotman for local use.

[ericaltendorf]

Sorry, the instructions explain the buffering approach because that was the use case it was optimized for, and we haven't optimized the use case of filling your dst drives directly and sequentially. You are correct, plotman does distribute plots to the dst drives randomly, and this is exacerbated for you by the fact that plotman also assumes you're archiving off the dst drives and doesn't stop plotting when they appear full.

This is a valid complaint, but since it's already covered by at least one existing issue, I'm going to suggest converting this into a discussion to try to get workaround from other users.

One option would be to only add one dst dir to your plotman config at a time. When it is close to full, change it to point at your next empty drive and restart plotman; it will start sending plots there.

[altendky]

I guess I'm not sure. I thought I had seen comments on keybase from other people doing it. In general, I want to get to where everyone is using archiving. That is how plotman is intended to work. There's a PR to allow dst to be whichever tmp even when there are multiple tmp and also one to allow arbitrary transfer commands to be configured for the archive step. That should go a long way towards making archival more usable by all.

But, that seems pretty odd that plotman could affect the symlink command. I can't say I'm fully convinced there.

[erikbaender]

You're right, my mistake. I was not using the -n switch, it links just fine when using it.

[DroptheHammer]

@Locque Can you explain how this was setup? I'm in the same situation where I'm randomly applying plots to drives HDD01 through HDD08 and have Archive commented out...

I've got 2 NVME as TEMP doing the plotting... then 8 HDDs that I want to be where the "farming" happens and currently set as DST. I assume the symlink fills the first drive in a list and then moves on down the line until the last location listed? How is this configured in the terminal etc?

[erikbaender]

@DroptheHammer I posted the way I set up the links on the chiaforum: https://chiaforum.com/t/how-do-you-set-up-plotman-if-you-dont-have-any-buffer-drives-or-remotes/1608/2?u=zictes

Basically I only have /tmp/dst/00 in my plotman config, which is the symlink. As soon as the last plot is written to a drive, I change where it points using ln -sfn /mnt/dst/XX /mnt/dst/00 , XX being the mountpoint of the the next empty drive.
I'm managing the symlink manually, but you could write a script that periodically checks if a drive is full and then automatically adjusts the symlink.

[DroptheHammer]

thanks for the link to your Chiaforum post, that really helps.

Just to be 100% sure. I navigate to the /MNT/DST/00 Directory via terminal. In my case, this is /mnt/HDD00, which is the first of my farm/long term storage hard drives.) Using terminal I run the command ln -sfn /mnt/HDD01 /mnt/HDD00 And ensure my plotman.yaml has /mnt/HDD00 as the only DST location? Then in the Plotman.yaml, I continue to leave the archive section as disabled (commented out) while using the DST location via symlink?.

Assuming the above is correct...

Is there any difference between this (without symlink automation) and just leaving /MNT/HDD00 as the only DST entry and then changing that when the drive fills? I assume there will be an error since Plotman wont be able to write to the full drive any longer and the plots will back up on the NVME and eventually all pause (similar to when I hadn't given Plotman permission to write to HDD00) and just "hold" until I quite plotman interactive, change the DST entry to HDD01 and re-run Plotman Interactive?