diff --git a/cis-setup.html b/cis-setup.html new file mode 100644 index 0000000..585462c --- /dev/null +++ b/cis-setup.html @@ -0,0 +1,2119 @@ + + + + + + + + +CIS hardening for FSL11 + + + + + +
+
+

1. Introduction

+
+
+

These notes detail adding extra security features to Field System +Linux 11, FSL11, as advised by Center for Internet Security (CIS). +This is provided as an option for sites that might need additional +security.

+
+
+

At the time of this writing, there is no CIS security benchmark for +Debian 11, which is what FSL11 is based on. Instead, the latest +benchmark for Debian 10 was used. Although this is not technically +correct, the changes in the benchmark for Debian 10 compared to +Debian 9 were relatively minor; primarily removal of items. It seems +plausible that the Debian 10 benchmark will be adequate for Debian +11. In fact, all of the items that fail for FSL11 are either things +that are required for FS operation, items that appear to actually be +secure despite the benchmark not recognizing them as such, and one +change since Debian 10 that we worked around. However, we don’t +know what additional checks will be in the Debian 11 benchmark.

+
+
+ + + + + +
+
Note
+		 +
+

Additional steps needed to use the benchmark:

+
+
+
    +
  1. +

    Install the package default-jre with apt-get.

    +
    2. +
  2. +

    Disable the java version check in CIS-CAT.sh.

    +
    3. +
  3. +

    Change ignore.platform.mismatch to true in +misc/ciscat.properties.

    +
    4. +
+
+
+
+
+

With the exception of the partition configuration, all actions are to +be performed post-installation (see the FSL11 +Installation document). Applying the remediations, the exceptions +that are still present, all tests that failed, and problems with the +benchmark remediations are provided in separate sections below. An +appendix covers additional setup that is needed for FS operations +after the remediations have been applied.

+
+
+

This document is based on the results for the “CIS Debian Linux 10 +Benchmark v1.0.0 - Level 2 - Server”.

+
+
+
+
+

2. Applying CIS remediations

+
+
+

2.1. Installation partition configuration

+
+

During installation, be sure to create the logical volumes marked +optional in the partition setup section.

+
+
+
+

2.2. Post-installation remediations

+
+

All commands need to be run as root.

+
+
+

2.2.1. Scripted remediations

+
+

As many remediations as possible are implemented by the remediate +script. The script is intended to be run after the “Third Stage +Installation” steps in the FSL11 instructions, before any further +changes have been made to the system (however initializing and adding +other disks to the RAID can intervene).

+
+
+
+
cd /root/fsl11
+script ../remediate.txt
+./remediate
+exit
+
+
+
+ + + + + +
+
Important
+		 +This script should not be run more than once on a system. +
+
+
+ + + + + +
+
Tip
+		 +The use of the script command causes the output to be recorded +in the specified file. This can be very helpful for understanding what +went wrong if the script fails. The script itself uses the -x option +to echo the commands as they are executed to make it easy to match the +output with the commands being executed. +
+
+
+
+

2.2.2. Reboot

+
+

The system should be rebooted to make sure all the remediations have +been applied. Some aren’t enforced until a reboot.

+
+
+

After the reboot, all the CIS remediations that can applied at this +point have been completed. The next subsection describes some other +policies that can be considered.

+
+
+
+
+

2.3. CIS issues that need to be addressed

+
+

This section lists further topics related to the benchmark that should +be discussed. The items are listed by benchmark section numbers.

+
+
+

1.5.2 Ensure bootloader password is set

+
+

You may wish to create an encrypted password with +grub-mkpasswd-pbkdf2:

+
+
+
+
grub-mkpasswd-pbkdf2
+Enter password: <password>
+Reenter password: <password>
+Your PBKDF2 is <encrypted-password>
+
+
+
+

Add the following into a custom /etc/grub.d configuration file +(don’t use /etc/grub.d/00_header as it can be overwritten by a +package update):

+
+
+
+
cat <<EOF
+set superusers="<username>"
+password_pbkdf2 <username> <encrypted-password>
+EOF
+
+
+
+

If there is a requirement to be able to boot/reboot without entering +the password, edit /etc/grub.d/10_linux and add --unrestricted to the +line CLASS=

+
+
+ + + + + +
+
Important
+		 +It is strongly recommended that booting without a password +be permitted. Otherwise, if a reboot is required to continue +operations it will not be possible unless some one with the password +is available. If they aren’t available, this could lead to a safety +issue or loss of VLBI data. +
+
+
+

Example:

+
+
+
+
CLASS="--class gnu-linux --class gnu --class os --unrestricted"
+
+
+
+

Run the following commands to update the grub2 configuration and reset +the grub.cfg permissions:

+
+
+
+
update-grub
+chmod go-rwx /boot/grub/grub.cfg
+
+
+
+
+

1.8.1.2 Ensure the local login warning banner is configured properly

+
+

You may want to update /etc/issue to have a more tailored message +with sterner warnings. The message must not include use of \m, \r, +\s, \v, or references to the OS platform.

+
+
+
+

1.8.1.3 Ensure the remote login warning banner is configured properly

+
+

You may want to update /etc/issue.net to have a more tailored +message with sterner warnings. The message must not include use of +\m, \r, \s, \v, or references to the OS platform.

+
+
+
+

1.8.2 Ensure GDM login banner is configured properly

+
+

You may want to update /etc/gdm3/greeter.dconf-defaults to have a +more tailored message with sterner warnings.

+
+
+

If desired, you can remove the Debian logo from the GUI login page by +renaming the file specified for the logo option of the +[org/gnome/login-screen] section in +/etc/gdm3/greeter/dconf-defaults. For example, if appropriate, you +might use:

+
+
+
+
cd /usr/share/images/vendor-logos
+mv logo-text-version-64.png logo-text-version-64.png.bak
+
+
+
+

If desired, you can remove the Debian logo from the grub menu by +renaming the file specified for in the if clause for the +background_image file in the /etc/grub.d/05_debian_theme section +of /boot/grub/grub.cfg. For example, if appropriate, you might use:

+
+
+
+
cd /usr/share/desktop-base/homeworld-theme/grub
+mv grub-4x3.png grub-4x3.png.bak
+
+
+
+ + + + + +
+
Important
+		 +
+

Caveat Emptor! The changes below in this IMPORTANT section may not +be safe. Even if they appear to be successful, they may case problems +later. The problems may include failure of automatic updates. They may +also need to be reinstalled after updates.

+
+
+

After making any or all of these changes, it is necessary to execute:

+
+
+
+
update-grub
+
+
+
+

for them to take effect.

+
+
+
    +
  1. +

    In principle, you can remove the Debian label from the grub boot +menus by editing /etc/default/grub and inserting a line:

    +
    +
    +
    GRUB_DISTRIBTOR=FSL11
    +
    +
    +
    +

    immediately after the existing GRUB_DISTRIBUTOR=…​ line.

    +
    +
    2. +
  2. +

    In principle, you can remove the GNU/Linux label from the grub +boot menus by editing /etc/grub.d/10_linux and inserting a line:

    +
    +
    +
    OS="${GRUB_DISTRIBUTOR}"
    +
    +
    +
    +

    immediately after the existing OS="${GRUB_DISTRIBUTOR} GNU/Linux" +line.

    +
    +
    3. +
  3. +

    In principle, you can remove the Linux label from the lines +displaying kernel image files, by editing /etc/grub.d/10_linux and +globally replacing  Linux  (not the single leading and +single trailing spaces) with  FSL11  (not the single +leading and single trailing spaces).

    +
    4. +
+
+
+
+
+
+

2.2.1.4 Ensure ntp is configured

+
+

This needs the FS NTP configuration. That is more secure than the +benchmark since it uses ignore by default.

+
+
+
+

4.1.2.3 Ensure system is disabled when audit logs are full

+
+

This may not be appropriate for an operational system.

+
+
+
+

4.2.1.5 Ensure rsyslog is configured to send logs to a remote host

+
+

To set a remote log host, edit the /etc/rsyslog.conf and/or the +/etc/rsyslog.d/*.conf files and add lines like the following +(replace angle bracket items, <…​>, with your values):

+
+
+
+
<files to sent to the remote log server> action(type="omfwd" target="<FQDN or ip of loghost>" port="<port number>" protocol="tcp"
+action.resumeRetryCount="<number of re-tries>"
+queue.type="linkList" queue.size=<number of messages to queue>")
+
+
+
+

or

+
+
+
+
*.* @@<FQDN or ip of loghost>
+
+
+
+

Run the following command to reload the rsyslog configuration:

+
+
+
+
systemctl reload rsyslog
+
+
+
+
+

5.2.16 Ensure SSH Idle Timeout Interval is configured

+
+

Five minutes is too short and is not commensurate with the recommended +15 minute auto-logout interval.

+
+
+
+

5.3.1 Ensure password creation requirements are configured

+
+

Should the minimum be reduced to 12 characters?

+
+
+
+

5.3.2 Ensure lockout for failed password attemtps is enabled

+
+

The number of login failures before lock-out can cause a problem if it +is set too low. The main issue is for an operator working at odd +hours, alone, at a remote location, and dealing with multiple issue, +which might include: power failures, equipment problems, and +logistical issues. It can be a chaotic situation. Typing long and +complicated passwords in the heat of battle, particularly if they vary +between machines, can be error-prone. Being locked-out will make the +situation more difficult and may increase the amount of data that will +be lost.

+
+
+

If you find that the number of login failures before lock-out is too +small, you can increase it by increasing the value of the deny +parameter (5 in the example below, other typical parameters are +omitted and should not be changed) in:

+
+
+
/etc/pam.d/common-auth
+
+
auth required pam_faillock.so deny=5
+
+
+
+

Small integer values (20 or less) should not be a significant risk +with long and complicated passwords and a unlock time of several +minutes.

+
+
+
+

5.4.1.4 Ensure inactive password lock is 30 days or less

+
+

This is too short for developers/troubleshooters. A value of 60 +would be commensurate with the password reset interval.

+
+
+
+
+

2.4. Other policies

+
+

This subsection describes other policies beyond the CIS benchmark that +may be desirable.

+
+
+

2.4.1. TCP wrappers configuration

+
+

You may wish to configure TCP wrappers.

+
+
+
/etc/hosts.deny
+
+

Add:

+
+
+
+
ALL:ALL
+
+
+
+
+
/etc/hosts.allow
+
+

Add:

+
+
+
+
sshd:ALL
+
+
+
+

It is recommended that you further restrict sshd to specific hosts +and/or sub-domains.

+
+
+
+
+

2.4.2. Set log retention period

+
+

You may want to set the retention period of system logs by +editing /etc/logrotate.conf and/or /etc/logrotate.d/*, as +appropriate.

+
+
+
+
+
+
+

3. CIS Exceptions

+
+
+

This section addresses the tests that failed in the CIS benchmark +after all the remediations in this document were applied. The items +are listed by benchmark section numbers.

+
+
+

1.4.2 Ensure filesystem integrity is regularly checked

+
+

The AIDE system now performs a check daily and generates a report, so +this is no longer needed.

+
+
+
+

1.5.2 Ensure bootloader password is set

+
+

This must be set later by the system administrator.

+
+
+
+

2.2.2 Ensure X Window System is not installed

+
+

The X11 Window system is required for FS use.

+
+
+
+

2.2.4 Ensure CUPS is not enabled

+
+

The CUPS printing systems is required for operations.

+
+
+
+

3.5.2.5 Ensure firewall rules exist for all open ports

+
+

There is a ufw rule for Openssh (port 22), but the benchmark +doesn’t accept that. Additional openings can be added as needed.

+
+
+
+

3.5.3.7 Ensure nftables service is enabled

+
+

Although the benchmark also uses ufw, which is enabled and uses +nftables, for some reason this is not recognized.

+
+
+
+

3.5.3.8 Ensure nftables rules are permanent

+
+

Although the benchmark also uses ufw, which has permanent rules and +uses nftables, for some reason this is not recognized.

+
+
+
+

4.2.1.5 Ensure rsyslog is configured to send logs to a remote log host

+
+

A remote log server must be configured later by the system +administrator.

+
+
+
+

5.2.6 Ensure SSH X11 forwarding is disabled

+
+

Using ssh X11 forwarding is required for for remote FS operations +and testing.

+
+
+
+

5.3.2 Ensure lockout for failed password attempts is configured

+
+

The benchmark, which is for Debian 10, uses pam_tally2.so for +this. However pam_tally2.so is not available in Debian 11, having +been replaced with pam_faillock.so. The remediate script +implements the intent of the recommended pam_tally2.so configuration +with pam_faillock.so.

+
+
+ + + + + +
+
Note
+		 +To reset a locked-out user after CIS hardening, as root use +faillock --user username --reset where username is the +user account. Leave off the --reset to see what the current failure +count is. +
+
+
+
+
+
+

4. CIS Remediation problems

+
+
+

This section details problems with the recommended remediations. The +items are listed by benchmark section numbers.

+
+
+

Some problems were worked around by adding a boot time systemd +service CISfix to correct changes that occur on a reboot.

+
+
+

1.1.10 Ensure noexec option set on /var/tmp partition

+
+

Enforcing this requirement for the currently running system before all +the other remediations have been applied can interfere with execution +of apt-get install …​ to install packages needed for the +remediation. Instead, although /etc/fstab is updated in sequence, +remounting the file systm is deferred to the end.

+
+
+
+

1.4.2 Ensure filesystem integrity is regularly checked

+
+

The /etc/crontab entry that should be added is missing the user +(root) field. Additionally Debian no longer provides aide.wrapper. +However, the AIDE system now performs a check daily and generates a +report, so this is no longer needed.

+
+
+
+

1.5.1 Ensure the permissions on the bootloader are configured

+
+

The permissions are reset every time update-grub is run, e.g., for a +kernel update. Fixing them was added to the CISfix systemd +service at boot.

+
+
+
+

2.2.1.4 Ensure ntp is configured

+
+

The remediation makes it less secure. A default policy of ignore is +better.

+
+
+
+

3.3.4 Ensure suspicious packets are logged

+
+

The remediation lines added in /etc/sysctl.d/* for this issue are +not respected at boot (unlike all others). To overcome this, the +following lines are used in the CISfix systemd service at boot.

+
+
+
+
sysctl -w net.ipv4.conf.all.log_martians=1
+sysctl -w net.ipv4.conf.default.log_martians=1
+sysctl -w net.ipv4.route.flush=1
+
+
+
+
+

4.1.5 Ensure events that modify the system’s network environment are collected

+
+

The 64-bit remediation had the b64 and the b32 rules concatenated +on one line.

+
+
+
+

4.1.16 Ensure kernel module loading and unloading is collected

+
+

The 64-bit remediation was missing the b32 rule.

+
+
+
+

4.2.3 Ensure permissions on all logfiles are configured

+
+

There are two issues:

+
+
+
    +
  1. +

    The recommended remediation makes the entire directory tree +/var/log unsearchable by everyone except root. This breaks some +functionality, in particular email. As a result, the remediation was +scaled back to just the minimum required to pass the test, which was +to just set the permission on the files themselves instead changing +the directory permissions as well. This could be made more targeted. +For example to allow email use, just /var/log and /var/log/exim4 +could be made searchable.

    +
    2. +
  2. +

    The permissions for some logfiles are reset each time the system +reboots. Fixing them was added to the CISfix systemd service at +boot.

    +
    3. +
+
+
+
+

6.1.6 Ensure permissions on /etc/passwd- are configured

+
+

The permissions are reset each time the system reboots. Fixing them +was added to the CISfix systemd service at boot.

+
+
+
+

6.1.7 Ensure permissions on /etc/shadow- are configured

+
+

The permissions are reset each time the system reboots. Fixing them +was added to the CISfix systemd service at boot.

+
+
+
+

6.1.8 Ensure permissions on /etc/group- are configured

+
+

The permissions are reset each time the system reboots. Fixing them +was added to the CISfix systemd service at boot.

+
+
+
+

6.1.11 Ensure no unowned files or directories exist

+
+

After each boot, the file /var/cache/private/fwupdmgr has no owner. +Fixing that was added to the CISfix systemd service at boot.

+
+
+
+

6.1.12 Ensure no ungroup files or directories exist

+
+

After each boot, the file /var/cache/private/fwupdmgr has no group. +Fixing that was added to the CISfix systemd service at boot.

+
+
+
+
+
+

Appendix A: Additional set-up for FS operations

+
+
+

After the CIS hardening is completed, some additional set-up is needed.

+
+
+

A.1. Fix-ups

+
+

There are two issues that may need to be corrected after the CIS +hardening.

+
+
+
    +
  1. +

    Using the noexec option for /tmp causes a problem for the +package management system. The dpkg-preconfigure program places and +executes scripts on /tmp as part of package installation. The +noexec option prevents the execution of the scripts. To work around +this issue, you can exeucte:

    +
    +
    +
    cd /root/fsl11/
+./root_tmp
    +
    +
    +
    +

    The root_tmp script performs three actions:

    +
    +
    +
      +
    1. +

      Creates a one time service at boot to clean the /root/tmp directory

      +
      2. +
    2. +

      Sets dpkg-preconfigure to use /root/tmp for temporary files

      +
      3. +
    3. +

      Creates an initial /root/tmp directory

      +
      4. +
    +
    +
    +

    There may be other issues with using the noexec option for /tmp, +but we don’t have any specifics at this time.

    +
    +
    2. +
  2. +

    Sometimes the firewall (ufw) does not work properly after rebooting. +This has been noticed for remote access to gromet for met. data on +port 50001. There are no other known issues. An apparent fix for this +is to disable and re-enable the firewall. If you have this problem and +the same solution works, a one-time service at start-up can be created +to perform this action:

    +
    +
    +
    cd /root/fsl11
+./create_ufw_re-enable
    +
    +
    +
    +

    The new service will run at the next reboot. It is configured to run +after ufw has been started.

    +
    +
    3. +
+
+
+
+

A.2. Enabling user promotion to oper/prog and root

+
+

The model used in the FS assumes oper and prog accounts will be +used for operations and programming respectively. However, some +organizations may have security and auditing restrictions that mean +operators must login using their own account (possibly named with +their Agency User ID, or AUID). As the FS currently operates, users +will then need to switch to the oper or prog account after login. +Likewise, if a user is allowed to elevate to root, they will need to +do so after logging into their own account. This subsection covers how +to enable this capability. The next subsection, +Adding AUID accounts, covers how to add an AUID account. The +method described here, and in the next subsection, uses dhorsley as +an example AUID account name.

+
+
+

For oper and prog, we suggest creating two groups that can sudo +to the accounts.

+
+
+

run visudo then add at end:

+
+
+
+
%operators      ALL=(oper) NOPASSWD: ALL
+%programmers    ALL=(prog) NOPASSWD: ALL
+%programmers    ALL=(oper) NOPASSWD: ALL
+
+
+
+

To allow operators to use rotation_shutdown, refresh_secondary, +shutdown, and reboot, add (respectively):

+
+
+
+
%operators      ALL=(ALL) /usr/local/sbin/rotation_shutdown
+%operators      ALL=(ALL) /usr/local/sbin/refresh_secondary
+%operators      ALL=(ALL) /sbin/shutdown
+%operators      ALL=(ALL) /sbin/reboot
+
+
+
+

To use these commands the operators will need to enter +(respectively) from their AUID accounts:

+
+
+
+
sudo rotation_shutdown
+sudo refresh_secondary
+sudo shutdown
+sudo reboot
+
+
+
+

A password will be required. Trailing options can be used with the commands, as appropriate.

+
+
+

If the user can elevate to root, also add:

+
+
+
+
dhorsley       ALL=(root) ALL
+
+
+
+

If they don’t already exist, create the needed groups:

+
+
+
+
addgroup operators
+addgroup programmers
+
+
+
+

If they don’t already, make sure oper and prog have usable shells:

+
+
+
+
chsh -s /bin/bash oper
+chsh -s /bin/bash prog
+
+
+
+

If the accounts haven’t been disabled for login already, do so:

+
+
+
+
usermod -L desktop
+usermod -L oper
+usermod -L prog
+
+
+
+

To prevent connecting with ssh using a key, create (or add oper +and prog to an existing) DenyUsers line in /etc/ssh/sshd_config:

+
+
+ + + + + +
+
Note
+		 +If you used the CIS remediate script, you should comment out +the line: DenyGroup rtx as well. +
+
+
+
+
DenyUsers desktop oper prog
+
+
+
+

And restart sshd with:

+
+
+
+
systemctl restart sshd
+
+
+
+

Authorized users can then switch to oper with (similarly for +prog and root):

+
+
+
+
sudo -i -u oper
+
+
+
+

No password will be required (except for root).

+
+
+

To ensure X authorization works do the following (this example is for +user oper and works analogously for prog and root, but see the +note at the end of step (1) about root's Xresources:

+
+
+
    +
  1. +

    Add this to the following file:

    +
    +
    ~/.profile
    +
    +
    if ! [ -z "$XCOOKIE" ]; then
+   xauth add $XCOOKIE
+fi
+if echo $DISPLAY |grep -q localhost; then
+#   ssh from remote host with X display
+    xrdb -merge ~/.Xresources
+else
+    if ! [ -z $DISPLAY ]; then
+      if xhost|grep -q 'SI:localuser:oper'; then
+#       local X display
+        xrdb -merge ~/.Xresources
+      fi
+    else
+#     text terminal, do nothing
+      :
+    fi
+fi
    +
    +
    +
    +

    This will also set the Xresources to those of oper. (For root + only the first clause would be used since Xresources would not be + set.)

    +
    +
    2. +
  2. +

    Create the following file

    +
    +
    /usr/local/bin/oper_account
    +
    +
    #!/bin/bash
+set -e
+if echo $DISPLAY |grep -q localhost; then
+ sudo -u oper XCOOKIE="$(xauth list $DISPLAY)" -i
+else
+ if ! xhost|grep -q 'SI:localuser:oper'; then
+   xhost +SI:localuser:oper >/dev/null
+ fi
+ sudo -u oper -i
+fi
    +
    +
    +
    3. +
  3. +

    Execute:

    +
    +
    +
    chmod a+rx /usr/local/bin/oper_account
    +
    +
    +
    4. +
+
+
+

The three numbered steps above can be executed for oper, prog, and root +with:

+
+
+
+
~/fsl11/AUID/install_AUID
+
+
+
+
+

A.3. Adding AUID accounts

+
+

This subsection describes how to add AUID accounts to be used with the +ability to promote to oper, prog, and root as described in the +previous subsection. As in the previous subsection, the following +method uses dhorsley as an example AUID account name.

+
+
+
    +
  1. +

    Add any needed user accounts as appropriate:

    +
    +
    +
    adduser dhorsley --home /usr2/dhorsley
+chmod 0750 /usr2/dhorsley
    +
    +
    +
    + + + + + +
    +
    Important
    +    		 +If you are configuring a spare computer, you will need to +make sure the UID and GID for each user with a home directory on +/usr2 is the same on both computers for the system-to-system backup +of /usr2 to work properly. +
    +
    +
    + + + + + +
    +
    Note
    +    		 +
    +

    For normal operations, AUID users' home directories should be on +/usr2. However, for some maintenance accounts, it may make sense to +have the home directory some where else, typically on /home. In that +case, use these commands instead:

    +
    +
    +
    +
    adduser dhorsley
+chmod 0750 /home/dhorsley
    +
    +
    +
    +

    The step for setting the contents of the home directory below will +need to be adjusted accordingly, see NOTE.

    +
    +
    +

    The UID and GID of the account should still be kept in agreement +between the two computers.

    +
    +
    +
    +
    2. +
  2. +

    Add each user to these groups as appropriate, e.g.:

    +
    + + + + + +
    +
    Note
    +    		 +This step assumes that the operators and programmers groups +have been created as described in the previous subsection +Enabling user promotion to oper/prog and root. +
    +
    +
    +
    +
    adduser dhorsley operators
    +
    +
    +
    +

    and/or:

    +
    +
    +
    +
    adduser dhorsley programmers
    +
    +
    +
    3. +
  3. +

    If the user should be able to manage printers, use:

    +
    +
    +
    adduser dhorsley lpadmin
    +
    +
    +
    4. +
  4. +

    If the account will be used by an operator and/or programmer, the X11 +environment needs to be set-up. The following command will move an +existing /usr2/dhorsley to /usr2/dhorsley.FSCOPY and create a new +/usr2/dhorsley with a useful skeleton for use with the FS (you will + be prompted for the account name):

    +
    +
    +
    /usr2/fs/misc/auid_update
    +
    +
    +
    + + + + + +
    +
    Note
    +    		 +
    +

    NOTE: If the user’s home directory is not on /usr2, +but is for example on /home, the following commands should be used +instead:

    +
    +
    +
    +
    cd /home
+mv dhorsley dhorsley.FSCOPY
+cd /usr2/fs/st.default/auid
+find . -print|cpio -pmdu /home/dhorsley
+chown -R dhorsley.dhorsley /home/dhorsley
+chmod 0750 /home/dhorsley
    +
    +
    +
    +
    +
    5. +
  5. +

    Set default desktop

    +
    +

    To set the correct default desktop (it is remembered per user):

    +
    +
    +
    +
    cat > /var/lib/AccountsService/users/dehorsley <<EOF
+[User]
+Language=
+XSession=default
+Icon=/usr2/dehorsleyley/.face
+SystemAccount=false
+EOF
    +
    +
    +
    +

    Alternatively, if you have access to the console:

    +
    +
    +
      +
    1. +

      Press Ctrl+Alt+F1 to get to the GUI login.

      +
      2. +
    2. +

      Enter dhorsley as the Username.

      +
      3. +
    3. +

      Select the “gear” icon in the lower right-hand corner.

      +
      4. +
    4. +

      Select System X11 Default.

      +
      5. +
    5. +

      Complete logging in with the password.

      +
      6. +
    6. +

      Logout with exit.

      +
      7. +
    +
    +
    6. +
+
+
+
+

A.4. Setting hostname alias

+
+

These steps set a more user friendly alias for the computers of the +form fs1-<xx> and fs2-<xx> where <xx> is the station’s two +letter code. This provides a compact alias for local usage, even for +sites with more than one system, and makes the system identifiable for +remote users in a systematic way. Except as noted below, these steps +should be executed for both the operational and spare computers.

+
+
+
    +
  1. +

    Edit /etc/hosts and add the new aliases to the appropriate lines.

    +
    +

    If you have two computers, add the aliases for both to the file on each computer.

    +
    +
    2. +
  2. +

    Create a file /etc/hostname_alias that contains the new alias.

    +
    +
      +
    1. +

      Execute

      +
      +
      +
      cd /etc
+cp hostname hostname_alias
+chmod a+r hostname_alias
      +
      +
      +
      2. +
    2. +

      Edit the new file and change the contents to the new alias.

      +
      3. +
    +
    +
    3. +
  3. +

    Change the system’s mailname

    +
    + + + + + +
    +
    Note
    +    		 +To allow mail to mailman mail lists to work, you may need to +make a use a fake FQDN name, perhaps by appending .net to your +alias, for use in /etc/mailname and +/etc/exim4/update-exim4.conf.conf. The two files should be +consistent. +
    +
    +
    +
      +
    1. +

      Edit the file /etc/mailname and change its contents to the new +name, without a domain name unless that is required by remote mail +hosts or mail lists. If so, +Generate FQDN in HELO for outgoing mail +in the FSL11 Installation document may also be helpful.

      +
      2. +
    2. +

      Edit /etc/exim4/update-exim4.conf.conf, change the value of +dc_other_hostnames= to the new alias

      +
      3. +
    3. +

      Execute

      +
      +
      +
      update-exim4.conf
+systemctl restart exim4
      +
      +
      +
      4. +
    +
    +
    4. +
  4. +

    Use the new alias in the user prompts and xterm titles for oper, prog, and all non-system-administrator AUID accounts. In the +.bashrc file for each user to be changed:

    +
    +
      +
    1. +

      Before the if block that sets PS1 add:

      +
      +
      +
      hostalias_file=/etc/hostname_alias
+if [[ -f "$hostalias_file" ]]; then
+    hostalias=$(cat $hostalias_file)
+else
+    hostalias=$(hostname)
+fi
      +
      +
      +
      2. +
    2. +

      In the two statements setting PS1 in the if block, change the use of \h to $hostalias.

      +
      3. +
    3. +

      In the statement setting PS1 in the case block that sets the xterm window title, change the use of \h to $hostalias.

      +
      4. +
    +
    +
    5. +
  5. +

    For a spare computer only, if you have one:

    +
    +
      +
    1. +

      Update /usr/local/sbin/refresh_spare_usr2 to use the new alias of +the operational computer in the ssh line.

      +
      2. +
    2. +

      You will need to update the new alias for the operational +computer to be recognized as a known host to the root account on the +spare computer. You can do that, as root, by using ssh to +spare@operational where operational is the new alias for the +operational computer. The command will give you guidance for which +lines need to be deleted in /root/.ssh/known_hosts. After deleting +those lines, reconnect using the same ssh command and answer yes +to confirm connecting. The login will rejected because of the +forced-command setup on the operational computer. The error message +will probably not seem to make sense, but will end with a line like: +Connection to operational closed.. Still, the task of recording +the host key will have been accomplished.

      +
      3. +
    +
    +
    6. +
+
+
+
+

A.5. Installing refresh_spare_usr2 with CIS hardening

+
+

This section is useful if you have a spare computer and want it to +have a backup of your operational computer /usr2 partition. Using +this method will allow operators to run refresh_spare_user2 with +sudo. All steps must be performed as root on the specified system. +You should read all of each step or sub-step before following it.

+
+
+ + + + + +
+
Tip
+		 +Read the +refresh_spare_usr2 section of the +RAID Notes for FSL11 document for important information +on the refresh_spare_usr2 script. +
+
+
+
    +
  1. +

    On the operational system:

    +
    +
      +
    1. +

      Temporarily set sshd to allow root login:

      +
      +
        +
      1. +

        Edit /etc/ssh/sshd_config

        +
        +

        Add an uncommented line (or change an existing line) for +PermitRootLogin to set it to yes

        +
        +
        2. +
      2. +

        Restart sshd. Execute:

        +
        +
        +
        systemclt restart sshd
        +
        +
        +
        3. +
      +
      +
      2. +
    2. +

      Create spare account. Execute:

      +
      +
      +
      addgroup spare --gid 2000
+adduser spare --uid 2000 --gid 2000
      +
      +
      +
      + + + + + +
      +
      Note
      +      		 +The user’s home directory is on /home (by default), not +/usr2. +
      +
      +
      + + + + + +
      +
      Note
      +      		 +Since the spare account is unique to the operational system, +the UID and GID values of 2000 are chosen to make it easier to keep +the values in sync for other users on both systems. In other words, it +is not necessary to worry about jumping over a low value on the +spare system when values are assigned sequentially, as is the +default. If you think you might have more than 1000 users or groups, +you might want to increase the UID and GID values for the spare +account and group. +
      +
      +
      3. +
    +
    +
    2. +
  2. +

    On the spare system.

    +
    +
      +
    1. +

      Make sure the operational system is represented in the +/etc/hosts file.

      +
      +

      If it is not already there, add it. It is recommended that it be given +a simple alias for routine use.

      +
      +
      2. +
    2. +

      Install refresh_spare_usr2. Execute:

      +
      +
      +
      cd /usr/local/sbin
+cp -a /root/fsl11/RAID/refresh_spare_usr2 refresh_spare_usr2
+chown root.root refresh_spare_usr2
+chmod a+r,u+wx,go-wx refresh_spare_usr2
      +
      +
      +
      3. +
    3. +

      Customize refresh_spare_usr2, following the directions in the +comments in the script:

      +
      +
        +
      1. +

        Comment-out the lines (add leading #s):

        +
        +
        +
        echo "This script must be customized before use.  See script for details."
+exit 1
        +
        +
        +
        2. +
      2. +

        Change the operational in the line:

        +
        +
        +
        remote_node=operational
        +
        +
        +
        +

        to the alias (preferred), FQDN, or IP address of your operational +system.

        +
        +
        3. +
      3. +

        Uncomment the line for CIS hardened systems. The commented out +form is:

        +
        +
        +
        #remote_user=spare
        +
        +
        +
        4. +
      +
      +
      4. +
    4. +

      Create and copy a key for root. Execute:

      +
      + + + + + +
      +
      Caution
      +      		 +If root already has a key, you only use the second command +below, to copy it to the spare account. +
      +
      +
      + + + + + +
      +
      Tip
      +      		 +It is recommended to not set a passphrase. +
      +
      +
      +
      +
      ssh-keygen
+ssh-copy-id spare@operational
      +
      +
      +
      +

      where operational is the alias, name, or IP of your operational +system.

      +
      +
      5. +
    5. +

      Enable running the script with sudo. Use visudo to add:

      +
      +
      +
      %operators         ALL=(ALL) /usr/local/sbin/refresh_spare_usr2
      +
      +
      +
      + + + + + +
      +
      Note
      +      		 +It could be setup for a specific user (but not oper or prog +in a CIS hardened system) by replacing %operators with the user +account name. +
      +
      +
      6. +
    +
    +
    3. +
  3. +

    On the operational system:

    +
    +
      +
    1. +

      Set the spare account to only allow a forced command with ssh +by replacing the ssh-rsa at the start of the first (and only) line of +~spare/.ssh/authorized_keys line with:

      +
      +

      command="sudo --preserve-env rrsync -ro /usr2" ssh-rsa

      +
      +
      + + + + + +
      +
      Tip
      +      		 +If your spare system is registered with DNS, you can provide +some additional security by adding from="node"  (note +the trailing space) at the start of the line, where node is the +FQDN or IP address of the spare system. It may be necessary to +provide the FQDN, IP address, and/or alias of the spare system in a +comma separated list in place of node to get reliable +operation. +
      +
      +
      2. +
    2. +

      Enable the spare account to run rrsync with sudo without a +password and with passing environment variables. Use visudo to add:

      +
      +
      +
      spare          ALL=(ALL) NOPASSWD:SETENV: /usr/bin/rrsync
      +
      +
      +
      3. +
    3. +

      Lock-out the spare account from normal login (but it must have a +shell). This will disable password login, but not ssh login with +keys, for this account. Execute:

      +
      +
      +
      usermod -L spare
      +
      +
      +
      4. +
    4. +

      Disable password aging and inactivity time-out for the spare +account. Execute:

      +
      +
      +
      chage -I -1 -M 99999 spare
      +
      +
      +
      5. +
    +
    +
    4. +
+
+
+
+
+
+ + + \ No newline at end of file diff --git a/index.html b/index.html new file mode 100644 index 0000000..d5745eb --- /dev/null +++ b/index.html @@ -0,0 +1,4786 @@ + + + + + + + + +FS Linux 11 Installation Guide + + + + + +
+
+
+
+
+
+
+

1. Introduction

+
+
+

These instructions provide a complete method for system installation +and some tuning. They are not the only method for accomplishing these +goals, but have been well tested. Experts can of course use their own +means, but the farther they deviate from this model, the less support +we will be able to provide.

+
+
+

The standard configuration uses a RAID1 system with removable disks. +Normally, two disks would be in use at a given time. A third disk is +used as a back-up and rotated into use periodically. More disks can be +used for further redundancy. You can of course provide your own +back-up method and can install the system to a single disk if you do +not want to use the software RAID.

+
+
+

If you are using the RAID configuration, you may wish to review the +Recommended practices subsection +of the RAID notes for FSL11 document before installing. +However, all of the practices listed there can be implemented after +the installation steps below are complete.

+
+
+ + + + + +
+
Tip
+		 +Removable disks should be used with a carrier/receiver system +that can tolerate a large number of insertions; “bare” disks should +not be inserted repetitively. Two receivers would normally be mounted +in the computer chassis. Each disk would be in its own carrier. We can +provide a recommendation for a carrier/receiver system if you need +one. +
+
+
+

Please note that for each step in this guide, we recommend you +carefully read all the included caveats and notes as the material is +not always logically sequential, i.e., instructions may proceed +explanations that impact what you actually type.

+
+ + +++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 1. FSL distributions
FS LinuxRelease NameDebian VersionLinux kernelYear

1

(Slackware)

1.2.<x>

1994

2

bo

1.3.1

2.0.29

1997

3

hamm

2.0

2.0.34

1998

slink

2.1

2.0.36

1999

4

potato

2.2

2.2.18

2000

5

woody

3.0

2.2.20/2.4.18

2002

6

sarge

3.1

2.4.27

2005

7

etch

4.0

2.6.18

2007

8

lenny

5.0

2.6.26

2009

squeeze

6.0

2.6.32

9

wheezy

7.0

3.2.0

2014

jessie

8.0

3.16.0

10

stretch

9.0

4.9.0

2020

buster

10.0

4.19.0

11

bullseye

11.0

5.10.0

2023

+
+

The FSL11 documents follow the FS font conventions, which can be found +at: https://nvi-inc.github.io/fs/misc/font_conventions.html.

+
+
+
+
+

2. Choosing architecture and creating installation media

+
+
+

FSL11 can be configured for either the i386 or amd64 +architectures. With FSL11 it is necessary to use Field System version +10.2 or later. Those FS versions support both architectures +natively, so either may be used. The amd64 architecture is preferred +and should be used if possible (it should be unless the processor is +very old, from about 2010 or older). However, some work may be +required to port your station code from a 32-bit to a 64-bit OS. An +automatic tool has been developed to help with this, and can be +provided upon request. Usually the i386 architecture will work on +any processor, but requires use of the Legacy (or BIOS) boot mode +in most cases. The amd64 installation media will fail to boot on a +system that is 32-bit only.

+
+
+

To install Debian 11, you can either use a DVD or USB drive. The latter is +faster, and also easier if you wish to use UEFI. Directions for creating your +installation media can be found online.

+
+
+ + + + + +
+
Note
+		 +Don’t be confused by the amd64 name, this architecture supports both +AMD and Intel manufactured x86-64 processors. This includes CPU lines such as +Ryzen, Epyc, Core, and Xeon. The naming scheme dates back to when Intel had a +competing and incompatible 64-bit architecture ia64. +
+
+
+

You can install from a DVD drive, USB device, or over the network. Any revision of +11.<x> installer should work fine. +Note also that installing from DVDs as described here is +recommended mainly for sites with little to poor Internet connectivity (even +then, use of a single DVD may suffice) and the equivalent use of a “Debian +GNU/Linux 11.<x> Bullseye - Official i386/amd64 netinst CD” would suffice for +installation at most sites with good connectivity. Official images for the installer +can be found at: https://cdimage.debian.org/cdimage/release/ or alternatively, +should your hardware require non-free firmware, unofficial images for the +installer that also include all available non-free firmware can be found at: +https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/

+
+
+

The details of creating your installation media can be found in the Debian +installation guide available from: +https://www.debian.org/releases/bullseye/installmanual

+
+
+
+
+

3. Motherboard setup

+
+
+ + + + + +
+
Caution
+		 +If you are installing to a virtual machine, make sure it is +configured to have at least two cores. This is required for FS display +server support. +
+
+
+ + + + + +
+
Note
+		 +
+

Some hardware may require special procedures. For cases we know about, +instructions are provided in top-level sub-directories of the +repository, which is visible at https://github.com/nvi-inc/fsl11. In +addition to special instructions, there may be needed software/drivers +in the sub-directories. The following table lists the cases that are +currently covered. Some of these solutions may be useful for other +hardware with the same issues. If you have the listed hardware or +issue, we recommend reading the instructions before beginning the +installation.

+
+ + +++++ + + + + + + + + + + + + + + +
Table 2. Special installation instructions available
DirectoryHardwareIssue

None at this time

+
+
+
+

Modern motherboards offer two forms of booting: native UEFI or BIOS +emulation (“Legacy”). UEFI is the preferred approach. Either mode of +boot is supported by this installation guide, and you will be given +alternatives when the instructions differ.

+
+
+

Decide which boot mode you want to use and select it through the +motherboard setup menu (typically by pressing Delete during +the power-on self test, aka POST).

+
+
+

Make sure that the motherboard time is set to the current Coordinated +Universal Time, i.e., UTC, and the motherboard can boot from the +installation media.

+
+
+

While you are in the motherboard menu, make sure that hot-swapping is +enabled for the controllers of both the primary and secondary +disks. This is necessary for disk rotation and recoverable testing.

+
+
+ + + + + +
+
Tip
+		 +For UEFI, some motherboards may switch to booting to the UEFI +shell if they fail to find a hard disk that will boot. This might +happen, for example, if you attempt to boot from a blank disk. If you +become stuck booting to the UEFI shell, you may need to enter the +motherboard’s setup utility to restore booting from the hard disks. +The Boot menu may be where this is set. You may be able to disable +use of the UEFI Shell, which may eliminate this situation. +
+
+
+
+
+

4. First stage installation

+
+
+

This guide assumes that you have only one disk installed in the +machine initially even if you intend to use a RAID configuration. Use +of a single disk (for a test install, etc.) is also annotated below.

+
+
+ + + + + +
+
Note
+		 +The single dish install approach is used because it is faster +than a dual disk install. It also allows you to control when the +syncing for the second disk occurs, such as when you leave for the +evening. The setup of a second and third disk is covered in the +Setup additional disks subsection below. +
+
+
+
    +
  1. +

    Install your smallest disk in the primary slot (the one connected +to the lowest numbered SATA controller, usually 0.

    +
    + + + + + +
    +
    Caution
    +    		 +For the RAID to work seamlessly with other disks later, you +must make sure that the smallest disk of the ones available is used +for the installation. +
    +
    +
    2. +
+
+
+

4.1. Boot from the installation medium

+
+
    +
  1. +

    Connect an active network cable to your lowest numbered interface +(only). Usually it is on the left if there are two.

    +
    2. +
  2. +

    Insert/plug-in your installation media and reboot.

    +
    +

    To boot from the installation media you may need to bring up your +motherboard’s setup utility, which is typically accessed by pressing +Delete during the POST. From there you may need to access a menu +such as Save & Exit (or Boot), to select overriding to +boot with the installation media.

    +
    +
    + + + + + +
    +
    Tip
    +    		 +If the system was most recently booted from a hard disk, you may +need to boot one time with no hard disk installed for the +motherboard’s setup utility to recognize the USB drive as a valid +boot override option. If the setup utility does not recognize the +USB drive at all, it may be necessary to turn the power off, remove +the USB drive, reinsert it, and then reboot. Making the USB the first +boot device temporarily may be necessary. +
    +
    +
    3. +
+
+
+
+

4.2. Set boot options and boot installer

+
+

At the Installer boot menu:

+
+
+
    +
  1. +

    Highlight Install (or Graphical install — only the installer +interface differs — but this may not work on some video hardware)

    +
    +
      +
    • +

      UEFI: press e, then ↓ three times (vmlinuz), then End

      +
      + + + + + +
      +
      Note
      +      		 +If e doesn’t work, UEFI is not available. It may be possible to enable it in the BIOS. +
      +
      +
      • +
    • +

      BIOS: press Tab

      +
      • +
    +
    +
    2. +
  2. +

    To the end of the displayed command, add the additional options:

    +
    +
    +
    locale=en_US.UTF8 netcfg/disable_dhcp=true time/zone=UTC
    +
    +
    +
    + + + + + +
    +
    Note
    +    		 +Whilst typing a / (slash) it may automatically be changed (escaped) to + \/ (i.e. preceded by a backslash). This is normal behaviour and harmless. +
    +
    +
    3. +
  3. +

    Press:

    +
    +
      +
    • +

      UEFI: F10

      +
      • +
    • +

      BIOS: Enter

      +
      • +
    +
    +
    4. +
+
+
+ + + + + +
+
Note
+		 +You may omit the netcfg/disable_dhcp=true if you want to use DHCP to +configure the network settings of this machine, though this is not advised. +
+
+
+ + + + + +
+
Note
+		 +You can additionally use partman-partitioning/default_label=gpt if you wish +to force the use of a GPT partition table on a disk that is smaller than 2 GB, +but beware - some older BIOS versions cannot handle GPT formatted disks. +
+
+
+ + + + + +
+
Note
+		 +If you do not set a locale or set locale=C, you will be +prompted to select your language and your country. However some +applications may have problems if a UTF8 locale is not used. +
+
+
+

The installer will now boot.

+
+
+
+

4.3. Select a keyboard layout

+
+

Find your keyboard on the Keymap list, highlight it, and press +Enter. (The most common one is American English)

+
+
+

The installation media is now scanned and additional installer +components loaded.

+
+
+
+

4.4. If you are presented with a dialog asking for non-free firmware files

+
+

You may need to locate the files requested (especially if they relate +to your network or disk-drive interfaces) and place them on a USB +stick which should be inserted at this stage. If you do have the +required files select Yes, otherwise press Tab to select No +then press Enter to continue. It may well be simpler just to use +the unofficial installer images mentioned above that include all +available non-free firmware.

+
+
+
+

4.5. Configure the network

+
+
    +
  1. +

    If you are presented with a dialog asking which interface to use as +primary

    +
    +

    This is typically only shown if two or more network interfaces are found, +which might include a virtual FireWire interface in some cases. +Select the interface you require (usually eno1) and press +Enter.

    +
    +
    2. +
+
+
+

Unless you are using DHCP (which is not advisable) you will be +prompted to:

+
+
+
    +
  1. +

    Type in the required static IP address in the form xxx.xxx.xxx.xxx +(where each xxx is any integer from 0 - 255 inclusive) and press +Enter.

    +
    2. +
  2. +

    Type in the required netmask in the form 255.yyy.yyy.yyy (where each +yyy is typically 0, 64, 128, 192 or 255) and press Enter.

    +
    3. +
  3. +

    Type in the required gateway IP address in the form +xxx.xxx.xxx.xxx (where each xxx is any integer from 0 - 255 inclusive) +and press Enter.

    +
    4. +
  4. +

    Type in the required nameserver IP addresses, space separated, in +the form xxx.xxx.xxx.xxx (where each xxx is any integer from 0 - 255 +inclusive) and press Enter.

    +
    5. +
+
+
+
+

4.6. Set a hostname

+
+

Backspace over the default hostname debian and type in the name +you require (if not already retrieved via DNS), then press Enter. +Enter the required Internet Domain name (if not found) and press Enter.

+
+
+
+

4.7. Enter a suitable root password

+
+

Twice as prompted.

+
+
+
+

4.8. Setup first account

+
+

Enter Desktop User for the name of the new user +then press Enter to accept desktop as the username and enter a (real) +password twice as prompted.

+
+
+
+

4.9. Get network time

+
+

The installer now tries to set the time using NTP. If this is not +possible at your site due to your firewall etc., you may need to press +Enter to cancel this process.

+
+
+
+

4.10. Partition the disk

+
+ + + + + +
+
Note
+		 +If you are using UEFI and the disk was previously used for BIOS, you may need +to confirm forcing UEFI installation. +
+
+
+
    +
  1. +

    When prompted for a partitioning method, select Manual

    +
    2. +
+
+
+

4.10.1. Setup physical partitions

+
+
    +
  1. +

    Create a new partition table by:

    +
    +
      +
    1. +

      Select your disk, something like SCSI1 (0,0,0) (sda) - 4 TB ATA +SATA HARDDISK, and press Enter.

      +
      + + + + + +
      +
      Warning
      +      		 +Do not select your installation media. +
      +
      +
      2. +
    2. +

      The installer may warn: You have selected an entire device to +partition…. If so, select Yes. If you are prompted to delete RAID +partitions, select Yes.

      +
      3. +
    +
    +
    2. +
  2. +

    Select the (one and only entry) FREE SPACE under your disk. There +should be no RAID or LVM partitions shown.

    +
    + + + + + +
    +
    Note
    +    		 +
    +

    If other entries and/or RAID or LVM partitions are shown, you will +need to delete them before proceeding.

    +
    +
    +

    If no RAID and/or LVM partitions are shown, a possible solution may be +to delete individual partitions until you have a single entry, FREE +SPACE.

    +
    +
    +

    If that doesn’t work or RAID and/or LVM partitions are shown, you may +be able to use Guided partitioning to delete the existing +configuration (and temporarily create new partitions). In this case, +select Guided partitioning, then select Guided - use entire disk. +Then select your disk, such as listed above, do not select a RAID or +your installation media device. Then select All files in one +partition (recommended for new users). You may be prompted to confirm +deleting RAID partitions and/or removing logical volume data, which +you must do to continue. Then you should be able to continue with +selecting your disk, as above.

    +
    +
    +

    If the Guided partitioning method above doesn’t work or you have +problems later creating the RAID or LVM partitions, then other means +will be needed. There may be more complicated paths through the +partitioner that will work or, perhaps easier, you may need to +overwrite the start of the disk with a large number, say 2 GiB (but +possibly more, if that doesn’t solve the problem), of zeros.

    +
    +
    +

    Overwriting with zeros: can be implemented (for 2 +GiB) at this stage in the installer with:

    +
    +
    +
      +
    1. +

      Press Ctrl+Alt+F2 to switch to a different console.

      +
      2. +
    2. +

      Press Enter to activate the console.

      +
      3. +
    3. +

      Execute:

      +
      +
      +
      dd if=/dev/zero of=/dev/sda bs=1G count=2
+sync;sync
+reboot
      +
      +
      +
      4. +
    4. +

      When the system reboots, restart the installation.

      +
      5. +
    +
    +
    +
    +
    3. +
  3. +

    Select Create a new partition

    +
    4. +
  4. +

    Then for

    +
    +
      +
    • +

      UEFI: Enter 1GB in the size, then select Beginning of the disk.

      +
      • +
    • +

      BIOS: Enter 1MB in the size, choose Primary (rather than Logical) if asked for the partition type, then select Beginning of the disk.

      +
      • +
    +
    +
    5. +
  5. +

    Then for

    +
    +
      +
    • +

      UEFI: Select Use as then select EFI System Partition

      +
      • +
    • +

      BIOS: Select Use as then select Reserved BIOS boot area, or alternatively do not use the partition if the former option is not available.

      +
      • +
    +
    +
    6. +
  6. +

    Now select Done setting up the partition.

    +
    7. +
  7. +

    Next select the FREE SPACE and Create a new partition again.

    +
    + + + + + +
    +
    Note
    +    		 +You may see a small 1MB FREE SPACE at the start of the disk. This is +fine, just be sure to choose the large FREE SPACE at the end of the disk. +
    +
    +
    8. +
  8. +

    This time choose the whole amount of free space (the default) and choose Primary for the partition type if asked.

    +
    9. +
  9. +

    Select Use as, then select physical volume for RAID, then Done +setting up the partition

    +
    + + + + + +
    +
    Note
    +    		 +If you physically only have one disk bay and wish to construct a FSL11 test-bed, +it is possible to avoid using the software RAID layer entirely. Simply select Use as, then select physical volume for LVM +for this partition instead and skip ahead to Setup Logical Volume Manager (LVM) below. +However, please note that a single disk setup is not recommended for any operational system. +
    +
    +
    10. +
+
+
+
+

4.10.2. Setup RAID

+
+
    +
  1. +

    Select Configure software RAID. Then select Yes to write the +changes to the disk.

    +
    2. +
  2. +

    Select Create MD device, choose RAID1 and use 2 as the +number of devices and 0 as the number of spares.

    +
    3. +
  3. +

    Despite the fact that the instructions say you must select exactly +two partitions, select only one. Select the RAID partition you just +created by pressing Space. This should be /dev/sda2. Then +press Enter to continue. Select yes if prompted to write +changes to the disk.

    +
    + + + + + +
    +
    Note
    +    		 +If the newly created RAID partition doesn’t appear as an option, +you may need to use the method of Overwriting with zeros in +the Setup physical partitions step above. +
    +
    +
    4. +
  4. +

    Select Finish.

    +
    5. +
  5. +

    Back in partitioning, select the partition #1 (with no designated use) under +RAID1 device #0 and press Enter

    +
    + + + + + +
    +
    Note
    +    		 +If that partition appears immediately after being created +already having a designated use, perhaps lvm, you may need to use +the method of Overwriting with zeros in the +Setup physical partitions sub-step above. +
    +
    +
    6. +
  6. +

    Select Use as, then select physical volume for LVM, then Done +setting up the partition

    +
    7. +
+
+
+
+

4.10.3. Setup Logical Volume Manager (LVM)

+
+
    +
  1. +

    Now choose Configure the Logical Volume Manager and select Yes +if prompted to write the changes to the disk and keep the current +layout and configure LVM.

    +
    2. +
  2. +

    Choose Create volume group

    +
    3. +
  3. +

    Enter a name appropriate for the machine and group, e.g., vg0, and press Enter

    +
    4. +
  4. +

    Select the raid device md0 (or sda2 if not using RAID) by pressing Space, then press Enter +to continue

    +
    5. +
  5. +

    For each item in the following table run Create logical volume, +select your volume group and assign the corresponding name. Those +marked with * are optional unless you are applying CIS hardening.

    + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Table 3. Logical volumes
    Mount pointLV nameSize

    1

    /var/log/audit

    audit *

    4 G

    2

    /boot

    boot

    1 G

    3

    /home

    home

    4 G

    4

    /var/log

    log *

    4 G

    5

    /

    root

    50 G

    6

    (swap)

    swap

    8 G

    7

    /tmp

    tmp

    50 G

    8

    /var

    var *

    8 G

    9

    /var/tmp

    vartmp *

    8 G

    10

    /usr2

    usr2

    remaining disk space less ~100 GB

    +
    6. +
  6. +

    In the LVM configuration window, select Finish

    +
    7. +
  7. +

    Then for each logical volume in the table except swap, do the following:

    +
    +
      +
    1. +

      Select the partition (e.g., #1) for each LV name (and press Enter)

      +
      2. +
    2. +

      Select Use as and press Enter then select Ext4 journaling file system

      +
      3. +
    3. +

      Select Mount point, press Enter, then select the appropriate mount point from the list or use Enter manually if not there.

      +
      4. +
    4. +

      Select Done setting up this partition

      +
      5. +
    +
    +
    8. +
  8. +

    For the swap logical volume, select Use as then select swap area, followed by Done setting up this partition

    +
    9. +
  9. +

    Back in the partition screen, select Finish partitioning and write changes to +the disk and select Yes to write the changes. For big disks, it may take +a little time to create the ext4 file systems.

    +
    10. +
+
+
+

The Debian base system is now installed from the installation media, which +usually only takes a few minutes.

+
+
+
+
+

4.11. Configure the package manager

+
+

If you started from a netinst CD image, the installer now assumes +you will install only from the network, and jumps straight to the +Choose your Debian archive mirror country part of the +dialogue as detailed below.

+
+
+

If you are using DVD installer you will be prompted to scan additional DVDs. +Scanning the additional DVDs (and obtaining copies of them in the +first place) is entirely optional, and is only useful if you don’t have a +reliable network connection to a suitable Debian mirror and hence would +prefer not to download packages you could get from the DVD.

+
+
+ + + + + +
+
Note
+		 +If you do want to use a mirror in the future, it is better not +to scan any DVDs at this stage and to scan them later during Stage 2 +using apt-cdrom. +
+
+
+

For each additional DVD you wish to scan, insert it in the drive, select +Yes and press Enter to perform the scan (which takes a while.)

+
+
+

(If you are using DVDs, and are prompted to insert another DVD, you +will need to use eject /dev/cdrom from another virtual console to do this)

+
+
+

Select No and press Enter to continue once you are done. +If prompted, insert the “Debian GNU/Linux 11.<x> Bullseye - Official i386/amd64 +Binary-1 DVD” back into the DVD-ROM drive and press Enter.

+
+
+ + + + + +
+
Warning
+		 +If you do scan additional DVDs, the following useful dialogue +which allows you to select a suitable network mirror from a country-based +list may be suppressed. +
+
+
+

Select Yes and press Enter to use a network mirror (unless you +have inadequate Internet access - but then you must scan all DVDs.)

+
+
+

Choose your Debian archive mirror country: +Select from the list if available and press Enter. (If your +country is not available choose the country nearest to you in a +network connectivity sense.)

+
+
+

Select the fastest Debian mirror from those available.

+
+
+ + + + + +
+
Tip
+		 +The new deb.debian.org mirror is a good choice for most +sites as it uses DNS to find a local mirror. +
+
+
+

Enter any necessary HTTP proxy information (usually left blank).

+
+
+

Software is downloaded briefly.

+
+
+
+

4.12. Do not participate in popularity-contest

+
+

When prompted to join the popularity-contest, select No and press Enter

+
+
+
+

4.13. Choose your packages

+
+

When prompted to choose packages, select SSH server by moving to +that row with the arrow keys and pressing Space on it (unless +you don’t want it).

+
+
+ + + + + +
+
Tip
+		 +If you have a small disks and are worried about space, then you can +also press Space on Desktop Environment to unselect it (which may +then change the dialogue presented below). +
+
+
+

Finally press, Enter to install the standard system.

+
+
+

The Debian standard system is now installed from the installation media plus any +updates from the network mirror and/or security.debian.org site if they can be +reached.

+
+
+

This can take a while, up to one and a half hours or more.

+
+
+
+

4.14. Install the GRUB bootloader (BIOS boot only)

+
+ + + + + +
+
Note
+		 +With UEFI boot, you will not be presented with this option; GRUB will automatically be +installed to the first ESP partition. +
+
+
+

At Install GRUB to Master Boot Record select yes then select /dev/sda

+
+
+

When prompted, press Enter to install to the master boot record.

+
+
+
+

4.15. Disable Wayland (optional)

+
+

This step should only be needed if your CPU does not include a GPU and +you do not have an add-on graphics card. In that case, you are using +the motherboard graphics support. Disabling Wayland is known +specifically to be necessary for the X11SCA-F motherboard, which +uses the AST2500 graphics chip. If you don’t know that you need to +disable Wayland, we recommend that you initially leave it enabled. +Whether your choice works or not should be evident when you start the +Second stage installation step below. The console may be very +difficult, even impossible, to work with. In that case, please see the +Wayland recovery NOTE below.

+
+
+

To disable Wayland:

+
+
+ + + + + +
+
Tip
+		 +These instructions step can be executed when the installation +stops for input in the next step, Remove installation media. +
+
+
+
    +
  1. +

    Press Ctrl+Alt+F2 to switch to a different console.

    +
    2. +
  2. +

    Press Enter to activate the console.

    +
    3. +
  3. +

    Edit /target/etc/gdm3/daemon.config, uncomment Wayland=False, +and save the file.

    +
    +

    The only editor available at this point may be nano.

    +
    +
    4. +
  4. +

    Execute:

    +
    +
    +
    sync;sync
+exit
    +
    +
    +
    5. +
  5. +

    Press Ctrl+Alt+F1 to return to the Installer dialog.

    +
    6. +
+
+
+ + + + + +
+
Note
+		 +
+

Wayland recovery: If you find +you have made the wrong choice, there are at least three possible ways +to recover:

+
+
+
    +
  1. +

    If the console is marginally usable, you may be able to login on a +text console to adjust the contents of /etc/gdm3/daemon.config as +needed, then execute:

    +
    +
    +
    systemctl restart gmd3
    +
    +
    +
    2. +
  2. +

    Use the procedure in the Rescue Mode appendix and adjust the +contents of /etc/gdm3/daemon.config as needed.

    +
    3. +
  3. +

    Reinstall from scratch and make the opposite choice.

    +
    4. +
+
+
+
+
+
+

4.16. Remove installation media

+
+

Remove the DVD from the DVD-ROM drive (it should be auto-ejected), or +unplug the USB drive, and press Enter to reboot into the newly +installed system.

+
+
+ + + + + +
+
Tip
+		 +It would generally be wise to disable booting from DVD-ROM and +floppy i.e., anything other than the hard drive, in the BIOS just in +case someone leaves something nasty in the machine’s removable drives +by mistake. +
+
+
+
+
+
+

5. Second stage installation

+
+
+

You should now have booted to your new OS.

+
+
+

5.1. Login as root

+
+ + + + + +
+
Tip
+		 +Versions before Debian 9 ran X11 on virtual console 7. As of +Debian 9, the graphical environment login is on virtual console 1. +Each login there for a different user creates a session on the next +unused virtual console. +
+
+
+

Switch to Virtual Console 2, by pressing Ctrl+Alt+F2.

+
+
+

Enter root and press Enter, then enter the root password you set +earlier.

+
+
+
+

5.2. Remove the dummy Desktop User (optional)

+
+

Unless you want an account that is set up to use the default desktop +environment, delete the desktop user with:

+
+
+
+
deluser --remove-home desktop
+
+
+
+ + + + + +
+
Note
+		 +If you do keep this account, you will not be able to run the FS from +it unless you add this account into the additional hardware access groups +such as is done for oper and prog by fsadapt. +
+
+
+
+

5.3. Setup HTTP proxy for APT (optional)

+
+

Should you wish to make APT use an HTTP proxy for downloads, +create the new file /etc/apt/apt.conf.d/00proxies using vi containing:

+
+
+
+
ACQUIRE::http::Proxy "http://proxy.some.where:8080/";
+
+
+
+

to use a proxy proxy.some.where at port 8080 for example.

+
+
+
+

5.4. Edit /etc/apt/sources.list

+
+

Using your favourite text editor, eg vi, and comment out all cdrom +entries (unless you don’t have a decent Internet connection and need +to use DVDs, whereupon the dialogue presented below may differ) and +check you have the equivalent of the following entries towards the top +of the file, adding in contrib and/or non-free as needed:

+
+
+
+
deb http://deb.debian.org/debian/ bullseye main contrib non-free
+deb-src http://deb.debian.org/debian/ bullseye main contrib non-free
+
+
+
+

and likewise the equivalent of the following entries towards the +bottom of the file, again adding in contrib and/or non-free as +needed:

+
+
+
+
deb http://deb.debian.org/debian/ bullseye-updates main contrib non-free
+deb-src http://deb.debian.org/debian/ bullseye-updates main contrib non-free
+
+
+
+

(where you can use any suitable mirror instead of deb.debian.org)

+
+
+

Also add contrib and/or non-free to the lines referring to the +security.debian.org mirror in the middle of the file.

+
+
+ + + + + +
+
Warning
+		 +you MUST use bullseye and NOT stable for the +distribution in all these entries (but CD/DVD entries might use +unstable.) +
+
+
+
+

5.5. Update APT’s list of packages

+
+ + + + + +
+
Tip
+		 +Recent versions of Debian have the apt program, which gives a more + user-friendly interface to the package manager than apt-get. We + generally use apt-get except for applying updates. +
+
+
+

Next tell APT to update its internal source list of packages using

+
+
+
+
apt-get update
+
+
+
+ + + + + +
+
Note
+		 +It is also possible to add additional DVDs at this stage using the +apt-cdrom add command. +
+
+
+
+

5.6. Download the FS Linux 11 package selections

+
+
    +
  1. +

    Install git and dselect

    +
    +
    +
    apt-get install git dselect
    +
    +
    +
    2. +
  2. +

    Update dselect's package lists

    +
    +
    +
    dselect update
    +
    +
    +
    3. +
  3. +

    Get the selections by downloading this repository:

    +
    +
    +
    cd /root
+git clone https://github.com/nvi-inc/fsl11
+cd fsl11
    +
    +
    +
    4. +
  4. +

    Feed the package selections into dpkg using the command, for amd64

    +
    +
    +
    dpkg --set-selections < selections/fsl11_amd64.selections
    +
    +
    +
    +

    or, for i386

    +
    +
    +
    +
    dpkg --set-selections < selections/fsl11_i386.selections
    +
    +
    +
    5. +
  5. +

    Start the additional package installation with

    +
    +
    +
    apt-get dselect-upgrade
    +
    +
    +
    +

    then press Enter to confirm any updating of installed packages +(where you have an Internet connection) and the installation of +currently ~191 new packages (downloading ~185 MB from the Internet +and/or DVDs) for amd64 with UEFI — probably different for i386 +and/or BIOS — unless you did not select the Desktop or added other +tasks earlier.

    +
    +
    +

    Downloading commences for up to half an hour (depending on your Internet +access and the exact revision of DVDs used).

    +
    +
    +

    Installation runs to completion.

    +
    +
    6. +
+
+
+
+

5.7. Clean up the APT download directory

+
+

So that the update mechanism will work correctly, run

+
+
+
+
apt-get clean
+
+
+
+
+
+
+

6. Third stage installation

+
+
+

6.1. fsadapt

+
+

In the /root/fsl11 directory, start fsadapt with

+
+
+
+
./fsadapt
+
+
+
+

6.1.1. FS Adaptation: Modifications (Window 1)

+
+

Using the arrow keys and Space make your selections and press Enter.

+
+
+
    +
  • +

    If you are not using a GPIB board or USB dongle, you can deselect +the GPIB option.

    +
    • +
  • +

    If you are using the RAID configuration, you must not deselect +the mdinc option.

    +
    • +
+
+
+
+

6.1.2. FS Adaptation: Setup (Window 2)

+
+

All of the steps in Window 2 need to be done once (even if you do not +intend to use the serial ports) with the exception of sshkeys which +can be used to generate new SSH keys if required. +If you did not select the GPIB option in the previous page deselect the +two related options on this page (but do not deselect set_perms as it +is always required). Otherwise, simply press Enter with the OK +selected to continue.

+
+
+ + + + + +
+
Note
+		 +The updates option relies on email to root being re-directed +to some mailbox that will be read regularly, so make sure you set that +up and test it as well (see the Configure e-mail section in the +Additional Setup Items appendix). The installer sets it up to go +the desktop account by default which would definitely be a problem +if you have removed that! +
+
+
+
+

6.1.3. GPIB driver configuration (optional)

+
+

On the /etc/gpib.conf screen, use the up/down arrow keys to select the +required GPIB controller and press Enter on OK to continue.

+
+
+
+

6.1.4. Serial port configuration

+
+

On the /etc/default/grub: serial port configuration screen +up/down arrow keys to select the required RS232 serial card +(or None if you don’t have one) and press Enter on OK +to continue.

+
+
+
+

6.1.5. FS Adaptation: Settings (Window 3)

+
+

On Window 3 you can choose to modify the email or network settings if required. +Simply press Enter on OK to continue.

+
+
+
+

6.1.6. FS Adaptation: Network Services (Window 4)

+
+

The Window 4 will show what services are enabled. Use the up/down +arrows and Space to select secure and press Enter on +OK. Thereafter use the up/down arrows and Space to select +those services you actually need. If you need printing, you will need +to select netipp (remote access to this can be blocked by + configuring ufw with either not explicitly allowing or instead + denying the CUPS service). Press Enter on OK to set them +up and finish with fsadapt.

+
+
+

Note that the fsadapt script can be re-run at a later date should you need to +change the adaptations.

+
+
+
+
+

6.2. Set passwords

+
+

Set passwords for the oper and prog accounts with:

+
+
+
+
passwd oper
+passwd prog
+
+
+
+

entering the passwords twice as prompted.

+
+
+
+

6.3. Install tools for RAID (optional)

+
+

You can install some useful tools for working with the RAID, if you’re actually using it, with:

+
+
+
+
~/fsl11/RAID/install_tools
+
+
+
+

The rest of this document assumes the first three of these tools have +been installed. The six tools are:

+
+
+
    +
  • +

    mdstat — for all users — check on the RAID status

    +
    • +
  • +

    refresh_secondary — for root — refresh a secondary disk that +is from the same RAID

    +
    • +
  • +

    blank_secondary — for root — initialize a secondary disk, +must be used with extreme care

    +
    • +
  • +

    rotation_shutdown — for root — shutdown the system if it is +safe to rotate disks

    +
    • +
  • +

    drop_primary — for root — deliberately drop the primary disk +out of the RAID for use as a backup

    +
    • +
  • +

    recover_raid — for root — re-add a disk that fell out of (or +was removed from) the RAID back into it

    +
    • +
+
+
+ + + + + +
+
Tip
+		 +More information about RAID operation can be found in the +RAID notes for FSL11 document. +
+
+
+
+

6.4. Download the Field System

+
+
+
 cd /usr2
+ git clone https://github.com/nvi-inc/fs fs-git
+ cd /usr2/fs-git
+ git checkout -q tag
+
+
+
+

where tag is the latest available release, 10.2.0 or later.

+
+
+ + + + + +
+
Tip
+		 +
+

At the time of this writing, 10.2.0 has not been officially +released. The tag for the latest beta version is currently +10.2.0-beta2. That or a later tagged beta pre-release, should +suffice for an initial installation. However they should not be used +for operations. You should update to 10.2.0 as soon as it is +available. If 10.2.0 has not been officially released, you should +ignore the IMPORTANT block immediately below.

+
+
+

You can find the most recent beta release of 10.2 at:

+
+
+

https://github.com/nvi-inc/fs/releases

+
+
+

If none are available, you can find the most recent tagged alpha +version of 10.2 at:

+
+
+

https://github.com/nvi-inc/fs/tags

+
+
+
+
+ + + + + +
+
Important
+		 +
+

You should install the latest official release. To find it, go to:

+
+
+

https://github.com/nvi-inc/fs/releases

+
+
+

You should probably use the most recent feature release (ending in +.0 with no trailing -<string>, e.g., 10.2.0. However, if there +is a more recent patch release (not ending .0) for the most recent +feature release, you should use the most recent patch release. For +example, if 10.2.0 is the most recent feature release and there are +corresponding patch releases, 10.2.1 and 10.2.2, then the last +one, ending .2, is probably the best choice.

+
+
+
+
+
+

6.5. Run FS install script

+
+

This will set the /usr2/fs link, set /usr2/fs-git permissions, and +install default copies of all the FS related directories.

+
+
+
+
make install
+
+
+
+

and enter y to confirm installation.

+
+
+
+

6.6. Make the FS

+
+

The FS must always be compiled as prog.

+
+
+ + + + + +
+
Warning
+		 +Make sure you log-out as root, and log-in again as prog. +
+
+
+
+
cd /usr2/fs
+make >& /dev/null
+
+
+
+

then

+
+
+
+
make -s
+
+
+
+

to confirm that everything compiled correctly (no news is good news).

+
+
+
+

6.7. Reboot the new system

+
+

Remove any DVD from the machine and restart the machine using +reboot as root or Ctrl+Alt+Del whilst watching that +everything starts up smoothly.

+
+
+
+
+
+

7. Fourth stage Installation

+
+
+

7.1. Setup additional disks

+
+

If your are using a RAID, follow the steps in this subsection to setup +the second and third disks.

+
+
+ + + + + +
+
Note
+		 +Additional disks should be at least as large as the disk already +in use. +
+
+
+ + + + + +
+
Note
+		 +You will need to have hot-swapping enabled in your motherboard’s +setup menu, at least for the controller for the secondary disk (it +should also be enabled for the primary). +
+
+
+ + + + + +
+
Note
+		 +This subsection assumes you have installed the RAID tools +according to the Install tools for RAID (optional) subsection +above. +
+
+
+
    +
  1. +

    If you have a second disk (secondary) in the RAID:

    +
    +
      +
    1. +

      Shut the system down with the rotation_shutdown command.

      +
      +

      This command will check the status of the RAID and proceed to shutting +down only if the RAID is synced. There are three errors that can +prevent shutting down: (i) if the FS is running, you should terminate +it before trying again; (ii) if the RAID is recovering, you will +need to wait until the recovery is finished before shutting down, you +can check the progress with the mdstat command; and (iii) if the +RAID is degraded, seek expert advice.

      +
      +
      2. +
    2. +

      Remove the disk in the primary slot and place it on the shelf, +labelled appropriately as the shelf disk for this system with the +date.

      +
      3. +
    3. +

      Move the disk in the secondary slot to the primary slot.

      +
      4. +
    +
    +
    2. +
  2. +

    Initialize the new disk

    +
    + + + + + +
    +
    Important
    +    		 +Do not initialize a disk unless you are sure there is no +data on it that you need to preserve. +
    +
    +
    +

    For the first time use of an additional disk with a new install, the +disk should be initialized to make sure it has no existing structure. +This should be done even if the disk has been used in a different FS +computer or a previous install on this computer.

    +
    +
    +
      +
    1. +

      Boot with just the primary disk installed.

      +
      + + + + + +
      +
      Tip
      +      		 +If your system is already running with no second disk +(secondary) installed, you can skip rebooting. +
      +
      +
      2. +
    2. +

      Use the script:

      +
      +
      +
      blank_secondary
      +
      +
      +
      +

      The script will wait for the new disk to be turned on. Insert a new +disk in the secondary slot. The secondary slot is the one connected +to second lowest numbered SATA controller, usually 1. Turn the key +to turn the disk on. There will be a prompt asking if wish to proceed. +If it is a new disk or you are sure it safe to erase this disk, answer +y. If you are unsure about this or otherwise need to abort, answer +n.

      +
      +
      3. +
    +
    +
    3. +
  3. +

    Refresh the now blank secondary disk

    +
    +

    Run the script:

    +
    +
    +
    +
    refresh_secondary
    +
    +
    +
    +

    Once you reach the message that you can check on the recovery with +mdstat , you can resume using the computer as usual. You can safely +reboot at this point, if it is needed; just don’t remove either disk +until the recovery is finished.

    +
    +
    +

    You can check the progress of the recovery with:

    +
    +
    +
    +
    mdstat
    +
    +
    +
    +

    When the recovery is complete, you can repeat the process of this +entire subsection, Setup additional disks, to initialize another +disk.

    +
    +
    4. +
+
+
+
+
+
+

8. Post install

+
+
+

The FS on your newly installed system should now be ready to be +customized for your site’s requirements by tailoring the control files +in /usr2/control and adding suitable station specific software to +/usr2/st. See the files in the /usr2/fs/st.default/st-0.0.0 +directory for starter versions of the latter.

+
+
+

Please refer to the appendix Additional Setup Items for OS +customizations that you may find useful.

+
+
+
+
+

Appendix A: Additional Setup Items

+
+
+

This appendix covers several customizations that may be helpful +depending on the requirements for a system. It serves as a reference +for how to make these changes, but can also be helpful as a checklist +when setting up a new system. All actions in this section require +root permissions.

+
+
+

A.1. Additional security and CIS Benchmarks

+
+

For stations that wish to conform to the additional security +recommendations of the Center for Internet Security (CIS), move on to +the CIS hardening FSL11 document.

+
+
+

A.1.1. Alternate hardening

+
+

If you don’t want the complete CIS hardening, which creates some +inconveniences and is only required in certain environments, you may +still be interested in applying a subset of the remediations. You can +pick and choose those from the CIS hardening FSL11 +document and its script.

+
+
+

A useful minimum set of features to apply would be to install ufw +and block everything except ssh and further restrict ssh access with +TCP Wrappers.

+
+
+
A.1.1.1. ufw setup
+
+

To install and configure ufw to only allow ssh for incoming +connections, use the commands:

+
+
+
+
apt-get -y install ufw
+ufw allow OpenSSH
+ufw --force enable
+
+
+
+

Addition setup for ufw is covered below in the +More firewall rules subsection.

+
+
+
+
A.1.1.2. TCP Wrappers setup
+
+

A base setup for TCP Wrappers is

+
+
+
/etc/hosts.deny
+
+
ALL:ALL
+
+
+
+
/etc/hosts.allow
+
+
sshd:ALL
+
+
+
+

It is recommend that you further restrict sshd by using specific +hosts and/or sub-domains instead of ALL. Please use +man hosts_access for more information about configuring TCP +Wrappers

+
+
+
+
A.1.1.3. More firewall rules
+
+

The following tersely summarizes some ufw settings that may be +useful:

+
+
+
+
#SSH
+ufw allow OpenSSH
+#NTP
+ufw allow ntp
+#remote access to metserver (or gromet) on port 50001
+ufw allow 50001
+#anywhere from subnet
+ufw allow from 192.168.4.0/24
+#RDBE multicast to addresses from subnet
+ufw allow in proto udp to 239.0.2.0/24 from 192.168.4.0/24
+#? RDBE multicast to group from subnet ?
+#ufw allow in proto igmp to 239.0.2.0/24 from 192.168.4.0/24
+
+
+
+
+
+
+

A.2. Customize root’s .bashrc file

+
+

There are a few changes you should consider for root's .bashrc file.

+
+
+
    +
  1. +

    If you have applied the CIS remediations, you should consider +uncommenting the line that sets the umask to 022. The remediations +set it to 027 in /etc/profile, which may cause problems with +routinely created files, including some in this section covering optional changes.

    +
    2. +
  2. +

    Uncomment the the alias commands that add the -i option to the +commands cp, mv, and rm as the default. This can help avoid +some careless errors.

    +
    3. +
  3. +

    Add the command set -o noclobber to avoid accidently overwriting +existing files with I/O redirection. Other options to consider setting +are physical and ignoreeof.

    +
    4. +
+
+
+
+

A.3. Create root’s .inputrc file

+
+

The readline package is used by bash, and other programs, to +maintain a history of commands that can be edited and then +re-executed. By default, it will retain edits of history entries that +have not been re-executed. This makes the unedited history entries +more difficult to locate and re-execute. Retaining the un-executed +edits can be disabled for root by creating the file:

+
+
+
/root/.inputrc
+
+
$include /etc/inputrc
+set revert-all-at-newline on
+
+
+
+

The $include /etc/inputrc line preserves the other system wide +readline defaults.

+
+
+ + + + + +
+
Note
+		 +The standard fresh FS installation creates this file for the +oper and prog (and AUID) accounts. +
+
+
+
+

A.4. Setup /etc/hosts

+
+

You may want to add more hosts to the /etc/hosts, especially if do +not have DNS. This will allow you to give a short alias to use when +referring to other local machines. Even if you have DNS, you may wish +to add additional aliases for your local hosts.

+
+
+

For use with ntpq -p, is recommended that you use a short alias as +the canonical name (the first one after the IP address) for other +local machines (and possibly remote ones as well). This will make the +ntpq output easier to understand, particularly if the canonical +names of the local machines only differ at the end of their names. +That may make the differences hard to see given the short field +available for the remote node ID in the ntpq output.

+
+
+
+

A.5. Stabilize network configuration

+
+

This subsection requires using nm-connection-editor on a graphic +display (nmtui may be an option on a text terminal, but it has not +been fully verified). You may need to be root or desktop to do +this. All the subsections below assume you are in the program and have +sufficient permissions.

+
+
+ + + + + +
+
Note
+		 +If you someday move the disks to a computer with a different +mainboard model, the device names of the network interfaces may +change. If that happens, you will need to reselect the names as +described in the sub-steps below. This should not be necessary if the +other computer uses the same mainboard. +
+
+
+

A.5.1. Make the connection always appear on the same interface regardless of the MAC address.

+
+

This is useful both to make the connection appear on only one +interface and/or make it the same interface if the computer (or NIC) +is changed.

+
+
+
    +
  1. +

    Select your connection and click the “gear” icon.

    +
    2. +
  2. +

    Select the Ethernet tab.

    +
    3. +
  3. +

    Use the drop-down for the Device field to select your device +(typically eno1 with the MAC address in parentheses). Then edit the +field to just list the name of the interface (typically eno1) by +removing the MAC address in parentheses.

    +
    4. +
  4. +

    You may want to also set the IPv6 Settings to use Method: +Disabled.

    +
    5. +
  5. +

    Click Save.

    +
    6. +
  6. +

    Close the window by pressing Esc (while the focus is on that +window).

    +
    7. +
+
+
+
+

A.5.2. Disable the second Ethernet port

+
+

This may be useful, for example, if your second port has a IPMI +interface and the kernel detected a connection there and it is +interfering with the normal or the IPMI connection.

+
+
+
    +
  1. +

    If there is no Wired connection 2, click the + icon. Otherwise +select that connection, click the “gear” icon, and skip to step 4. +It may be benign to delete (- icon) any other connections except +Wired connection 1.

    +
    2. +
  2. +

    Make sure Ethernet is selected in the drop down box and click +Create…​.

    +
    3. +
  3. +

    Change the Connection name to Wired connection 2.

    +
    4. +
  4. +

    Select the Ethernet tab.

    +
    5. +
  5. +

    Use the drop-down for the Device field to select your device +(typically eno2 with the MAC address in parentheses). Then edit the +field to just list the name of the interface (typically eno2) by +removing the MAC address in parentheses.

    +
    6. +
  6. +

    Select the IPv4 Settings tab.

    +
    7. +
  7. +

    For Method select Disabled.

    +
    8. +
  8. +

    Select the IPv6 Settings tab.

    +
    9. +
  9. +

    For Method select Disabled.

    +
    10. +
  10. +

    Click Save.

    +
    11. +
  11. +

    Close the window by pressing Esc (while the focus is on that +window).

    +
    12. +
+
+
+
+
+

A.6. Disable Desktop User

+
+

If you do not need the functionality available in the Desktop +environment, you can disable the desktop account. You can re-enable +the account later if you need it. To disable it, execute:

+
+
+
+
usermod -L desktop
+
+
+
+

You can undo this by using the -U option instead.

+
+
+

To prevent connecting with ssh using a key, create (or add desktop +to an existing) DenyUsers line in /etc/ssh/sshd_config:

+
+
+
+
DenyUsers desktop
+
+
+
+

And restart sshd with:

+
+
+
+
systemctl restart sshd
+
+
+
+

You can undo the ssh block be removing the line (if it only has +desktop) or removing desktop from the line and then restarting +sshd.

+
+
+
+

A.7. Remove ModemManager package

+
+

If you use serial ports, it is strongly advised that you remove the +ModemManager package to avoid conflicts over access to the ports. +Execute this command:

+
+
+
+
apt-get purge modemmanager
+
+
+
+
+

A.8. Remove anacron package

+
+

If you enabled the weekly update job in fsadapt (it is strongly +recommended), we recommend that you also remove the anacron package +so that the job will run at a fixed time every week, even if the +system is turned off for some periods of time. Execute this +command:

+
+
+
+
apt-get purge anacron
+
+
+
+
+

A.9. Configure e-mail

+
+

The configuration described here (Internet site or mail sent by +smarthost in the exim4 configuration, no incoming mail, reply-to +filter, and modified user names), provides good support for system +messages and the FS msg and rdbemsg utilities.

+
+
+
    +
  1. +

    As root, enter:

    +
    +
    +
    dpkg-reconfigure exim4-config
    +
    +
    +
    +

    to change the setup. Typically you should select internet site, use +your host name in place of debian when it occurs, and otherwise +select defaults at all the other prompts. (The only other recommended +choices are local delivery only or mail sent by smarthost; received +via SMTP or fetchmail.) If you want to receive incoming mail, you +will also need to enable SMTP connections in Window 4 of fsadapt +(and if you are using a firewall, you will need to enable such +connections for it). We recommend that you NOT receive incoming mail +on this computer.

    +
    +
    2. +
  2. +

    Reply-To filter: If you follow the +recommendation not to receive incoming mail and your system is not +setup for local delivery only, you should set the Reply-To address +for outgoing messages to a real e-mail account at your institution +that is read regularly. You can do this by (all as root):

    +
    +
      +
    1. +

      Create the filter (four lines in file):

      +
      +
      +
      cat >/etc/exim4/reply-to-filter <<EOF
+# Exim filter          << THIS LINE REQUIRED
+
+headers remove "Reply-To"
+headers add "Reply-To: email@address"
+EOF
      +
      +
      +
      +

      Change email@address to the e-mail address you want replies to be +addressed to. If you want more than one, separate them with commas.

      +
      +
      2. +
    2. +

      Create a file for local customizations:

      +
      +
      +
      touch /etc/exim4/conf.d/main/00-exim-localmacros
+ln -sfn /etc/exim4/conf.d/main/00-exim-localmacros /etc/exim4/exim4.conf.localmacros
      +
      +
      +
      + + + + + +
      +
      Note
      +      		 +The file is constructed this way so that it will work for both +non-split or split exim4 configurations. +
      +
      +
      3. +
    3. +

      Add a call to the filter to /etc/exim4/exim4.conf.localmacros:

      +
      +
      +
      cat >>/etc/exim4/exim4.conf.localmacros <<EOF
+#set reply to
+system_filter = /etc/exim4/reply-to-filter
+EOF
      +
      +
      +
      4. +
    4. +

      Then execute

      +
      +
      +
      update-exim4.conf
+systemctl restart exim4
      +
      +
      +
      5. +
    +
    +
    3. +
  3. +

    You should change your /etc/aliases so root and prog e-mail goes to oper.

    +
    +
    +
    +
      +
    • +

      change root: desktop to root: oper

      +
      • +
    • +

      add prog: oper

      +
      • +
    • +

      add desktop: oper

      +
      • +
    +
    +
    +
    +
    +

    This is recommended as a “catch all” since the oper account is +presumably under regular use and any messages sent there are likely to +be noticed. This is particularly important for system error messages +since they should be delivered to a mail box on the system in case +there is a network problem that might prevent them from being +delivered off system. You can however add additional off machine +delivery of these messages to whatever addressees you wish and we +recommend this as well. These should include an e-mail account at your +institution that is read regularly (maybe the same address as the +Reply-To address you may have set above would be a good choice). To +do this, create a .forward file in oper's home directory. The +permissions should be -rw-r—​r--. The contents should be similar to +(left justified):

    +
    +
    +
    +
    \oper
+user@node.domain
    +
    +
    +
    +

    where user@node.domain is the off machine addressee you +want the messages to go to. You can add additional lines for +additional addressees. The backslash (\) before oper +prevents the mail system from getting into an infinite loop +re-checking oper's .forward file.

    +
    +
    4. +
  4. +

    If you have made the above changes to forward messages to another an +e-mail account on another machine, you should customize the User Name +(not login name, the User Name is the fifth field) of root, prog, + oper, and desktop in /etc/passwd to identify the source of the + message. For root and prog, it is recommended to append a + string like at node (it is probably best to avoid FQDNs), where + node is this machine, e.g., for atri you might change the 5th + field for root from

    +
    +
    +
    root
    +
    +
    +
    +

    to

    +
    +
    +
    +
    root at atri
    +
    +
    +
    +

    For oper, you might instead prepend your site name to the +accounts for clearer reading in ops e-mail messages, e.g., +for oper on atri at GSFC, we changed the 5th field for +oper to:

    +
    +
    +
    +
    GSFC VLBI Operator
    +
    +
    +
    +

    and for completeness, for prog and desktop we use:

    +
    +
    +
    +
    GSFC VLBI Programmer
+GSFC Desktop User
    +
    +
    +
    +

    These changes will help the recipient (possibly you) +determine which system generated this message since it may +not be obvious given the modified return address.

    +
    +
    5. +
  5. +

    To give oper an indication at login that there is mail to read, add +either (to get a count of messages):

    +
    +
    +
    test ! -f /var/mail/oper || from -c
    +
    +
    +
    +

    or (to see the senders and subjects):

    +
    +
    +
    +
    test ! -f /var/mail/oper || from
    +
    +
    +
    +

    to end of oper's .profile file (if using bash as the login +shell) or .login file (tcsh).

    +
    +
    6. +
  6. +

    Lastly, check the default mailbox directory /var/mail/ for +accounts that may have messages that arrived before the e-mail +system was fully configured. Be sure to resolve any system +messages that may have been received. You can check to see what +accounts have mail with:

    +
    +
    +
    ls /var/mail
    +
    +
    +
    +

    which will list each user account mail file that +exists. Check and clear each user’s mailbox (where user in +the line below is the account name) that has received mail +(as root):

    +
    +
    +
    +
    mail -f /var/mail/user
    +
    +
    +
    +

    If there are messages in the desktop user’s mailbox that you want to +preserve and oper's mailbox is empty or non-existent, you could +consider renaming desktop's mailbox to be oper's. If you do so, be +sure to change the owner of the file to be oper.

    +
    +
    7. +
+
+
+
+

A.10. Generate FQDN in HELO for outgoing mail

+
+

If mail from your system is being rejected by some servers because +exim4 is not providing a Fully Qualified Domain Name (FQDN), in its HELO +message, the following steps should fix the problem.

+
+
+
    +
  1. +

    If you have not already created +/etc/exim4/conf.d/main/00-exim-localmacros (see +Reply-To filter above), do so:

    +
    +
    +
    touch /etc/exim4/conf.d/main/00-exim-localmacros
+ln -sfn /etc/exim4/conf.d/main/00-exim-localmacros /etc/exim4/exim4.conf.localmacros
    +
    +
    +
    2. +
  2. +

    Add the necessary line to the file:

    +
    +
    +
    cat >>/etc/exim4/exim4.conf.localmacros <<EOF
+MAIN_HARDCODE_PRIMARY_HOSTNAME=ETC_MAILNAME
+EOF
    +
    +
    +
    3. +
  3. +

    Then execute:

    +
    +
    +
    update-exim4.conf
+systemctl restart exim4
    +
    +
    +
    4. +
  4. +

    Verify that the change has taken effect:

    +
    +
    +
    exim4 -bP primary_hostname
    +
    +
    +
    5. +
+
+
+
+

A.11. Set X display resolution at boot

+
+

If your display sometimes starts with the wrong resolution, you may be +able to configure a better resolution. The following is a description +of something that worked for at least one system. The details of your +system may require some changes (beyond the resolution and output name).

+
+
+

First you need to determine the correct resolution and output name. +You may be able to do this with xrandr. If the screen currently has +the correct resolution, you can just execute:

+
+
+
+
xrandr
+
+
+
+

The output might look like:

+
+
+
+
Screen 0: minimum 320 x 200, current 1920 x 1200, maximum 1920 x 2048
+VGA-1 connected primary 1920x1200+0+0 (normal left inverted right x axis y axis) 0mm x 0mm
+   1024x768      60.00
+   800x600       60.32    56.25
+   640x480       59.94
+  1920x1200 (0x42) 154.000MHz +HSync -VSync
+        h: width  1920 start 1968 end 2000 total 2080 skew    0 clock  74.04KHz
+        v: height 1200 start 1203 end 1209 total 1235           clock  59.95Hz
+
+
+
+

Where the current screen resolution is 1920x1200 and the output name is VGA-1.

+
+
+

You can then generate the needed Modeline by executing:

+
+
+
+
cvt 1920 1200
+
+
+
+

Which might generate output:

+
+
+
+
# 1920x1200 59.88 Hz (CVT 2.30MA) hsync: 74.56 kHz; pclk: 193.25 MHz
+ Modeline "1920x1200_60.00"  193.25  1920 2056 2256 2592  1200 1203 1209 1245 -hsync +vsync
+
+
+
+

As a test, you can make a script (use an appropriate name), that will +enable that resolution. Use the output name (VGA-1 in this example) +and the tokens following Modeline from above. There are three lines +after the #!/bin/bash line.

+
+
+
~/display_1920x1200
+
+
#!/bin/bash
+xrandr --newmode "1920x1200_60.00"  193.25  1920 2056 2256 2592  1200 1203 1209 1245 -hsync +vsync
+xrandr --addmode VGA-1 1920x1200_60.00
+xrandr --output VGA-1 --mode "1920x1200_60.00"
+
+
+
+

Be sure to chmod u+x the file before executing.

+
+
+

If that is successful, you can use output name (VGA-1 in this +example) and Modeline from above to make a file (you may need to create + the directory first):

+
+
+
/etc/X11/xorg.conf.d/10-monitor.conf
+
+
Section "Monitor"
+Identifier     "VGA-1"
+Option         "Enable" "true"
+Modeline "1920x1200_60.00"  193.25  1920 2056 2256 2592  1200 1203 1209 1245 -hsync +vsync
+EndSection
+
+Section "Screen"
+Identifier     "Screen0"
+Device         "Device0"
+Monitor        "VGA-1"
+DefaultDepth    24
+#Option         "TwinView" "0"
+SubSection "Display"
+    Depth          24
+    Modes          "1920x1200_60.00"
+EndSubSection
+EndSection
+
+
+
+

You should chmod the permissions for directory with o+rx and the +file with o+r, if those are not already set.

+
+
+

You could then try restarting the display (after closing all windows) with:

+
+
+
+
systemctl restart gdm3
+
+
+
+

or rebooting.

+
+
+
+

A.12. Use KeepAlive to prevent VLAN firewall inactivity time-out

+
+

If there is a VLAN firewall in use on the local network, it may be +necessary to use KeepAlive for TCP connections to prevent inactivity +time-outs for network connections from the FS to the VLBI equipment +when no activity is occurring with the system. For some devices, having +the time-out break the connection may cause an issue with the number of +connections available.

+
+
+

To use KeepAlive to prevent the inactivity time-outs, first install +the package libkeepalive0:

+
+
+
+
apt-get install libkeepalive0
+
+
+
+

Then add the follow lines for oper (and prog):

+
+
+
~/.profile
+
+
export KEEPCNT=20
+export KEEPIDLE=180
+export KEEPINTVL=60
+
+
+
+

Then add the following alias for oper (and prog):

+
+
+
~/.bash_aliases
+
+
alias fs='LD_PRELOAD=libkeepalive.so fs'
+
+
+
+

You will need to terminate the FS, log out, and log back in to activate these changes.

+
+
+ + + + + +
+
Note
+		 +If you run the FS from a script, you will need to include the +setting of LD_PRELOAD explicitly in the script since scripts do not +pick up aliases. +
+
+
+

A similar alias can used to allow other individual applications +to avoid the inactivity time-outs. (A better +solution is available for ssh, discussed below.) It is also possible to put +export LD_PRELOAD=libkeepalive.so in ~/.profile to enable it for all +applications, but this may generate some error messages (in the case of +xterm at least, the error is apparently benign).

+
+
+

If you need to have a persistent ssh connection, add the follow for oper (and prog):

+
+
+
~/.ssh/config file:
+
+
Host *
+    ServerAliveInterval 200
+    ServerAliveCountMax 2
+
+
+
+

This can be set selectively per remote system. The interval of 200 +seconds is chosen to be less than the 300 seconds that some (possibly +security hardened) servers may use.

+
+
+

If not already set correctly, set the ~/.ssh/config file’s +permissions and ownership for oper (analogously for prog) with:

+
+
+
+
chmod 644 ~oper/.ssh/config
+chown oper.rtx ~oper/.ssh/config
+
+
+
+
+

A.13. Remove login banners for commands run by ssh on remote systems

+
+

If you use ssh as oper (and maybe prog), to run commands on +other systems as part of FS operations, you may get login banners +mixed in with the output. You can suppress the banners by adding the +following for oper (and analogously for prog):

+
+
+
~/.ssh/config file:
+
+
Host *
+    LogLevel ERROR
+
+
+
+

This will allow errors to be displayed while suppressing the login +banners of remote systems. This can be set selectively per remote +system.

+
+
+

Please check the end of the Use KeepAlive to prevent VLAN firewall inactivity time-out +section for setting the ownership and permissions on ~/.ssh/config.

+
+
+
+

A.14. Suspend, shutdown, and restart issues

+
+
    +
  1. +

    Mouse cursor disappearing on text console after suspend

    +
    +

    The FSL11 installation disables suspend by default (as part of the +greeter item in the FS Adaptation: Setup (Window 2) sub-step of +fsadapt in the Third stage installation). If you did not +disable suspend, you may encounter this issue. A way to fix it is to +switch to a different text console and then back again. The cursor +should reappear.

    +
    +
    2. +
  2. +

    Disable the power switch from shutting the system down

    +
    +
      +
    1. +

      Add the following to the /etc/gdm3/greeter.dconf-defaults file:

      +
      +
      +
      # Disable restart buttons
+disable-restart-buttons=true
      +
      +
      +
      2. +
    2. +

      Restart gdm3:

      +
      +
      +
      systemctl restart gdm3
      +
      +
      +
      3. +
    +
    +
    3. +
  3. +

    Disable use of restart for ordinary users

    +
    +

    It is possible to disable all use of restart for ordinary users with a +bit more work — the details are available on request. The file +powerlock.tar.gz may be helpful for this. It contains sample +contents of the files that need to be changed or created.

    +
    +
    4. +
+
+
+
+

A.15. Printer setup

+
+
    +
  1. +

    Make sure your printer is connected, to the computer or the network, as appropriate.

    +
    + + + + + +
    +
    Tip
    +    		 +Newer computers usually do not have a parallel port +(IEEE 1284). If not, and your printer requires a +parallel connection, you should be able to obtain a +USB/Parallel converter for less than US$20. +
    +
    +
    2. +
  2. +

    Login in to the X-display or remotely using an X-capable display.

    +
    3. +
  3. +

    Start firefox

    +
    4. +
  4. +

    Enter URL: localhost:631

    +
    5. +
  5. +

    Select Add printers and classes.

    +
    +

    You may be prompted to enter credentials. If your account is a member +of the lpadmin group, you can use your own credentials; if not, those of the +root account or another account that is a member of lpadmin will be required.

    +
    +
    6. +
  6. +

    Add your printers.

    +
    +

    Connected printers may be automatically offered to be added. You may +also be able to find printers using the Find Printer function. If +CUPS offers you the wrong type of printer to be automatically added or +it is unclear what driver to select for a printer, you may be able to +get some useful information to help with manually installing your +printer by searching the Internet for the string cups and your +printer model.

    +
    +
    +

    Some printers will work with an AppSocket/HP JetDirect connection of the form socket://hostname.

    +
    +
    7. +
  7. +

    Be sure to select a printer as the default (usually by selecting +Printers at the top of the page, then select the printer to be set as the +default, then from the Administration drop down: Set As Server Default).

    +
    8. +
  8. +

    Quit firefox

    +
    9. +
+
+
+
+

A.16. NTP configuration

+
+

For good performance with NTP, please follow the recommendations in +/usr2/fs/misc/ntp.txt.

+
+
+

Additionally, to make the ntpq -c pe output more readable for local +devices, you can adjust the contents of /etc/hosts. The local +devices should be listed in the file, but use a nickname (15 +characters or less) that is meaningful locally in place of the +canonical name (the first name after the IP address). The canonical +name can be listed after the nickname.

+
+
+
+

A.17. Add raid-events scripts

+
+

If your system is using a RAID configuration, you may want to install +the raid-events script. The script provides email notifications of +when Rebuilds (and array checks) start and end. For full details on +the script and installation instructions, please see the +raid-events subsection in the +Script descriptions section of the +RAID Notes for FSL 11 document.

+
+
+
+

A.18. Add refresh_spare_usr2

+
+

If you are using two systems, an operational and a spare, you may +want to install the refresh_spare_usr2 script. The script can be +used to backup the /usr2 partition on the operational system to +the spare system. For full details on the script and installation +instructions, please see the +refresh_spare_usr2 subsection in the +Script descriptions section of the +RAID Notes for FSL 11 document.

+
+
+
+

A.19. Install pgplot version of pgperl

+
+ + + + + +
+
Important
+		 +This step is “use as at your own risk.” Every effort has +been made to make it safe, but it installs a non-standard package. You +should only use it if you need it and accept the risk. +
+
+
+

This replaces the use of the giza package in pgperl with pgplot. +It will restore the behavior of pgperl (used by plotlog) from +distributions FSL10 and earlier. Full directions can be found in the +INSTALL file in sub-directory +libpgplot-perl.

+
+
+

This package uses the same pgperl source as the standard version, +but it is built against pgplot instead. If pgperl receives a +security update, the pgplot version will be overwritten. It is +possible to prevent that if you prefer.

+
+
+
+
+
+

Appendix B: Managing Security Updates

+
+
+

It is strongly recommended that you use the weekly cron update +download (the “weekly cron job”) as configured according to the +Window 2 subsection in the fsadapt section above. This will +keep you informed of the available updates on a weekly basis.

+
+
+

It is also recommended that you remove anacron as described in the +Remove anacron package section below. This will cause the updates +to always be downloaded at what should be innocuous time, early Sunday +morning (but this can be adjusted if need be).

+
+
+ + + + + +
+
Note
+		 +An optional method for identifying available updates without using +the weekly cron job is described below in the section +Manually checking for updates. +
+
+
+

B.1. Installing updates (upgrading)

+
+ + + + + +
+
Tip
+		 +It is recommended that a disk rotation be performed before any +update is installed. This will make recovery much easier if a problem with the +update is discovered. Please see the FSL11 Raid document section +Recoverable testing for a +streamlined method to manage testing of updates. +
+
+
+

If updates are needed, the weekly cron job will send a message to root +(or whoever e-mail to root is aliased to, typically oper) with +instructions on how to install the updates. You can choose a +convenient time, when not in (or about to start) operations, to install +the updates and test the system.

+
+
+ + + + + +
+
Important
+		 +The weekly cron job message will include instructions for +handling a kernel update if one is available. See the +Kernel updates subsection below for additional considerations for +kernel updates. +
+
+
+

The commands for installing the updates given by the message are (note + the use of apt instead of apt-get):

+
+
+
+
apt upgrade
+
+
+
+

Enter y to confirm as needed. Then

+
+
+
+
apt clean
+
+
+
+

If the weekly cron job was installed according to the fsadapt +section above (for Window 2), the first of these commands (with + upgrade) will show if any NEWS items are included in the +update. If there are, they will be displayed by a paging program at the beginning of the upgrade and +you will be given an extra chance to abort before installing.

+
+
+ + + + + +
+
Note
+		 +NEWS items are, rarely occurring, announcements that may +indicate additional steps are needed beyond the standard installation +process. If any NEWS items are displayed, you should consider +whether these will effect your system and how to handle them before +installing. The first command above (with upgrade) will also cause e-mails +to be sent to root with the NEWS information. +
+
+
+
+

B.2. Kernel updates

+
+ + + + + +
+
Warning
+		 +Kernel updates require extra care and testing. If you are +using a RAID, you should consider using the +Recoverable testing +procedure to give more, and easier, options for recovery in case there +is a problem. That procedure contains special instructions for kernel +update testing. +
+
+
+ + + + + +
+
Note
+		 +
+

When a kernel update is available, you may see messages at the start of the cron job output similar to:

+
+
+
+
apt-listchanges: Unable to retrieve changelog for package linux-headers-amd64; 'apt-get changelog' failed with: E: Version '5.10.120+1' for 'linux-headers-amd64' was not found
+E: No packages found
+
+apt-listchanges: Unable to retrieve changelog for package linux-image-amd64; 'apt-get changelog' failed with: E: Version '5.10.120+1' for 'linux-image-amd64' was not found
+E: No packages found
+
+
+
+

and

+
+
+
+
Calling ['apt-get', '-qq', 'changelog', 'linux-headers-amd64=5.10.120+1'] to retrieve changelog
+Calling ['apt-get', '-qq', 'changelog', 'linux-image-amd64=5.10.120+1'] to retrieve changelog
+
+
+
+

These appear to be benign. Our only advice at this time is to ignore +them.

+
+
+
+
+

If there is a kernel update available, the weekly cron job output +will include a warning at the end with additional instructions +depending on which type is available. There are two types of kernel +updates:

+
+
+
    +
  1. +

    ABI updates, e.g., from 4.9.0-11-amd64 to +4.9.0-12-amd64 (with 11 and 12 being the ABI versions), which change the kernel ABI (Application Binary + Interface). The warning for this case is:

    +
    +
    +
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+NB: The Linux kernel image is one of the packages due to be upgraded.
+NB: (The kernal ABI has changed as per the linux-latest source package above
+NB:  so all out-of-tree modules WILL NEED TO BE REBUILT after you REBOOT.)
+NB: Please allow _extra time_ for TESTING after the upgrade.
+!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    +
    +
    +
    2. +
  2. +

    Non-ABI updates, which update the kernel, but do not change the +ABI. The warning for this case is:

    +
    +
    +
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+NB: The Linux kernel image is one of the packages due to be upgraded.
+NB: (Upgrading will OVERWRITE the running kernel and require you to REBOOT!)
+NB: Please allow _extra time_ for TESTING after the upgrade.
+!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    +
    +
    +
    3. +
+
+
+

Be sure to allow time to follow the instructions when planning to +install these updates. As described in the ABI update warning, you +will need to rebuild any out-of-tree modules after rebooting for that +case. This is discussed in the Updating out-of-tree modules +subsection below.

+
+
+ + + + + +
+
Caution
+		 +In extreme circumstances, an ABI (but not a non-ABI) kernel +update can be deferred to a later date when more extensive testing can +be performed by using apt-get in place of apt in the instructions +for installing the update. This works because an ABI update involves +new packages. The apt-get command will install the updates for existing +packages, but it will not install the new packages. While this method can +be used to install the other updates, it is not recommended since +there are presumably security patches needed for the kernel and they +are not being installed in this case. +
+
+
+ + + + + +
+
Tip
+		 +
+

When the kernel is upgraded, you may get messages such as:

+
+
+
+
update-initramfs: Generating /boot/initrd.img-5.10.0-16-amd64
+W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast
+
+
+
+

These are usually benign, unless you need that firmware. If you don’t, +these messages can be silenced for future upgrades by creating an +empty version of the file. For this example, enter:

+
+
+
+
touch /lib/firmware/ast_dp501_fw.bin
+
+
+
+
+
+

B.2.1. Updating out-of-tree modules

+
+

When a ABI update is installed, it will be necessary to update any, +so-called, out-of-tree modules that use the kernel ABI. This must be +done after rebooting with the new kernel installed.

+
+
+

For a normal FSL11 installations, unless you have installed other +out-of-tree modules, the only module that needs to be rebuilt is the +GPIB driver (if it is installed). You will need to recompile it (usually using fsadapt, + Window 2, config_gpib only) after the initial reboot + and then (to keep these instructions simple) reboot again.

+
+
+

If you have installed other out-of-tree modules (e.g., you use a +special driver for some of your NICs), you will need to update them +appropriately after the initial reboot and then (to keep these + instructions simple) reboot again.

+
+
+
+
+

B.3. Recovery from a failed update

+
+

If an update fails, e.g., an updated kernel fails to boot or another problem is discovered, +you can recover as described in FSL11 RAID document +Recoverable testing +section, if you were following that method, or from a shelf disk +according to the FSL11 RAID document Recover from +a shelf disk section if not and you have a good shelf disk.

+
+
+

B.3.1. Additional recovery option for a failed ABI kernel update

+
+

For a ABI update that has failed, it is also possible to try to use +the previous kernel on the current system. For a single boot, use the +Advanced option in the grub menu at boot and then select the +previous kernel. You can change back permanently to the previous +kernel by purging the new kernel and its headers. To do this, use:

+
+
+
+
dpkg -l|grep linux-image
+dpkg -l|grep linux-headers
+
+
+
+

to determine the ABI version to be removed. For example, for the +first command above, you may get:

+
+
+
+
linux-image-4.9.0-11-amd64
+linux-image-4.9.0-12-amd64
+
+
+
+

The package with 12 would be the later version that should be purged:

+
+
+
+
apt-get purge linux-image-4.9.0-12-amd64
+
+
+
+

Likewise with the linux-headers. For example, for the 12 ABI +version, there will be two packages you should purge:

+
+
+
+
linux-headers-4.9.0-12-amd64
+linux-headers-4.9.0-12-common
+
+
+
+
+
+

B.4. Manually checking for updates

+
+

If you do not use the weekly cron job to check for updates, or if +you want to make sure you have the very latest updates when you +install them, you can run the distributed copy of the weekly update +script manually to check for updates:

+
+
+
+
/root/fsl11/etc_cron.weekly_apt-show-upgradeable
+
+
+
+

If there is no output, there are no updates to install.

+
+
+

If there is output, there are updates to install. You can install them +by following the installation procedure in subsection +Installing updates (upgrading) above, except you will use the +instructions from the output of the script above instead of from the +weekly cron job (the outputs should be equivalent for the same set +of updates). Additionally, please read the following NOTE.

+
+
+ + + + + +
+
Note
+		 +If the weekly cron job has not been installed, you may not get a + display of NEWS items and a chance to abort when you install the updates. You + can use the method below with the --which=news parameter to + check for NEWS before installing an update. +
+
+
+

Any NEWS items will be included in the script output along with the +packages to be updated. If you would like to see any NEWS items more +distinctly after the previous command and before installing the +updates, you can run the script again using the --which=news option:

+
+
+
+
/root/fsl11/etc_cron.weekly_apt-show-upgradeable --which=news
+
+
+
+

If there are updates available and no NEWS items, you will only get +the installation instructions.

+
+
+

You can use this second form of running the script to check for +updates initially, if you do not need to review which updates are +available (you will still get warnings about kernel updates). As +usual, you will see no output at all if there are no updates +available.

+
+
+
+

B.5. End of security updates

+
+

When support for bullseye ends, currently expected in May 2024, +there will be no more security updates. At that time, the existing +packages will be migrated to the Debian archive site. This will be +visible in the output from the weekly cron job script as errors that +the packages files can’t be found. Two steps are needed at that time:

+
+
+
    +
  1. +

    If you have been using the weekly cron job, it should be deleted:

    +
    +
    +
    rm /etc/cron.weekly/apt-show-upgradeable
    +
    +
    +
    +

    (you may need to answer y to confirm)

    +
    +
    2. +
  2. +

    Change the /etc/apt/sources.list file to point to the archive +site. Although there will be no more security updates, this will enable +downloading of additional packages if they are needed. The new lines that +should replace the corresponding existing lines are:

    +
    +
    +
    deb http://archive.debian.org/debian/ bullseye main contrib non-free
+deb http://archive.debian.org/debian-security bullseye/updates main contrib non-free
    +
    +
    +
    +

    And if you are using deb-src lines:

    +
    +
    +
    +
    deb-src http://archive.debian.org/debian/ bullseye main contrib non-free
+deb-src http://archive.debian.org/debian-security bullseye/updates main contrib non-free
    +
    +
    +
    +

    Otherwise the deb-src lines can be commented out (with a leading #). Any other deb or +deb-src lines relating to updates, proposed-updates etc. should likewise be commented out.

    +
    +
    +

    In addition, if you want to install packages from more recent +distributions that have been backported to bullseye you can add:

    +
    +
    +
    +
    deb http://archive.debian.org/debian-backports bullseye-backports main contrib non-free
    +
    +
    +
    +

    However, the “backports” are not normally needed.

    +
    +
    +

    Lastly, update the index files:

    +
    +
    +
    +
    apt-get update
    +
    +
    +
    +

    This may generate an error about a Release file having expired, but that is benign.

    +
    +
    3. +
+
+
+ + + + + +
+
Note
+		 +When bullseye becomes no longer supported, it is strongly advised that you move +your FS machine behind a firewall or upgrade it to a more recent FS Linux release. +
+
+
+
+
+
+

Appendix C: Other Maintenance Procedures

+
+
+

This appendix covers additional procedures for maintaining your +system.

+
+
+

C.1. Update IP address, hostname, FQDN, and other network information

+
+

This is useful if the computer is physically moved to a different +site, its IP address changes, or its network information needs to be +updated for a different reason. This is typically not needed if you +use DHCP, though that may still require some of the changes in the +Modify other system files step below (please let us know +if you gain experience).

+
+
+

This subsection requires using nm-connection-editor on a graphic +display (nmtui may be an option on a text terminal, but it has not +been fully verified). You may need to be root or desktop to do +this. This subsection assumes you are in the program and have +sufficient permissions.

+
+
+ + + + + +
+
Note
+		 +If you move the disks to a computer with a different mainboard +model, the device names of the network interfaces may change. In that +case, you will need to reselect the names as described in the +sub-steps of the Stabilize network configuration section of the +Additional Setup Items appendix. This should not be necessary if +the origin and destination computers have the same mainboard. +
+
+
+
    +
  1. +

    Select your connection and click the “gear” icon.

    +
    2. +
  2. +

    Select the IPv4 Settings (or IPv6 Settings if you are using +IPv6) tab.

    +
    3. +
  3. +

    Adjust your Manual Method configuration: Addresses, DNS +Servers (comma separated), and Search domains.

    +
    4. +
  4. +

    Click Save.

    +
    5. +
  5. +

    Close the window by pressing Esc (while the focus is on that +window).

    +
    6. +
  6. +

    Modify other system files:

    +
    +

    Update the information as appropriate. The system may have initially +been installed with the default hostname debian and no domain name.

    +
    +
    +
    /etc/hostname
    +

    Change your hostname

    +
    +
    +
    /etc/hosts
    +

    Update your IP address, FQDN (canonical name), and alias (typically +the hostname, but multiple aliases/nicknames are allowed).

    +
    +
    +

    If you moved your computer to a new LAN environment, you may also want +to update the nodes and aliases listed, see also Setup /etc/hosts.

    +
    +
    +
    /etc/networks
    +

    Use your local subnet (class A, B, or C) for the localnet line.

    +
    +
    +
    /etc/mailname
    +

    Use fully qualified node name.

    +
    +
    + + + + + +
    +
    Note
    +    		 +
    +

    If your system doesn’t have a FQDN or you don’t want to show it in +e-mail messages, you may be able to use a fake one. A FQDN may be +necessary to allow messages to be sent successfully to some remote +hosts and mailman mail lists. A possible strategy for this is to +append .net to the node name you use in this file and the next. The +node name in these two files can be different than the official +hostname. However, these two mail related files should be consistent. +You might consider fs1-<xx>.net (or fs2-<xx>.net), where <xx> is +your station two letter code (lower case).

    +
    +
    +
    +
    +
    /etc/exim4/update-exim4.conf.conf
    +

    Look for hostnames=, use fully qualified domain name.

    +
    +
    +

    Then execute:

    +
    +
    +
    +
    update-exim4.conf
    +
    +
    +
    +

    When finished, reboot.

    +
    +
    7. +
+
+
+
+

C.2. Increase the size of an LVM volume

+
+

It is possible to increase the size of an LVM volume if there is +additional room available in its volume group. These instructions +assume you will be resizing a logical volume for a typical +configuration. For example, for the logical volume mounted at /usr2, +on RAID device /dev/md0, which is using /dev/sda2 and /dev/sdb2. +Additionally, example pathnames are given in the instructions below +for adjusting the size of the logical volume for /usr2. All these +names may be different if you want to resize a different volume and/or +your disk configuration is different.

+
+
+
    +
  1. +

    Preparation

    +
    +
      +
    1. +

      Check that there is enough free space available.

      +
      +

      Examine the output of:

      +
      +
      +
      +
      vgs
      +
      +
      +
      +

      You can increase the size of a logical volume if the volume group +(under the VGS column heading) has enough free space (VFree +heading) for the increase. Typically, the volume group would be vg0.

      +
      +
      2. +
    2. +

      Determine the Path of the logical volume you want to extend.

      +
      +
        +
      1. +

        Get a listing to relate the internal device-mapper pathnames +(under the Filesystem column heading) and where the logical volumes +are mounted (Mounted on heading). For example, +/dev/mapper/vg0-usr2 would typically be mounted at /usr2.

        +
        +
        +
        df -h
        +
        +
        +
        2. +
      2. +

        Get a listing to relate the internal device-mapper pathname (under +the DMPath column heading) to the logical volume Path. For +example, for /dev/mapper/vg0-usr2, the Path would typically be +/dev/vg0/usr2.

        +
        +
        +
        lvdisplay -C -o lv_dm_path,lv_path
        +
        +
        +
        3. +
      3. +

        For the mount point of the logical volume you want to extend, +determine the Path using the internal device-mapper pathname from +the above two sub-steps. For example, the logical volume for /usr2 +would typically correspond to /dev/mapper/vg0-usr2 and the +corresponding Path would be /dev/vg0/usr2.

        +
        4. +
      +
      +
      3. +
    +
    +
    2. +
  2. +

    Pre-check (optional)

    +
    +

    This sub-step is not required but can be used, along with the +“Post-check” sub-step below, to check that the volume size changed as +expected and that no files were lost or changed +size/modification-time.

    +
    +
    +
      +
    1. +

      Get the size (under the 1G-block column heading) of the logical +volume (Mounted on heading) for the volume of interest:

      +
      +
      +
      df -BG
      +
      +
      +
      +

      Record the size to compare to the results in the “Post-check” +sub-step below.

      +
      +
      2. +
    2. +

      Make a listing of the files on the mount_point (include the +leading /) to be changed. For example, the mount_point might be +/usr2.

      +
      +
      +
      ls -ltR mount_point >/tmp/before.txt
      +
      +
      +
      3. +
    +
    +
    3. +
  3. +

    Make the change, using the Path you determined in the +“Preparation” sub-step above.

    +
    +
      +
    1. +

      Make a backup of your system.

      +
      + + + + + +
      +
      Note
      +      		 +This sub-step, and recovery in case of a problem, is much easier +if you using the FSL11 RAID system. If not, it is strongly recommended +that you make your own backup of your entire system. The remainder of +this sub-step assumes you are using a RAID, following the approach of +the Recoverable testing procedure +in the Raid Notes for FSL11 document. +
      +
      +
      +

      If you are using a RAID, you can drop the primary disk out of the +RAID to save as a backup:

      +
      +
      +
      +
      drop_primary
      +
      +
      +
      2. +
    2. +

      Extend Path

      +
      +

      For the logical volume (mount point) you want to extend, you can +either:

      +
      +
      +
        +
      1. +

        Incrementally increase the size. For example, to increase Path +by 4 GB:

        +
        +
        +
        lvextend -L+4G Path
        +
        +
        +
        2. +
      2. +

        Set the size to a new larger total size, say 8GB:

        +
        +
        +
        lvextend -L8G Path
        +
        +
        +
        3. +
      +
      +
      3. +
    3. +

      Resize Path

      +
      + + + + + +
      +
      Important
      +      		 +Do not interrupt the next command. If it is interrupted +and you are using the Recoverable +testing procedure in the Raid Notes for FSL11 +document, you will need to utilize the +If the update is +deemed to have failed subsection of that procedure. Otherwise, if +you are not using that procedure, you will need to use your own +recovery method. +
      +
      +
      +
      +
      resize2fs Path
      +
      +
      +
      4. +
    +
    +
    4. +
  4. +

    Post-check (optional)

    +
    +

    This sub-step is not required but can be used, if the “Pre-check” +sub-step above was used, to check that the new size is correct and no +files were lost or changed size/modification-time.

    +
    +
    +
      +
    1. +

      Check that the size of the logical volume (under the Mounted on +column heading) has the expected new size in the output of:

      +
      +
      +
      df -BG
      +
      +
      +
      +

      Compare the result to that in the “Pre-check” sub-step above.

      +
      +
      2. +
    2. +

      Make a listing of the files on the mount_point (include the +leading /) that was changed. For example, the mount_point might +be /usr2.

      +
      +
      +
      ls -ltR mount_point >/tmp/after.txt
      +
      +
      +
      3. +
    3. +

      Compare the before and after listings of the files

      +
      +
      +
      diff /tmp/before.txt /tmp/after.txt
      +
      +
      +
      4. +
    +
    +
    +

    There should be no differences in the listings except any changes that +can explained by other expected activity that occurred since the +“Pre-check” sub-step above. If there was no other activity on the +logical volume, there should be no differences.

    +
    +
    5. +
  5. +

    Cleanup

    +
    + + + + + +
    +
    Note
    +    		 +If you not are using the +Recoverable testing procedure in +the Raid Notes for FSL11 document, you will need to use +your own methods to restore the system if there was a problem. This +step describes how to proceed if you are using the referenced +procedure. +
    +
    +
    +

    There are two options:

    +
    +
    +
      +
    1. +

      If you are satisfied with the change, you can recover the RAID +with:

      +
      +
      +
      recover_raid
      +
      +
      +
      +

      This should only take a few minutes.

      +
      +
      + + + + + +
      +
      Note
      +      		 +The change in the volume size will not propagate to the shelf +disk until the next disk rotation. +
      +
      +
      2. +
    2. +

      If you are not satisfied with the change, you can try again if you +first restore the RAID using the +If the update is +deemed to have failed subsection of the +Recoverable testing procedure in +the Raid Notes for FSL11 document.

      +
      3. +
    +
    +
    6. +
+
+
+
+
+
+

Appendix D: Rescue Mode

+
+
+

Rescue mode is useful for repairing some problems that prevent booting +and/or logging in.

+
+
+ + + + + +
+
Note
+		 +If your computer’s setup utility is locked with a password, you +may need that password to select booting from your installation media. +
+
+
+ + + + + +
+
Note
+		 +You should provide suitable values for your system when a +specific value is required. Values that agree with the FSL11 install +described in this document (or reasonable defaults) are shown in parentheses. +
+
+
+
    +
  1. +

    Boot from installation media

    +
    2. +
  2. +

    Select Advanced options …​

    +
    3. +
  3. +

    Select …​ Rescue mode

    +
    + + + + + +
    +
    Note
    +    		 +
    +

    You could instead add parameters to the boot line (by entering e for UEFI or +Tab for BIOS on the …​ Rescue mode line instead), following the +directions in the Set boot options and boot installer section above. +This is not necessary nor usually helpful, but if you use this approach the +most useful parameters are probably netcfg/disable_dhcp=true and/or +time/zone=UTC. Use of added parameters will change the dialogue +below.

    +
    +
    +
    +
    4. +
  4. +

    Select Language (English)

    +
    5. +
  5. +

    Select Location (United States)

    +
    6. +
  6. +

    Select Keymap (American English)

    +
    7. +
  7. +

    Network configuration

    +
    +

    If no network is currently available (or you know that you do not need it +for the rescue), simply press Enter when DHCP autoconfiguration starts and +press Enter again for the resulting Network autoconfiguration failed +message. Thereafter select Do not configure the network at this time and +enter in the machine’s hostname when prompted before continuing below.

    +
    +
    +

    If the DHCP autoconfiguration succeeds before you can stop it, you may +as well confirm the hostname and domainname and continue with the +network anyway, since you never know when it might prove useful. +(However, if you want to make sure you don’t use the network, you can + select Go Back and press Enter for the resulting Network + autoconfiguration failed message. Thereafter select Do not + configure the network at this time and enter in the machine’s + hostname when prompted before continuing below.)

    +
    +
    +

    Otherwise if the DHCP autoconfiguration fails and you want to use the +network, press Enter for the resulting Network autoconfiguration +failed message. You can then select the appropriate option, most +likely Configure network manually and give appropriate responses to the +prompts, ultimately continuing below.

    +
    +
    8. +
  8. +

    Select time zone (Eastern)

    +
    + + + + + +
    +
    Note
    +    		 +The selected time zone will have no effect on the timestamps +stored on the disk for any changes you may make, but will affect the displayed times you see. +
    +
    +
    9. +
  9. +

    Unless you are not using Software RAID, select Assemble RAID array

    +
    +

    Press Space on Automatic and Enter to continue

    +
    +
    10. +
  10. +

    Select your root file system (/dev/vg0/root)

    +
    11. +
  11. +

    Select Yes to mount separate /boot partition (/boot), unless it is corrupt

    +
    +

    For UEFI boot also select Yes to mount separate /boot/efi partition (/boot/efi), +unless it is corrupt

    +
    +
    12. +
  12. +

    Select Execute a shell in /dev/vg0/root (or whatever your root file system is)

    +
    13. +
  13. +

    Select Continue to enter rescue mode

    +
    14. +
  14. +

    Use whatever commands are needed for your repair

    +
    + + + + + +
    +
    Note
    +    		 +
    +

    If you need to use the network, DNS does not appear to work by +default in recovery mode. Use of explicit IP addresses does work. If +you need to use DNS, you can make it functional by deleting the symbolic +link /etc/resolv.conf and creating it as a normal file with the +nameserver information you want, e.g.:

    +
    +
    +
    +
    rm /etc/resolv.conf
+cat >>/etc/resolv.conf <<EOF
+nameserver 8.8.8.8
+EOF
    +
    +
    +
    +
    +
    15. +
  15. +

    Use the exit command to exit when done

    +
    16. +
  16. +

    Select Reboot the system

    +
    17. +
  17. +

    “Bob’s your uncle” (i.e., you are done!)

    +
    18. +
+
+
+
+
+ + + \ No newline at end of file diff --git a/installation.html b/installation.html new file mode 100644 index 0000000..d5745eb --- /dev/null +++ b/installation.html @@ -0,0 +1,4786 @@ + + + + + + + + +FS Linux 11 Installation Guide + + + + + +
+
+
+
+
+
+
+

1. Introduction

+
+
+

These instructions provide a complete method for system installation +and some tuning. They are not the only method for accomplishing these +goals, but have been well tested. Experts can of course use their own +means, but the farther they deviate from this model, the less support +we will be able to provide.

+
+
+

The standard configuration uses a RAID1 system with removable disks. +Normally, two disks would be in use at a given time. A third disk is +used as a back-up and rotated into use periodically. More disks can be +used for further redundancy. You can of course provide your own +back-up method and can install the system to a single disk if you do +not want to use the software RAID.

+
+
+

If you are using the RAID configuration, you may wish to review the +Recommended practices subsection +of the RAID notes for FSL11 document before installing. +However, all of the practices listed there can be implemented after +the installation steps below are complete.

+
+
+ + + + + +
+
Tip
+		 +Removable disks should be used with a carrier/receiver system +that can tolerate a large number of insertions; “bare” disks should +not be inserted repetitively. Two receivers would normally be mounted +in the computer chassis. Each disk would be in its own carrier. We can +provide a recommendation for a carrier/receiver system if you need +one. +
+
+
+

Please note that for each step in this guide, we recommend you +carefully read all the included caveats and notes as the material is +not always logically sequential, i.e., instructions may proceed +explanations that impact what you actually type.

+
+ + +++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 1. FSL distributions
FS LinuxRelease NameDebian VersionLinux kernelYear

1

(Slackware)

1.2.<x>

1994

2

bo

1.3.1

2.0.29

1997

3

hamm

2.0

2.0.34

1998

slink

2.1

2.0.36

1999

4

potato

2.2

2.2.18

2000

5

woody

3.0

2.2.20/2.4.18

2002

6

sarge

3.1

2.4.27

2005

7

etch

4.0

2.6.18

2007

8

lenny

5.0

2.6.26

2009

squeeze

6.0

2.6.32

9

wheezy

7.0

3.2.0

2014

jessie

8.0

3.16.0

10

stretch

9.0

4.9.0

2020

buster

10.0

4.19.0

11

bullseye

11.0

5.10.0

2023

+
+

The FSL11 documents follow the FS font conventions, which can be found +at: https://nvi-inc.github.io/fs/misc/font_conventions.html.

+
+
+
+
+

2. Choosing architecture and creating installation media

+
+
+

FSL11 can be configured for either the i386 or amd64 +architectures. With FSL11 it is necessary to use Field System version +10.2 or later. Those FS versions support both architectures +natively, so either may be used. The amd64 architecture is preferred +and should be used if possible (it should be unless the processor is +very old, from about 2010 or older). However, some work may be +required to port your station code from a 32-bit to a 64-bit OS. An +automatic tool has been developed to help with this, and can be +provided upon request. Usually the i386 architecture will work on +any processor, but requires use of the Legacy (or BIOS) boot mode +in most cases. The amd64 installation media will fail to boot on a +system that is 32-bit only.

+
+
+

To install Debian 11, you can either use a DVD or USB drive. The latter is +faster, and also easier if you wish to use UEFI. Directions for creating your +installation media can be found online.

+
+
+ + + + + +
+
Note
+		 +Don’t be confused by the amd64 name, this architecture supports both +AMD and Intel manufactured x86-64 processors. This includes CPU lines such as +Ryzen, Epyc, Core, and Xeon. The naming scheme dates back to when Intel had a +competing and incompatible 64-bit architecture ia64. +
+
+
+

You can install from a DVD drive, USB device, or over the network. Any revision of +11.<x> installer should work fine. +Note also that installing from DVDs as described here is +recommended mainly for sites with little to poor Internet connectivity (even +then, use of a single DVD may suffice) and the equivalent use of a “Debian +GNU/Linux 11.<x> Bullseye - Official i386/amd64 netinst CD” would suffice for +installation at most sites with good connectivity. Official images for the installer +can be found at: https://cdimage.debian.org/cdimage/release/ or alternatively, +should your hardware require non-free firmware, unofficial images for the +installer that also include all available non-free firmware can be found at: +https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/

+
+
+

The details of creating your installation media can be found in the Debian +installation guide available from: +https://www.debian.org/releases/bullseye/installmanual

+
+
+
+
+

3. Motherboard setup

+
+
+ + + + + +
+
Caution
+		 +If you are installing to a virtual machine, make sure it is +configured to have at least two cores. This is required for FS display +server support. +
+
+
+ + + + + +
+
Note
+		 +
+

Some hardware may require special procedures. For cases we know about, +instructions are provided in top-level sub-directories of the +repository, which is visible at https://github.com/nvi-inc/fsl11. In +addition to special instructions, there may be needed software/drivers +in the sub-directories. The following table lists the cases that are +currently covered. Some of these solutions may be useful for other +hardware with the same issues. If you have the listed hardware or +issue, we recommend reading the instructions before beginning the +installation.

+
+ + +++++ + + + + + + + + + + + + + + +
Table 2. Special installation instructions available
DirectoryHardwareIssue

None at this time

+
+
+
+

Modern motherboards offer two forms of booting: native UEFI or BIOS +emulation (“Legacy”). UEFI is the preferred approach. Either mode of +boot is supported by this installation guide, and you will be given +alternatives when the instructions differ.

+
+
+

Decide which boot mode you want to use and select it through the +motherboard setup menu (typically by pressing Delete during +the power-on self test, aka POST).

+
+
+

Make sure that the motherboard time is set to the current Coordinated +Universal Time, i.e., UTC, and the motherboard can boot from the +installation media.

+
+
+

While you are in the motherboard menu, make sure that hot-swapping is +enabled for the controllers of both the primary and secondary +disks. This is necessary for disk rotation and recoverable testing.

+
+
+ + + + + +
+
Tip
+		 +For UEFI, some motherboards may switch to booting to the UEFI +shell if they fail to find a hard disk that will boot. This might +happen, for example, if you attempt to boot from a blank disk. If you +become stuck booting to the UEFI shell, you may need to enter the +motherboard’s setup utility to restore booting from the hard disks. +The Boot menu may be where this is set. You may be able to disable +use of the UEFI Shell, which may eliminate this situation. +
+
+
+
+
+

4. First stage installation

+
+
+

This guide assumes that you have only one disk installed in the +machine initially even if you intend to use a RAID configuration. Use +of a single disk (for a test install, etc.) is also annotated below.

+
+
+ + + + + +
+
Note
+		 +The single dish install approach is used because it is faster +than a dual disk install. It also allows you to control when the +syncing for the second disk occurs, such as when you leave for the +evening. The setup of a second and third disk is covered in the +Setup additional disks subsection below. +
+
+
+
    +
  1. +

    Install your smallest disk in the primary slot (the one connected +to the lowest numbered SATA controller, usually 0.

    +
    + + + + + +
    +
    Caution
    +    		 +For the RAID to work seamlessly with other disks later, you +must make sure that the smallest disk of the ones available is used +for the installation. +
    +
    +
    2. +
+
+
+

4.1. Boot from the installation medium

+
+
    +
  1. +

    Connect an active network cable to your lowest numbered interface +(only). Usually it is on the left if there are two.

    +
    2. +
  2. +

    Insert/plug-in your installation media and reboot.

    +
    +

    To boot from the installation media you may need to bring up your +motherboard’s setup utility, which is typically accessed by pressing +Delete during the POST. From there you may need to access a menu +such as Save & Exit (or Boot), to select overriding to +boot with the installation media.

    +
    +
    + + + + + +
    +
    Tip
    +    		 +If the system was most recently booted from a hard disk, you may +need to boot one time with no hard disk installed for the +motherboard’s setup utility to recognize the USB drive as a valid +boot override option. If the setup utility does not recognize the +USB drive at all, it may be necessary to turn the power off, remove +the USB drive, reinsert it, and then reboot. Making the USB the first +boot device temporarily may be necessary. +
    +
    +
    3. +
+
+
+
+

4.2. Set boot options and boot installer

+
+

At the Installer boot menu:

+
+
+
    +
  1. +

    Highlight Install (or Graphical install — only the installer +interface differs — but this may not work on some video hardware)

    +
    +
      +
    • +

      UEFI: press e, then ↓ three times (vmlinuz), then End

      +
      + + + + + +
      +
      Note
      +      		 +If e doesn’t work, UEFI is not available. It may be possible to enable it in the BIOS. +
      +
      +
      • +
    • +

      BIOS: press Tab

      +
      • +
    +
    +
    2. +
  2. +

    To the end of the displayed command, add the additional options:

    +
    +
    +
    locale=en_US.UTF8 netcfg/disable_dhcp=true time/zone=UTC
    +
    +
    +
    + + + + + +
    +
    Note
    +    		 +Whilst typing a / (slash) it may automatically be changed (escaped) to + \/ (i.e. preceded by a backslash). This is normal behaviour and harmless. +
    +
    +
    3. +
  3. +

    Press:

    +
    +
      +
    • +

      UEFI: F10

      +
      • +
    • +

      BIOS: Enter

      +
      • +
    +
    +
    4. +
+
+
+ + + + + +
+
Note
+		 +You may omit the netcfg/disable_dhcp=true if you want to use DHCP to +configure the network settings of this machine, though this is not advised. +
+
+
+ + + + + +
+
Note
+		 +You can additionally use partman-partitioning/default_label=gpt if you wish +to force the use of a GPT partition table on a disk that is smaller than 2 GB, +but beware - some older BIOS versions cannot handle GPT formatted disks. +
+
+
+ + + + + +
+
Note
+		 +If you do not set a locale or set locale=C, you will be +prompted to select your language and your country. However some +applications may have problems if a UTF8 locale is not used. +
+
+
+

The installer will now boot.

+
+
+
+

4.3. Select a keyboard layout

+
+

Find your keyboard on the Keymap list, highlight it, and press +Enter. (The most common one is American English)

+
+
+

The installation media is now scanned and additional installer +components loaded.

+
+
+
+

4.4. If you are presented with a dialog asking for non-free firmware files

+
+

You may need to locate the files requested (especially if they relate +to your network or disk-drive interfaces) and place them on a USB +stick which should be inserted at this stage. If you do have the +required files select Yes, otherwise press Tab to select No +then press Enter to continue. It may well be simpler just to use +the unofficial installer images mentioned above that include all +available non-free firmware.

+
+
+
+

4.5. Configure the network

+
+
    +
  1. +

    If you are presented with a dialog asking which interface to use as +primary

    +
    +

    This is typically only shown if two or more network interfaces are found, +which might include a virtual FireWire interface in some cases. +Select the interface you require (usually eno1) and press +Enter.

    +
    +
    2. +
+
+
+

Unless you are using DHCP (which is not advisable) you will be +prompted to:

+
+
+
    +
  1. +

    Type in the required static IP address in the form xxx.xxx.xxx.xxx +(where each xxx is any integer from 0 - 255 inclusive) and press +Enter.

    +
    2. +
  2. +

    Type in the required netmask in the form 255.yyy.yyy.yyy (where each +yyy is typically 0, 64, 128, 192 or 255) and press Enter.

    +
    3. +
  3. +

    Type in the required gateway IP address in the form +xxx.xxx.xxx.xxx (where each xxx is any integer from 0 - 255 inclusive) +and press Enter.

    +
    4. +
  4. +

    Type in the required nameserver IP addresses, space separated, in +the form xxx.xxx.xxx.xxx (where each xxx is any integer from 0 - 255 +inclusive) and press Enter.

    +
    5. +
+
+
+
+

4.6. Set a hostname

+
+

Backspace over the default hostname debian and type in the name +you require (if not already retrieved via DNS), then press Enter. +Enter the required Internet Domain name (if not found) and press Enter.

+
+
+
+

4.7. Enter a suitable root password

+
+

Twice as prompted.

+
+
+
+

4.8. Setup first account

+
+

Enter Desktop User for the name of the new user +then press Enter to accept desktop as the username and enter a (real) +password twice as prompted.

+
+
+
+

4.9. Get network time

+
+

The installer now tries to set the time using NTP. If this is not +possible at your site due to your firewall etc., you may need to press +Enter to cancel this process.

+
+
+
+

4.10. Partition the disk

+
+ + + + + +
+
Note
+		 +If you are using UEFI and the disk was previously used for BIOS, you may need +to confirm forcing UEFI installation. +
+
+
+
    +
  1. +

    When prompted for a partitioning method, select Manual

    +
    2. +
+
+
+

4.10.1. Setup physical partitions

+
+
    +
  1. +

    Create a new partition table by:

    +
    +
      +
    1. +

      Select your disk, something like SCSI1 (0,0,0) (sda) - 4 TB ATA +SATA HARDDISK, and press Enter.

      +
      + + + + + +
      +
      Warning
      +      		 +Do not select your installation media. +
      +
      +
      2. +
    2. +

      The installer may warn: You have selected an entire device to +partition…. If so, select Yes. If you are prompted to delete RAID +partitions, select Yes.

      +
      3. +
    +
    +
    2. +
  2. +

    Select the (one and only entry) FREE SPACE under your disk. There +should be no RAID or LVM partitions shown.

    +
    + + + + + +
    +
    Note
    +    		 +
    +

    If other entries and/or RAID or LVM partitions are shown, you will +need to delete them before proceeding.

    +
    +
    +

    If no RAID and/or LVM partitions are shown, a possible solution may be +to delete individual partitions until you have a single entry, FREE +SPACE.

    +
    +
    +

    If that doesn’t work or RAID and/or LVM partitions are shown, you may +be able to use Guided partitioning to delete the existing +configuration (and temporarily create new partitions). In this case, +select Guided partitioning, then select Guided - use entire disk. +Then select your disk, such as listed above, do not select a RAID or +your installation media device. Then select All files in one +partition (recommended for new users). You may be prompted to confirm +deleting RAID partitions and/or removing logical volume data, which +you must do to continue. Then you should be able to continue with +selecting your disk, as above.

    +
    +
    +

    If the Guided partitioning method above doesn’t work or you have +problems later creating the RAID or LVM partitions, then other means +will be needed. There may be more complicated paths through the +partitioner that will work or, perhaps easier, you may need to +overwrite the start of the disk with a large number, say 2 GiB (but +possibly more, if that doesn’t solve the problem), of zeros.

    +
    +
    +

    Overwriting with zeros: can be implemented (for 2 +GiB) at this stage in the installer with:

    +
    +
    +
      +
    1. +

      Press Ctrl+Alt+F2 to switch to a different console.

      +
      2. +
    2. +

      Press Enter to activate the console.

      +
      3. +
    3. +

      Execute:

      +
      +
      +
      dd if=/dev/zero of=/dev/sda bs=1G count=2
+sync;sync
+reboot
      +
      +
      +
      4. +
    4. +

      When the system reboots, restart the installation.

      +
      5. +
    +
    +
    +
    +
    3. +
  3. +

    Select Create a new partition

    +
    4. +
  4. +

    Then for

    +
    +
      +
    • +

      UEFI: Enter 1GB in the size, then select Beginning of the disk.

      +
      • +
    • +

      BIOS: Enter 1MB in the size, choose Primary (rather than Logical) if asked for the partition type, then select Beginning of the disk.

      +
      • +
    +
    +
    5. +
  5. +

    Then for

    +
    +
      +
    • +

      UEFI: Select Use as then select EFI System Partition

      +
      • +
    • +

      BIOS: Select Use as then select Reserved BIOS boot area, or alternatively do not use the partition if the former option is not available.

      +
      • +
    +
    +
    6. +
  6. +

    Now select Done setting up the partition.

    +
    7. +
  7. +

    Next select the FREE SPACE and Create a new partition again.

    +
    + + + + + +
    +
    Note
    +    		 +You may see a small 1MB FREE SPACE at the start of the disk. This is +fine, just be sure to choose the large FREE SPACE at the end of the disk. +
    +
    +
    8. +
  8. +

    This time choose the whole amount of free space (the default) and choose Primary for the partition type if asked.

    +
    9. +
  9. +

    Select Use as, then select physical volume for RAID, then Done +setting up the partition

    +
    + + + + + +
    +
    Note
    +    		 +If you physically only have one disk bay and wish to construct a FSL11 test-bed, +it is possible to avoid using the software RAID layer entirely. Simply select Use as, then select physical volume for LVM +for this partition instead and skip ahead to Setup Logical Volume Manager (LVM) below. +However, please note that a single disk setup is not recommended for any operational system. +
    +
    +
    10. +
+
+
+
+

4.10.2. Setup RAID

+
+
    +
  1. +

    Select Configure software RAID. Then select Yes to write the +changes to the disk.

    +
    2. +
  2. +

    Select Create MD device, choose RAID1 and use 2 as the +number of devices and 0 as the number of spares.

    +
    3. +
  3. +

    Despite the fact that the instructions say you must select exactly +two partitions, select only one. Select the RAID partition you just +created by pressing Space. This should be /dev/sda2. Then +press Enter to continue. Select yes if prompted to write +changes to the disk.

    +
    + + + + + +
    +
    Note
    +    		 +If the newly created RAID partition doesn’t appear as an option, +you may need to use the method of Overwriting with zeros in +the Setup physical partitions step above. +
    +
    +
    4. +
  4. +

    Select Finish.

    +
    5. +
  5. +

    Back in partitioning, select the partition #1 (with no designated use) under +RAID1 device #0 and press Enter

    +
    + + + + + +
    +
    Note
    +    		 +If that partition appears immediately after being created +already having a designated use, perhaps lvm, you may need to use +the method of Overwriting with zeros in the +Setup physical partitions sub-step above. +
    +
    +
    6. +
  6. +

    Select Use as, then select physical volume for LVM, then Done +setting up the partition

    +
    7. +
+
+
+
+

4.10.3. Setup Logical Volume Manager (LVM)

+
+
    +
  1. +

    Now choose Configure the Logical Volume Manager and select Yes +if prompted to write the changes to the disk and keep the current +layout and configure LVM.

    +
    2. +
  2. +

    Choose Create volume group

    +
    3. +
  3. +

    Enter a name appropriate for the machine and group, e.g., vg0, and press Enter

    +
    4. +
  4. +

    Select the raid device md0 (or sda2 if not using RAID) by pressing Space, then press Enter +to continue

    +
    5. +
  5. +

    For each item in the following table run Create logical volume, +select your volume group and assign the corresponding name. Those +marked with * are optional unless you are applying CIS hardening.

    + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Table 3. Logical volumes
    Mount pointLV nameSize

    1

    /var/log/audit

    audit *

    4 G

    2

    /boot

    boot

    1 G

    3

    /home

    home

    4 G

    4

    /var/log

    log *

    4 G

    5

    /

    root

    50 G

    6

    (swap)

    swap

    8 G

    7

    /tmp

    tmp

    50 G

    8

    /var

    var *

    8 G

    9

    /var/tmp

    vartmp *

    8 G

    10

    /usr2

    usr2

    remaining disk space less ~100 GB

    +
    6. +
  6. +

    In the LVM configuration window, select Finish

    +
    7. +
  7. +

    Then for each logical volume in the table except swap, do the following:

    +
    +
      +
    1. +

      Select the partition (e.g., #1) for each LV name (and press Enter)

      +
      2. +
    2. +

      Select Use as and press Enter then select Ext4 journaling file system

      +
      3. +
    3. +

      Select Mount point, press Enter, then select the appropriate mount point from the list or use Enter manually if not there.

      +
      4. +
    4. +

      Select Done setting up this partition

      +
      5. +
    +
    +
    8. +
  8. +

    For the swap logical volume, select Use as then select swap area, followed by Done setting up this partition

    +
    9. +
  9. +

    Back in the partition screen, select Finish partitioning and write changes to +the disk and select Yes to write the changes. For big disks, it may take +a little time to create the ext4 file systems.

    +
    10. +
+
+
+

The Debian base system is now installed from the installation media, which +usually only takes a few minutes.

+
+
+
+
+

4.11. Configure the package manager

+
+

If you started from a netinst CD image, the installer now assumes +you will install only from the network, and jumps straight to the +Choose your Debian archive mirror country part of the +dialogue as detailed below.

+
+
+

If you are using DVD installer you will be prompted to scan additional DVDs. +Scanning the additional DVDs (and obtaining copies of them in the +first place) is entirely optional, and is only useful if you don’t have a +reliable network connection to a suitable Debian mirror and hence would +prefer not to download packages you could get from the DVD.

+
+
+ + + + + +
+
Note
+		 +If you do want to use a mirror in the future, it is better not +to scan any DVDs at this stage and to scan them later during Stage 2 +using apt-cdrom. +
+
+
+

For each additional DVD you wish to scan, insert it in the drive, select +Yes and press Enter to perform the scan (which takes a while.)

+
+
+

(If you are using DVDs, and are prompted to insert another DVD, you +will need to use eject /dev/cdrom from another virtual console to do this)

+
+
+

Select No and press Enter to continue once you are done. +If prompted, insert the “Debian GNU/Linux 11.<x> Bullseye - Official i386/amd64 +Binary-1 DVD” back into the DVD-ROM drive and press Enter.

+
+
+ + + + + +
+
Warning
+		 +If you do scan additional DVDs, the following useful dialogue +which allows you to select a suitable network mirror from a country-based +list may be suppressed. +
+
+
+

Select Yes and press Enter to use a network mirror (unless you +have inadequate Internet access - but then you must scan all DVDs.)

+
+
+

Choose your Debian archive mirror country: +Select from the list if available and press Enter. (If your +country is not available choose the country nearest to you in a +network connectivity sense.)

+
+
+

Select the fastest Debian mirror from those available.

+
+
+ + + + + +
+
Tip
+		 +The new deb.debian.org mirror is a good choice for most +sites as it uses DNS to find a local mirror. +
+
+
+

Enter any necessary HTTP proxy information (usually left blank).

+
+
+

Software is downloaded briefly.

+
+
+
+

4.12. Do not participate in popularity-contest

+
+

When prompted to join the popularity-contest, select No and press Enter

+
+
+
+

4.13. Choose your packages

+
+

When prompted to choose packages, select SSH server by moving to +that row with the arrow keys and pressing Space on it (unless +you don’t want it).

+
+
+ + + + + +
+
Tip
+		 +If you have a small disks and are worried about space, then you can +also press Space on Desktop Environment to unselect it (which may +then change the dialogue presented below). +
+
+
+

Finally press, Enter to install the standard system.

+
+
+

The Debian standard system is now installed from the installation media plus any +updates from the network mirror and/or security.debian.org site if they can be +reached.

+
+
+

This can take a while, up to one and a half hours or more.

+
+
+
+

4.14. Install the GRUB bootloader (BIOS boot only)

+
+ + + + + +
+
Note
+		 +With UEFI boot, you will not be presented with this option; GRUB will automatically be +installed to the first ESP partition. +
+
+
+

At Install GRUB to Master Boot Record select yes then select /dev/sda

+
+
+

When prompted, press Enter to install to the master boot record.

+
+
+
+

4.15. Disable Wayland (optional)

+
+

This step should only be needed if your CPU does not include a GPU and +you do not have an add-on graphics card. In that case, you are using +the motherboard graphics support. Disabling Wayland is known +specifically to be necessary for the X11SCA-F motherboard, which +uses the AST2500 graphics chip. If you don’t know that you need to +disable Wayland, we recommend that you initially leave it enabled. +Whether your choice works or not should be evident when you start the +Second stage installation step below. The console may be very +difficult, even impossible, to work with. In that case, please see the +Wayland recovery NOTE below.

+
+
+

To disable Wayland:

+
+
+ + + + + +
+
Tip
+		 +These instructions step can be executed when the installation +stops for input in the next step, Remove installation media. +
+
+
+
    +
  1. +

    Press Ctrl+Alt+F2 to switch to a different console.

    +
    2. +
  2. +

    Press Enter to activate the console.

    +
    3. +
  3. +

    Edit /target/etc/gdm3/daemon.config, uncomment Wayland=False, +and save the file.

    +
    +

    The only editor available at this point may be nano.

    +
    +
    4. +
  4. +

    Execute:

    +
    +
    +
    sync;sync
+exit
    +
    +
    +
    5. +
  5. +

    Press Ctrl+Alt+F1 to return to the Installer dialog.

    +
    6. +
+
+
+ + + + + +
+
Note
+		 +
+

Wayland recovery: If you find +you have made the wrong choice, there are at least three possible ways +to recover:

+
+
+
    +
  1. +

    If the console is marginally usable, you may be able to login on a +text console to adjust the contents of /etc/gdm3/daemon.config as +needed, then execute:

    +
    +
    +
    systemctl restart gmd3
    +
    +
    +
    2. +
  2. +

    Use the procedure in the Rescue Mode appendix and adjust the +contents of /etc/gdm3/daemon.config as needed.

    +
    3. +
  3. +

    Reinstall from scratch and make the opposite choice.

    +
    4. +
+
+
+
+
+
+

4.16. Remove installation media

+
+

Remove the DVD from the DVD-ROM drive (it should be auto-ejected), or +unplug the USB drive, and press Enter to reboot into the newly +installed system.

+
+
+ + + + + +
+
Tip
+		 +It would generally be wise to disable booting from DVD-ROM and +floppy i.e., anything other than the hard drive, in the BIOS just in +case someone leaves something nasty in the machine’s removable drives +by mistake. +
+
+
+
+
+
+

5. Second stage installation

+
+
+

You should now have booted to your new OS.

+
+
+

5.1. Login as root

+
+ + + + + +
+
Tip
+		 +Versions before Debian 9 ran X11 on virtual console 7. As of +Debian 9, the graphical environment login is on virtual console 1. +Each login there for a different user creates a session on the next +unused virtual console. +
+
+
+

Switch to Virtual Console 2, by pressing Ctrl+Alt+F2.

+
+
+

Enter root and press Enter, then enter the root password you set +earlier.

+
+
+
+

5.2. Remove the dummy Desktop User (optional)

+
+

Unless you want an account that is set up to use the default desktop +environment, delete the desktop user with:

+
+
+
+
deluser --remove-home desktop
+
+
+
+ + + + + +
+
Note
+		 +If you do keep this account, you will not be able to run the FS from +it unless you add this account into the additional hardware access groups +such as is done for oper and prog by fsadapt. +
+
+
+
+

5.3. Setup HTTP proxy for APT (optional)

+
+

Should you wish to make APT use an HTTP proxy for downloads, +create the new file /etc/apt/apt.conf.d/00proxies using vi containing:

+
+
+
+
ACQUIRE::http::Proxy "http://proxy.some.where:8080/";
+
+
+
+

to use a proxy proxy.some.where at port 8080 for example.

+
+
+
+

5.4. Edit /etc/apt/sources.list

+
+

Using your favourite text editor, eg vi, and comment out all cdrom +entries (unless you don’t have a decent Internet connection and need +to use DVDs, whereupon the dialogue presented below may differ) and +check you have the equivalent of the following entries towards the top +of the file, adding in contrib and/or non-free as needed:

+
+
+
+
deb http://deb.debian.org/debian/ bullseye main contrib non-free
+deb-src http://deb.debian.org/debian/ bullseye main contrib non-free
+
+
+
+

and likewise the equivalent of the following entries towards the +bottom of the file, again adding in contrib and/or non-free as +needed:

+
+
+
+
deb http://deb.debian.org/debian/ bullseye-updates main contrib non-free
+deb-src http://deb.debian.org/debian/ bullseye-updates main contrib non-free
+
+
+
+

(where you can use any suitable mirror instead of deb.debian.org)

+
+
+

Also add contrib and/or non-free to the lines referring to the +security.debian.org mirror in the middle of the file.

+
+
+ + + + + +
+
Warning
+		 +you MUST use bullseye and NOT stable for the +distribution in all these entries (but CD/DVD entries might use +unstable.) +
+
+
+
+

5.5. Update APT’s list of packages

+
+ + + + + +
+
Tip
+		 +Recent versions of Debian have the apt program, which gives a more + user-friendly interface to the package manager than apt-get. We + generally use apt-get except for applying updates. +
+
+
+

Next tell APT to update its internal source list of packages using

+
+
+
+
apt-get update
+
+
+
+ + + + + +
+
Note
+		 +It is also possible to add additional DVDs at this stage using the +apt-cdrom add command. +
+
+
+
+

5.6. Download the FS Linux 11 package selections

+
+
    +
  1. +

    Install git and dselect

    +
    +
    +
    apt-get install git dselect
    +
    +
    +
    2. +
  2. +

    Update dselect's package lists

    +
    +
    +
    dselect update
    +
    +
    +
    3. +
  3. +

    Get the selections by downloading this repository:

    +
    +
    +
    cd /root
+git clone https://github.com/nvi-inc/fsl11
+cd fsl11
    +
    +
    +
    4. +
  4. +

    Feed the package selections into dpkg using the command, for amd64

    +
    +
    +
    dpkg --set-selections < selections/fsl11_amd64.selections
    +
    +
    +
    +

    or, for i386

    +
    +
    +
    +
    dpkg --set-selections < selections/fsl11_i386.selections
    +
    +
    +
    5. +
  5. +

    Start the additional package installation with

    +
    +
    +
    apt-get dselect-upgrade
    +
    +
    +
    +

    then press Enter to confirm any updating of installed packages +(where you have an Internet connection) and the installation of +currently ~191 new packages (downloading ~185 MB from the Internet +and/or DVDs) for amd64 with UEFI — probably different for i386 +and/or BIOS — unless you did not select the Desktop or added other +tasks earlier.

    +
    +
    +

    Downloading commences for up to half an hour (depending on your Internet +access and the exact revision of DVDs used).

    +
    +
    +

    Installation runs to completion.

    +
    +
    6. +
+
+
+
+

5.7. Clean up the APT download directory

+
+

So that the update mechanism will work correctly, run

+
+
+
+
apt-get clean
+
+
+
+
+
+
+

6. Third stage installation

+
+
+

6.1. fsadapt

+
+

In the /root/fsl11 directory, start fsadapt with

+
+
+
+
./fsadapt
+
+
+
+

6.1.1. FS Adaptation: Modifications (Window 1)

+
+

Using the arrow keys and Space make your selections and press Enter.

+
+
+
    +
  • +

    If you are not using a GPIB board or USB dongle, you can deselect +the GPIB option.

    +
    • +
  • +

    If you are using the RAID configuration, you must not deselect +the mdinc option.

    +
    • +
+
+
+
+

6.1.2. FS Adaptation: Setup (Window 2)

+
+

All of the steps in Window 2 need to be done once (even if you do not +intend to use the serial ports) with the exception of sshkeys which +can be used to generate new SSH keys if required. +If you did not select the GPIB option in the previous page deselect the +two related options on this page (but do not deselect set_perms as it +is always required). Otherwise, simply press Enter with the OK +selected to continue.

+
+
+ + + + + +
+
Note
+		 +The updates option relies on email to root being re-directed +to some mailbox that will be read regularly, so make sure you set that +up and test it as well (see the Configure e-mail section in the +Additional Setup Items appendix). The installer sets it up to go +the desktop account by default which would definitely be a problem +if you have removed that! +
+
+
+
+

6.1.3. GPIB driver configuration (optional)

+
+

On the /etc/gpib.conf screen, use the up/down arrow keys to select the +required GPIB controller and press Enter on OK to continue.

+
+
+
+

6.1.4. Serial port configuration

+
+

On the /etc/default/grub: serial port configuration screen +up/down arrow keys to select the required RS232 serial card +(or None if you don’t have one) and press Enter on OK +to continue.

+
+
+
+

6.1.5. FS Adaptation: Settings (Window 3)

+
+

On Window 3 you can choose to modify the email or network settings if required. +Simply press Enter on OK to continue.

+
+
+
+

6.1.6. FS Adaptation: Network Services (Window 4)

+
+

The Window 4 will show what services are enabled. Use the up/down +arrows and Space to select secure and press Enter on +OK. Thereafter use the up/down arrows and Space to select +those services you actually need. If you need printing, you will need +to select netipp (remote access to this can be blocked by + configuring ufw with either not explicitly allowing or instead + denying the CUPS service). Press Enter on OK to set them +up and finish with fsadapt.

+
+
+

Note that the fsadapt script can be re-run at a later date should you need to +change the adaptations.

+
+
+
+
+

6.2. Set passwords

+
+

Set passwords for the oper and prog accounts with:

+
+
+
+
passwd oper
+passwd prog
+
+
+
+

entering the passwords twice as prompted.

+
+
+
+

6.3. Install tools for RAID (optional)

+
+

You can install some useful tools for working with the RAID, if you’re actually using it, with:

+
+
+
+
~/fsl11/RAID/install_tools
+
+
+
+

The rest of this document assumes the first three of these tools have +been installed. The six tools are:

+
+
+
    +
  • +

    mdstat — for all users — check on the RAID status

    +
    • +
  • +

    refresh_secondary — for root — refresh a secondary disk that +is from the same RAID

    +
    • +
  • +

    blank_secondary — for root — initialize a secondary disk, +must be used with extreme care

    +
    • +
  • +

    rotation_shutdown — for root — shutdown the system if it is +safe to rotate disks

    +
    • +
  • +

    drop_primary — for root — deliberately drop the primary disk +out of the RAID for use as a backup

    +
    • +
  • +

    recover_raid — for root — re-add a disk that fell out of (or +was removed from) the RAID back into it

    +
    • +
+
+
+ + + + + +
+
Tip
+		 +More information about RAID operation can be found in the +RAID notes for FSL11 document. +
+
+
+
+

6.4. Download the Field System

+
+
+
 cd /usr2
+ git clone https://github.com/nvi-inc/fs fs-git
+ cd /usr2/fs-git
+ git checkout -q tag
+
+
+
+

where tag is the latest available release, 10.2.0 or later.

+
+
+ + + + + +
+
Tip
+		 +
+

At the time of this writing, 10.2.0 has not been officially +released. The tag for the latest beta version is currently +10.2.0-beta2. That or a later tagged beta pre-release, should +suffice for an initial installation. However they should not be used +for operations. You should update to 10.2.0 as soon as it is +available. If 10.2.0 has not been officially released, you should +ignore the IMPORTANT block immediately below.

+
+
+

You can find the most recent beta release of 10.2 at:

+
+
+

https://github.com/nvi-inc/fs/releases

+
+
+

If none are available, you can find the most recent tagged alpha +version of 10.2 at:

+
+
+

https://github.com/nvi-inc/fs/tags

+
+
+
+
+ + + + + +
+
Important
+		 +
+

You should install the latest official release. To find it, go to:

+
+
+

https://github.com/nvi-inc/fs/releases

+
+
+

You should probably use the most recent feature release (ending in +.0 with no trailing -<string>, e.g., 10.2.0. However, if there +is a more recent patch release (not ending .0) for the most recent +feature release, you should use the most recent patch release. For +example, if 10.2.0 is the most recent feature release and there are +corresponding patch releases, 10.2.1 and 10.2.2, then the last +one, ending .2, is probably the best choice.

+
+
+
+
+
+

6.5. Run FS install script

+
+

This will set the /usr2/fs link, set /usr2/fs-git permissions, and +install default copies of all the FS related directories.

+
+
+
+
make install
+
+
+
+

and enter y to confirm installation.

+
+
+
+

6.6. Make the FS

+
+

The FS must always be compiled as prog.

+
+
+ + + + + +
+
Warning
+		 +Make sure you log-out as root, and log-in again as prog. +
+
+
+
+
cd /usr2/fs
+make >& /dev/null
+
+
+
+

then

+
+
+
+
make -s
+
+
+
+

to confirm that everything compiled correctly (no news is good news).

+
+
+
+

6.7. Reboot the new system

+
+

Remove any DVD from the machine and restart the machine using +reboot as root or Ctrl+Alt+Del whilst watching that +everything starts up smoothly.

+
+
+
+
+
+

7. Fourth stage Installation

+
+
+

7.1. Setup additional disks

+
+

If your are using a RAID, follow the steps in this subsection to setup +the second and third disks.

+
+
+ + + + + +
+
Note
+		 +Additional disks should be at least as large as the disk already +in use. +
+
+
+ + + + + +
+
Note
+		 +You will need to have hot-swapping enabled in your motherboard’s +setup menu, at least for the controller for the secondary disk (it +should also be enabled for the primary). +
+
+
+ + + + + +
+
Note
+		 +This subsection assumes you have installed the RAID tools +according to the Install tools for RAID (optional) subsection +above. +
+
+
+
    +
  1. +

    If you have a second disk (secondary) in the RAID:

    +
    +
      +
    1. +

      Shut the system down with the rotation_shutdown command.

      +
      +

      This command will check the status of the RAID and proceed to shutting +down only if the RAID is synced. There are three errors that can +prevent shutting down: (i) if the FS is running, you should terminate +it before trying again; (ii) if the RAID is recovering, you will +need to wait until the recovery is finished before shutting down, you +can check the progress with the mdstat command; and (iii) if the +RAID is degraded, seek expert advice.

      +
      +
      2. +
    2. +

      Remove the disk in the primary slot and place it on the shelf, +labelled appropriately as the shelf disk for this system with the +date.

      +
      3. +
    3. +

      Move the disk in the secondary slot to the primary slot.

      +
      4. +
    +
    +
    2. +
  2. +

    Initialize the new disk

    +
    + + + + + +
    +
    Important
    +    		 +Do not initialize a disk unless you are sure there is no +data on it that you need to preserve. +
    +
    +
    +

    For the first time use of an additional disk with a new install, the +disk should be initialized to make sure it has no existing structure. +This should be done even if the disk has been used in a different FS +computer or a previous install on this computer.

    +
    +
    +
      +
    1. +

      Boot with just the primary disk installed.

      +
      + + + + + +
      +
      Tip
      +      		 +If your system is already running with no second disk +(secondary) installed, you can skip rebooting. +
      +
      +
      2. +
    2. +

      Use the script:

      +
      +
      +
      blank_secondary
      +
      +
      +
      +

      The script will wait for the new disk to be turned on. Insert a new +disk in the secondary slot. The secondary slot is the one connected +to second lowest numbered SATA controller, usually 1. Turn the key +to turn the disk on. There will be a prompt asking if wish to proceed. +If it is a new disk or you are sure it safe to erase this disk, answer +y. If you are unsure about this or otherwise need to abort, answer +n.

      +
      +
      3. +
    +
    +
    3. +
  3. +

    Refresh the now blank secondary disk

    +
    +

    Run the script:

    +
    +
    +
    +
    refresh_secondary
    +
    +
    +
    +

    Once you reach the message that you can check on the recovery with +mdstat , you can resume using the computer as usual. You can safely +reboot at this point, if it is needed; just don’t remove either disk +until the recovery is finished.

    +
    +
    +

    You can check the progress of the recovery with:

    +
    +
    +
    +
    mdstat
    +
    +
    +
    +

    When the recovery is complete, you can repeat the process of this +entire subsection, Setup additional disks, to initialize another +disk.

    +
    +
    4. +
+
+
+
+
+
+

8. Post install

+
+
+

The FS on your newly installed system should now be ready to be +customized for your site’s requirements by tailoring the control files +in /usr2/control and adding suitable station specific software to +/usr2/st. See the files in the /usr2/fs/st.default/st-0.0.0 +directory for starter versions of the latter.

+
+
+

Please refer to the appendix Additional Setup Items for OS +customizations that you may find useful.

+
+
+
+
+

Appendix A: Additional Setup Items

+
+
+

This appendix covers several customizations that may be helpful +depending on the requirements for a system. It serves as a reference +for how to make these changes, but can also be helpful as a checklist +when setting up a new system. All actions in this section require +root permissions.

+
+
+

A.1. Additional security and CIS Benchmarks

+
+

For stations that wish to conform to the additional security +recommendations of the Center for Internet Security (CIS), move on to +the CIS hardening FSL11 document.

+
+
+

A.1.1. Alternate hardening

+
+

If you don’t want the complete CIS hardening, which creates some +inconveniences and is only required in certain environments, you may +still be interested in applying a subset of the remediations. You can +pick and choose those from the CIS hardening FSL11 +document and its script.

+
+
+

A useful minimum set of features to apply would be to install ufw +and block everything except ssh and further restrict ssh access with +TCP Wrappers.

+
+
+
A.1.1.1. ufw setup
+
+

To install and configure ufw to only allow ssh for incoming +connections, use the commands:

+
+
+
+
apt-get -y install ufw
+ufw allow OpenSSH
+ufw --force enable
+
+
+
+

Addition setup for ufw is covered below in the +More firewall rules subsection.

+
+
+
+
A.1.1.2. TCP Wrappers setup
+
+

A base setup for TCP Wrappers is

+
+
+
/etc/hosts.deny
+
+
ALL:ALL
+
+
+
+
/etc/hosts.allow
+
+
sshd:ALL
+
+
+
+

It is recommend that you further restrict sshd by using specific +hosts and/or sub-domains instead of ALL. Please use +man hosts_access for more information about configuring TCP +Wrappers

+
+
+
+
A.1.1.3. More firewall rules
+
+

The following tersely summarizes some ufw settings that may be +useful:

+
+
+
+
#SSH
+ufw allow OpenSSH
+#NTP
+ufw allow ntp
+#remote access to metserver (or gromet) on port 50001
+ufw allow 50001
+#anywhere from subnet
+ufw allow from 192.168.4.0/24
+#RDBE multicast to addresses from subnet
+ufw allow in proto udp to 239.0.2.0/24 from 192.168.4.0/24
+#? RDBE multicast to group from subnet ?
+#ufw allow in proto igmp to 239.0.2.0/24 from 192.168.4.0/24
+
+
+
+
+
+
+

A.2. Customize root’s .bashrc file

+
+

There are a few changes you should consider for root's .bashrc file.

+
+
+
    +
  1. +

    If you have applied the CIS remediations, you should consider +uncommenting the line that sets the umask to 022. The remediations +set it to 027 in /etc/profile, which may cause problems with +routinely created files, including some in this section covering optional changes.

    +
    2. +
  2. +

    Uncomment the the alias commands that add the -i option to the +commands cp, mv, and rm as the default. This can help avoid +some careless errors.

    +
    3. +
  3. +

    Add the command set -o noclobber to avoid accidently overwriting +existing files with I/O redirection. Other options to consider setting +are physical and ignoreeof.

    +
    4. +
+
+
+
+

A.3. Create root’s .inputrc file

+
+

The readline package is used by bash, and other programs, to +maintain a history of commands that can be edited and then +re-executed. By default, it will retain edits of history entries that +have not been re-executed. This makes the unedited history entries +more difficult to locate and re-execute. Retaining the un-executed +edits can be disabled for root by creating the file:

+
+
+
/root/.inputrc
+
+
$include /etc/inputrc
+set revert-all-at-newline on
+
+
+
+

The $include /etc/inputrc line preserves the other system wide +readline defaults.

+
+
+ + + + + +
+
Note
+		 +The standard fresh FS installation creates this file for the +oper and prog (and AUID) accounts. +
+
+
+
+

A.4. Setup /etc/hosts

+
+

You may want to add more hosts to the /etc/hosts, especially if do +not have DNS. This will allow you to give a short alias to use when +referring to other local machines. Even if you have DNS, you may wish +to add additional aliases for your local hosts.

+
+
+

For use with ntpq -p, is recommended that you use a short alias as +the canonical name (the first one after the IP address) for other +local machines (and possibly remote ones as well). This will make the +ntpq output easier to understand, particularly if the canonical +names of the local machines only differ at the end of their names. +That may make the differences hard to see given the short field +available for the remote node ID in the ntpq output.

+
+
+
+

A.5. Stabilize network configuration

+
+

This subsection requires using nm-connection-editor on a graphic +display (nmtui may be an option on a text terminal, but it has not +been fully verified). You may need to be root or desktop to do +this. All the subsections below assume you are in the program and have +sufficient permissions.

+
+
+ + + + + +
+
Note
+		 +If you someday move the disks to a computer with a different +mainboard model, the device names of the network interfaces may +change. If that happens, you will need to reselect the names as +described in the sub-steps below. This should not be necessary if the +other computer uses the same mainboard. +
+
+
+

A.5.1. Make the connection always appear on the same interface regardless of the MAC address.

+
+

This is useful both to make the connection appear on only one +interface and/or make it the same interface if the computer (or NIC) +is changed.

+
+
+
    +
  1. +

    Select your connection and click the “gear” icon.

    +
    2. +
  2. +

    Select the Ethernet tab.

    +
    3. +
  3. +

    Use the drop-down for the Device field to select your device +(typically eno1 with the MAC address in parentheses). Then edit the +field to just list the name of the interface (typically eno1) by +removing the MAC address in parentheses.

    +
    4. +
  4. +

    You may want to also set the IPv6 Settings to use Method: +Disabled.

    +
    5. +
  5. +

    Click Save.

    +
    6. +
  6. +

    Close the window by pressing Esc (while the focus is on that +window).

    +
    7. +
+
+
+
+

A.5.2. Disable the second Ethernet port

+
+

This may be useful, for example, if your second port has a IPMI +interface and the kernel detected a connection there and it is +interfering with the normal or the IPMI connection.

+
+
+
    +
  1. +

    If there is no Wired connection 2, click the + icon. Otherwise +select that connection, click the “gear” icon, and skip to step 4. +It may be benign to delete (- icon) any other connections except +Wired connection 1.

    +
    2. +
  2. +

    Make sure Ethernet is selected in the drop down box and click +Create…​.

    +
    3. +
  3. +

    Change the Connection name to Wired connection 2.

    +
    4. +
  4. +

    Select the Ethernet tab.

    +
    5. +
  5. +

    Use the drop-down for the Device field to select your device +(typically eno2 with the MAC address in parentheses). Then edit the +field to just list the name of the interface (typically eno2) by +removing the MAC address in parentheses.

    +
    6. +
  6. +

    Select the IPv4 Settings tab.

    +
    7. +
  7. +

    For Method select Disabled.

    +
    8. +
  8. +

    Select the IPv6 Settings tab.

    +
    9. +
  9. +

    For Method select Disabled.

    +
    10. +
  10. +

    Click Save.

    +
    11. +
  11. +

    Close the window by pressing Esc (while the focus is on that +window).

    +
    12. +
+
+
+
+
+

A.6. Disable Desktop User

+
+

If you do not need the functionality available in the Desktop +environment, you can disable the desktop account. You can re-enable +the account later if you need it. To disable it, execute:

+
+
+
+
usermod -L desktop
+
+
+
+

You can undo this by using the -U option instead.

+
+
+

To prevent connecting with ssh using a key, create (or add desktop +to an existing) DenyUsers line in /etc/ssh/sshd_config:

+
+
+
+
DenyUsers desktop
+
+
+
+

And restart sshd with:

+
+
+
+
systemctl restart sshd
+
+
+
+

You can undo the ssh block be removing the line (if it only has +desktop) or removing desktop from the line and then restarting +sshd.

+
+
+
+

A.7. Remove ModemManager package

+
+

If you use serial ports, it is strongly advised that you remove the +ModemManager package to avoid conflicts over access to the ports. +Execute this command:

+
+
+
+
apt-get purge modemmanager
+
+
+
+
+

A.8. Remove anacron package

+
+

If you enabled the weekly update job in fsadapt (it is strongly +recommended), we recommend that you also remove the anacron package +so that the job will run at a fixed time every week, even if the +system is turned off for some periods of time. Execute this +command:

+
+
+
+
apt-get purge anacron
+
+
+
+
+

A.9. Configure e-mail

+
+

The configuration described here (Internet site or mail sent by +smarthost in the exim4 configuration, no incoming mail, reply-to +filter, and modified user names), provides good support for system +messages and the FS msg and rdbemsg utilities.

+
+
+
    +
  1. +

    As root, enter:

    +
    +
    +
    dpkg-reconfigure exim4-config
    +
    +
    +
    +

    to change the setup. Typically you should select internet site, use +your host name in place of debian when it occurs, and otherwise +select defaults at all the other prompts. (The only other recommended +choices are local delivery only or mail sent by smarthost; received +via SMTP or fetchmail.) If you want to receive incoming mail, you +will also need to enable SMTP connections in Window 4 of fsadapt +(and if you are using a firewall, you will need to enable such +connections for it). We recommend that you NOT receive incoming mail +on this computer.

    +
    +
    2. +
  2. +

    Reply-To filter: If you follow the +recommendation not to receive incoming mail and your system is not +setup for local delivery only, you should set the Reply-To address +for outgoing messages to a real e-mail account at your institution +that is read regularly. You can do this by (all as root):

    +
    +
      +
    1. +

      Create the filter (four lines in file):

      +
      +
      +
      cat >/etc/exim4/reply-to-filter <<EOF
+# Exim filter          << THIS LINE REQUIRED
+
+headers remove "Reply-To"
+headers add "Reply-To: email@address"
+EOF
      +
      +
      +
      +

      Change email@address to the e-mail address you want replies to be +addressed to. If you want more than one, separate them with commas.

      +
      +
      2. +
    2. +

      Create a file for local customizations:

      +
      +
      +
      touch /etc/exim4/conf.d/main/00-exim-localmacros
+ln -sfn /etc/exim4/conf.d/main/00-exim-localmacros /etc/exim4/exim4.conf.localmacros
      +
      +
      +
      + + + + + +
      +
      Note
      +      		 +The file is constructed this way so that it will work for both +non-split or split exim4 configurations. +
      +
      +
      3. +
    3. +

      Add a call to the filter to /etc/exim4/exim4.conf.localmacros:

      +
      +
      +
      cat >>/etc/exim4/exim4.conf.localmacros <<EOF
+#set reply to
+system_filter = /etc/exim4/reply-to-filter
+EOF
      +
      +
      +
      4. +
    4. +

      Then execute

      +
      +
      +
      update-exim4.conf
+systemctl restart exim4
      +
      +
      +
      5. +
    +
    +
    3. +
  3. +

    You should change your /etc/aliases so root and prog e-mail goes to oper.

    +
    +
    +
    +
      +
    • +

      change root: desktop to root: oper

      +
      • +
    • +

      add prog: oper

      +
      • +
    • +

      add desktop: oper

      +
      • +
    +
    +
    +
    +
    +

    This is recommended as a “catch all” since the oper account is +presumably under regular use and any messages sent there are likely to +be noticed. This is particularly important for system error messages +since they should be delivered to a mail box on the system in case +there is a network problem that might prevent them from being +delivered off system. You can however add additional off machine +delivery of these messages to whatever addressees you wish and we +recommend this as well. These should include an e-mail account at your +institution that is read regularly (maybe the same address as the +Reply-To address you may have set above would be a good choice). To +do this, create a .forward file in oper's home directory. The +permissions should be -rw-r—​r--. The contents should be similar to +(left justified):

    +
    +
    +
    +
    \oper
+user@node.domain
    +
    +
    +
    +

    where user@node.domain is the off machine addressee you +want the messages to go to. You can add additional lines for +additional addressees. The backslash (\) before oper +prevents the mail system from getting into an infinite loop +re-checking oper's .forward file.

    +
    +
    4. +
  4. +

    If you have made the above changes to forward messages to another an +e-mail account on another machine, you should customize the User Name +(not login name, the User Name is the fifth field) of root, prog, + oper, and desktop in /etc/passwd to identify the source of the + message. For root and prog, it is recommended to append a + string like at node (it is probably best to avoid FQDNs), where + node is this machine, e.g., for atri you might change the 5th + field for root from

    +
    +
    +
    root
    +
    +
    +
    +

    to

    +
    +
    +
    +
    root at atri
    +
    +
    +
    +

    For oper, you might instead prepend your site name to the +accounts for clearer reading in ops e-mail messages, e.g., +for oper on atri at GSFC, we changed the 5th field for +oper to:

    +
    +
    +
    +
    GSFC VLBI Operator
    +
    +
    +
    +

    and for completeness, for prog and desktop we use:

    +
    +
    +
    +
    GSFC VLBI Programmer
+GSFC Desktop User
    +
    +
    +
    +

    These changes will help the recipient (possibly you) +determine which system generated this message since it may +not be obvious given the modified return address.

    +
    +
    5. +
  5. +

    To give oper an indication at login that there is mail to read, add +either (to get a count of messages):

    +
    +
    +
    test ! -f /var/mail/oper || from -c
    +
    +
    +
    +

    or (to see the senders and subjects):

    +
    +
    +
    +
    test ! -f /var/mail/oper || from
    +
    +
    +
    +

    to end of oper's .profile file (if using bash as the login +shell) or .login file (tcsh).

    +
    +
    6. +
  6. +

    Lastly, check the default mailbox directory /var/mail/ for +accounts that may have messages that arrived before the e-mail +system was fully configured. Be sure to resolve any system +messages that may have been received. You can check to see what +accounts have mail with:

    +
    +
    +
    ls /var/mail
    +
    +
    +
    +

    which will list each user account mail file that +exists. Check and clear each user’s mailbox (where user in +the line below is the account name) that has received mail +(as root):

    +
    +
    +
    +
    mail -f /var/mail/user
    +
    +
    +
    +

    If there are messages in the desktop user’s mailbox that you want to +preserve and oper's mailbox is empty or non-existent, you could +consider renaming desktop's mailbox to be oper's. If you do so, be +sure to change the owner of the file to be oper.

    +
    +
    7. +
+
+
+
+

A.10. Generate FQDN in HELO for outgoing mail

+
+

If mail from your system is being rejected by some servers because +exim4 is not providing a Fully Qualified Domain Name (FQDN), in its HELO +message, the following steps should fix the problem.

+
+
+
    +
  1. +

    If you have not already created +/etc/exim4/conf.d/main/00-exim-localmacros (see +Reply-To filter above), do so:

    +
    +
    +
    touch /etc/exim4/conf.d/main/00-exim-localmacros
+ln -sfn /etc/exim4/conf.d/main/00-exim-localmacros /etc/exim4/exim4.conf.localmacros
    +
    +
    +
    2. +
  2. +

    Add the necessary line to the file:

    +
    +
    +
    cat >>/etc/exim4/exim4.conf.localmacros <<EOF
+MAIN_HARDCODE_PRIMARY_HOSTNAME=ETC_MAILNAME
+EOF
    +
    +
    +
    3. +
  3. +

    Then execute:

    +
    +
    +
    update-exim4.conf
+systemctl restart exim4
    +
    +
    +
    4. +
  4. +

    Verify that the change has taken effect:

    +
    +
    +
    exim4 -bP primary_hostname
    +
    +
    +
    5. +
+
+
+
+

A.11. Set X display resolution at boot

+
+

If your display sometimes starts with the wrong resolution, you may be +able to configure a better resolution. The following is a description +of something that worked for at least one system. The details of your +system may require some changes (beyond the resolution and output name).

+
+
+

First you need to determine the correct resolution and output name. +You may be able to do this with xrandr. If the screen currently has +the correct resolution, you can just execute:

+
+
+
+
xrandr
+
+
+
+

The output might look like:

+
+
+
+
Screen 0: minimum 320 x 200, current 1920 x 1200, maximum 1920 x 2048
+VGA-1 connected primary 1920x1200+0+0 (normal left inverted right x axis y axis) 0mm x 0mm
+   1024x768      60.00
+   800x600       60.32    56.25
+   640x480       59.94
+  1920x1200 (0x42) 154.000MHz +HSync -VSync
+        h: width  1920 start 1968 end 2000 total 2080 skew    0 clock  74.04KHz
+        v: height 1200 start 1203 end 1209 total 1235           clock  59.95Hz
+
+
+
+

Where the current screen resolution is 1920x1200 and the output name is VGA-1.

+
+
+

You can then generate the needed Modeline by executing:

+
+
+
+
cvt 1920 1200
+
+
+
+

Which might generate output:

+
+
+
+
# 1920x1200 59.88 Hz (CVT 2.30MA) hsync: 74.56 kHz; pclk: 193.25 MHz
+ Modeline "1920x1200_60.00"  193.25  1920 2056 2256 2592  1200 1203 1209 1245 -hsync +vsync
+
+
+
+

As a test, you can make a script (use an appropriate name), that will +enable that resolution. Use the output name (VGA-1 in this example) +and the tokens following Modeline from above. There are three lines +after the #!/bin/bash line.

+
+
+
~/display_1920x1200
+
+
#!/bin/bash
+xrandr --newmode "1920x1200_60.00"  193.25  1920 2056 2256 2592  1200 1203 1209 1245 -hsync +vsync
+xrandr --addmode VGA-1 1920x1200_60.00
+xrandr --output VGA-1 --mode "1920x1200_60.00"
+
+
+
+

Be sure to chmod u+x the file before executing.

+
+
+

If that is successful, you can use output name (VGA-1 in this +example) and Modeline from above to make a file (you may need to create + the directory first):

+
+
+
/etc/X11/xorg.conf.d/10-monitor.conf
+
+
Section "Monitor"
+Identifier     "VGA-1"
+Option         "Enable" "true"
+Modeline "1920x1200_60.00"  193.25  1920 2056 2256 2592  1200 1203 1209 1245 -hsync +vsync
+EndSection
+
+Section "Screen"
+Identifier     "Screen0"
+Device         "Device0"
+Monitor        "VGA-1"
+DefaultDepth    24
+#Option         "TwinView" "0"
+SubSection "Display"
+    Depth          24
+    Modes          "1920x1200_60.00"
+EndSubSection
+EndSection
+
+
+
+

You should chmod the permissions for directory with o+rx and the +file with o+r, if those are not already set.

+
+
+

You could then try restarting the display (after closing all windows) with:

+
+
+
+
systemctl restart gdm3
+
+
+
+

or rebooting.

+
+
+
+

A.12. Use KeepAlive to prevent VLAN firewall inactivity time-out

+
+

If there is a VLAN firewall in use on the local network, it may be +necessary to use KeepAlive for TCP connections to prevent inactivity +time-outs for network connections from the FS to the VLBI equipment +when no activity is occurring with the system. For some devices, having +the time-out break the connection may cause an issue with the number of +connections available.

+
+
+

To use KeepAlive to prevent the inactivity time-outs, first install +the package libkeepalive0:

+
+
+
+
apt-get install libkeepalive0
+
+
+
+

Then add the follow lines for oper (and prog):

+
+
+
~/.profile
+
+
export KEEPCNT=20
+export KEEPIDLE=180
+export KEEPINTVL=60
+
+
+
+

Then add the following alias for oper (and prog):

+
+
+
~/.bash_aliases
+
+
alias fs='LD_PRELOAD=libkeepalive.so fs'
+
+
+
+

You will need to terminate the FS, log out, and log back in to activate these changes.

+
+
+ + + + + +
+
Note
+		 +If you run the FS from a script, you will need to include the +setting of LD_PRELOAD explicitly in the script since scripts do not +pick up aliases. +
+
+
+

A similar alias can used to allow other individual applications +to avoid the inactivity time-outs. (A better +solution is available for ssh, discussed below.) It is also possible to put +export LD_PRELOAD=libkeepalive.so in ~/.profile to enable it for all +applications, but this may generate some error messages (in the case of +xterm at least, the error is apparently benign).

+
+
+

If you need to have a persistent ssh connection, add the follow for oper (and prog):

+
+
+
~/.ssh/config file:
+
+
Host *
+    ServerAliveInterval 200
+    ServerAliveCountMax 2
+
+
+
+

This can be set selectively per remote system. The interval of 200 +seconds is chosen to be less than the 300 seconds that some (possibly +security hardened) servers may use.

+
+
+

If not already set correctly, set the ~/.ssh/config file’s +permissions and ownership for oper (analogously for prog) with:

+
+
+
+
chmod 644 ~oper/.ssh/config
+chown oper.rtx ~oper/.ssh/config
+
+
+
+
+

A.13. Remove login banners for commands run by ssh on remote systems

+
+

If you use ssh as oper (and maybe prog), to run commands on +other systems as part of FS operations, you may get login banners +mixed in with the output. You can suppress the banners by adding the +following for oper (and analogously for prog):

+
+
+
~/.ssh/config file:
+
+
Host *
+    LogLevel ERROR
+
+
+
+

This will allow errors to be displayed while suppressing the login +banners of remote systems. This can be set selectively per remote +system.

+
+
+

Please check the end of the Use KeepAlive to prevent VLAN firewall inactivity time-out +section for setting the ownership and permissions on ~/.ssh/config.

+
+
+
+

A.14. Suspend, shutdown, and restart issues

+
+
    +
  1. +

    Mouse cursor disappearing on text console after suspend

    +
    +

    The FSL11 installation disables suspend by default (as part of the +greeter item in the FS Adaptation: Setup (Window 2) sub-step of +fsadapt in the Third stage installation). If you did not +disable suspend, you may encounter this issue. A way to fix it is to +switch to a different text console and then back again. The cursor +should reappear.

    +
    +
    2. +
  2. +

    Disable the power switch from shutting the system down

    +
    +
      +
    1. +

      Add the following to the /etc/gdm3/greeter.dconf-defaults file:

      +
      +
      +
      # Disable restart buttons
+disable-restart-buttons=true
      +
      +
      +
      2. +
    2. +

      Restart gdm3:

      +
      +
      +
      systemctl restart gdm3
      +
      +
      +
      3. +
    +
    +
    3. +
  3. +

    Disable use of restart for ordinary users

    +
    +

    It is possible to disable all use of restart for ordinary users with a +bit more work — the details are available on request. The file +powerlock.tar.gz may be helpful for this. It contains sample +contents of the files that need to be changed or created.

    +
    +
    4. +
+
+
+
+

A.15. Printer setup

+
+
    +
  1. +

    Make sure your printer is connected, to the computer or the network, as appropriate.

    +
    + + + + + +
    +
    Tip
    +    		 +Newer computers usually do not have a parallel port +(IEEE 1284). If not, and your printer requires a +parallel connection, you should be able to obtain a +USB/Parallel converter for less than US$20. +
    +
    +
    2. +
  2. +

    Login in to the X-display or remotely using an X-capable display.

    +
    3. +
  3. +

    Start firefox

    +
    4. +
  4. +

    Enter URL: localhost:631

    +
    5. +
  5. +

    Select Add printers and classes.

    +
    +

    You may be prompted to enter credentials. If your account is a member +of the lpadmin group, you can use your own credentials; if not, those of the +root account or another account that is a member of lpadmin will be required.

    +
    +
    6. +
  6. +

    Add your printers.

    +
    +

    Connected printers may be automatically offered to be added. You may +also be able to find printers using the Find Printer function. If +CUPS offers you the wrong type of printer to be automatically added or +it is unclear what driver to select for a printer, you may be able to +get some useful information to help with manually installing your +printer by searching the Internet for the string cups and your +printer model.

    +
    +
    +

    Some printers will work with an AppSocket/HP JetDirect connection of the form socket://hostname.

    +
    +
    7. +
  7. +

    Be sure to select a printer as the default (usually by selecting +Printers at the top of the page, then select the printer to be set as the +default, then from the Administration drop down: Set As Server Default).

    +
    8. +
  8. +

    Quit firefox

    +
    9. +
+
+
+
+

A.16. NTP configuration

+
+

For good performance with NTP, please follow the recommendations in +/usr2/fs/misc/ntp.txt.

+
+
+

Additionally, to make the ntpq -c pe output more readable for local +devices, you can adjust the contents of /etc/hosts. The local +devices should be listed in the file, but use a nickname (15 +characters or less) that is meaningful locally in place of the +canonical name (the first name after the IP address). The canonical +name can be listed after the nickname.

+
+
+
+

A.17. Add raid-events scripts

+
+

If your system is using a RAID configuration, you may want to install +the raid-events script. The script provides email notifications of +when Rebuilds (and array checks) start and end. For full details on +the script and installation instructions, please see the +raid-events subsection in the +Script descriptions section of the +RAID Notes for FSL 11 document.

+
+
+
+

A.18. Add refresh_spare_usr2

+
+

If you are using two systems, an operational and a spare, you may +want to install the refresh_spare_usr2 script. The script can be +used to backup the /usr2 partition on the operational system to +the spare system. For full details on the script and installation +instructions, please see the +refresh_spare_usr2 subsection in the +Script descriptions section of the +RAID Notes for FSL 11 document.

+
+
+
+

A.19. Install pgplot version of pgperl

+
+ + + + + +
+
Important
+		 +This step is “use as at your own risk.” Every effort has +been made to make it safe, but it installs a non-standard package. You +should only use it if you need it and accept the risk. +
+
+
+

This replaces the use of the giza package in pgperl with pgplot. +It will restore the behavior of pgperl (used by plotlog) from +distributions FSL10 and earlier. Full directions can be found in the +INSTALL file in sub-directory +libpgplot-perl.

+
+
+

This package uses the same pgperl source as the standard version, +but it is built against pgplot instead. If pgperl receives a +security update, the pgplot version will be overwritten. It is +possible to prevent that if you prefer.

+
+
+
+
+
+

Appendix B: Managing Security Updates

+
+
+

It is strongly recommended that you use the weekly cron update +download (the “weekly cron job”) as configured according to the +Window 2 subsection in the fsadapt section above. This will +keep you informed of the available updates on a weekly basis.

+
+
+

It is also recommended that you remove anacron as described in the +Remove anacron package section below. This will cause the updates +to always be downloaded at what should be innocuous time, early Sunday +morning (but this can be adjusted if need be).

+
+
+ + + + + +
+
Note
+		 +An optional method for identifying available updates without using +the weekly cron job is described below in the section +Manually checking for updates. +
+
+
+

B.1. Installing updates (upgrading)

+
+ + + + + +
+
Tip
+		 +It is recommended that a disk rotation be performed before any +update is installed. This will make recovery much easier if a problem with the +update is discovered. Please see the FSL11 Raid document section +Recoverable testing for a +streamlined method to manage testing of updates. +
+
+
+

If updates are needed, the weekly cron job will send a message to root +(or whoever e-mail to root is aliased to, typically oper) with +instructions on how to install the updates. You can choose a +convenient time, when not in (or about to start) operations, to install +the updates and test the system.

+
+
+ + + + + +
+
Important
+		 +The weekly cron job message will include instructions for +handling a kernel update if one is available. See the +Kernel updates subsection below for additional considerations for +kernel updates. +
+
+
+

The commands for installing the updates given by the message are (note + the use of apt instead of apt-get):

+
+
+
+
apt upgrade
+
+
+
+

Enter y to confirm as needed. Then

+
+
+
+
apt clean
+
+
+
+

If the weekly cron job was installed according to the fsadapt +section above (for Window 2), the first of these commands (with + upgrade) will show if any NEWS items are included in the +update. If there are, they will be displayed by a paging program at the beginning of the upgrade and +you will be given an extra chance to abort before installing.

+
+
+ + + + + +
+
Note
+		 +NEWS items are, rarely occurring, announcements that may +indicate additional steps are needed beyond the standard installation +process. If any NEWS items are displayed, you should consider +whether these will effect your system and how to handle them before +installing. The first command above (with upgrade) will also cause e-mails +to be sent to root with the NEWS information. +
+
+
+
+

B.2. Kernel updates

+
+ + + + + +
+
Warning
+		 +Kernel updates require extra care and testing. If you are +using a RAID, you should consider using the +Recoverable testing +procedure to give more, and easier, options for recovery in case there +is a problem. That procedure contains special instructions for kernel +update testing. +
+
+
+ + + + + +
+
Note
+		 +
+

When a kernel update is available, you may see messages at the start of the cron job output similar to:

+
+
+
+
apt-listchanges: Unable to retrieve changelog for package linux-headers-amd64; 'apt-get changelog' failed with: E: Version '5.10.120+1' for 'linux-headers-amd64' was not found
+E: No packages found
+
+apt-listchanges: Unable to retrieve changelog for package linux-image-amd64; 'apt-get changelog' failed with: E: Version '5.10.120+1' for 'linux-image-amd64' was not found
+E: No packages found
+
+
+
+

and

+
+
+
+
Calling ['apt-get', '-qq', 'changelog', 'linux-headers-amd64=5.10.120+1'] to retrieve changelog
+Calling ['apt-get', '-qq', 'changelog', 'linux-image-amd64=5.10.120+1'] to retrieve changelog
+
+
+
+

These appear to be benign. Our only advice at this time is to ignore +them.

+
+
+
+
+

If there is a kernel update available, the weekly cron job output +will include a warning at the end with additional instructions +depending on which type is available. There are two types of kernel +updates:

+
+
+
    +
  1. +

    ABI updates, e.g., from 4.9.0-11-amd64 to +4.9.0-12-amd64 (with 11 and 12 being the ABI versions), which change the kernel ABI (Application Binary + Interface). The warning for this case is:

    +
    +
    +
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+NB: The Linux kernel image is one of the packages due to be upgraded.
+NB: (The kernal ABI has changed as per the linux-latest source package above
+NB:  so all out-of-tree modules WILL NEED TO BE REBUILT after you REBOOT.)
+NB: Please allow _extra time_ for TESTING after the upgrade.
+!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    +
    +
    +
    2. +
  2. +

    Non-ABI updates, which update the kernel, but do not change the +ABI. The warning for this case is:

    +
    +
    +
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+NB: The Linux kernel image is one of the packages due to be upgraded.
+NB: (Upgrading will OVERWRITE the running kernel and require you to REBOOT!)
+NB: Please allow _extra time_ for TESTING after the upgrade.
+!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    +
    +
    +
    3. +
+
+
+

Be sure to allow time to follow the instructions when planning to +install these updates. As described in the ABI update warning, you +will need to rebuild any out-of-tree modules after rebooting for that +case. This is discussed in the Updating out-of-tree modules +subsection below.

+
+
+ + + + + +
+
Caution
+		 +In extreme circumstances, an ABI (but not a non-ABI) kernel +update can be deferred to a later date when more extensive testing can +be performed by using apt-get in place of apt in the instructions +for installing the update. This works because an ABI update involves +new packages. The apt-get command will install the updates for existing +packages, but it will not install the new packages. While this method can +be used to install the other updates, it is not recommended since +there are presumably security patches needed for the kernel and they +are not being installed in this case. +
+
+
+ + + + + +
+
Tip
+		 +
+

When the kernel is upgraded, you may get messages such as:

+
+
+
+
update-initramfs: Generating /boot/initrd.img-5.10.0-16-amd64
+W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast
+
+
+
+

These are usually benign, unless you need that firmware. If you don’t, +these messages can be silenced for future upgrades by creating an +empty version of the file. For this example, enter:

+
+
+
+
touch /lib/firmware/ast_dp501_fw.bin
+
+
+
+
+
+

B.2.1. Updating out-of-tree modules

+
+

When a ABI update is installed, it will be necessary to update any, +so-called, out-of-tree modules that use the kernel ABI. This must be +done after rebooting with the new kernel installed.

+
+
+

For a normal FSL11 installations, unless you have installed other +out-of-tree modules, the only module that needs to be rebuilt is the +GPIB driver (if it is installed). You will need to recompile it (usually using fsadapt, + Window 2, config_gpib only) after the initial reboot + and then (to keep these instructions simple) reboot again.

+
+
+

If you have installed other out-of-tree modules (e.g., you use a +special driver for some of your NICs), you will need to update them +appropriately after the initial reboot and then (to keep these + instructions simple) reboot again.

+
+
+
+
+

B.3. Recovery from a failed update

+
+

If an update fails, e.g., an updated kernel fails to boot or another problem is discovered, +you can recover as described in FSL11 RAID document +Recoverable testing +section, if you were following that method, or from a shelf disk +according to the FSL11 RAID document Recover from +a shelf disk section if not and you have a good shelf disk.

+
+
+

B.3.1. Additional recovery option for a failed ABI kernel update

+
+

For a ABI update that has failed, it is also possible to try to use +the previous kernel on the current system. For a single boot, use the +Advanced option in the grub menu at boot and then select the +previous kernel. You can change back permanently to the previous +kernel by purging the new kernel and its headers. To do this, use:

+
+
+
+
dpkg -l|grep linux-image
+dpkg -l|grep linux-headers
+
+
+
+

to determine the ABI version to be removed. For example, for the +first command above, you may get:

+
+
+
+
linux-image-4.9.0-11-amd64
+linux-image-4.9.0-12-amd64
+
+
+
+

The package with 12 would be the later version that should be purged:

+
+
+
+
apt-get purge linux-image-4.9.0-12-amd64
+
+
+
+

Likewise with the linux-headers. For example, for the 12 ABI +version, there will be two packages you should purge:

+
+
+
+
linux-headers-4.9.0-12-amd64
+linux-headers-4.9.0-12-common
+
+
+
+
+
+

B.4. Manually checking for updates

+
+

If you do not use the weekly cron job to check for updates, or if +you want to make sure you have the very latest updates when you +install them, you can run the distributed copy of the weekly update +script manually to check for updates:

+
+
+
+
/root/fsl11/etc_cron.weekly_apt-show-upgradeable
+
+
+
+

If there is no output, there are no updates to install.

+
+
+

If there is output, there are updates to install. You can install them +by following the installation procedure in subsection +Installing updates (upgrading) above, except you will use the +instructions from the output of the script above instead of from the +weekly cron job (the outputs should be equivalent for the same set +of updates). Additionally, please read the following NOTE.

+
+
+ + + + + +
+
Note
+		 +If the weekly cron job has not been installed, you may not get a + display of NEWS items and a chance to abort when you install the updates. You + can use the method below with the --which=news parameter to + check for NEWS before installing an update. +
+
+
+

Any NEWS items will be included in the script output along with the +packages to be updated. If you would like to see any NEWS items more +distinctly after the previous command and before installing the +updates, you can run the script again using the --which=news option:

+
+
+
+
/root/fsl11/etc_cron.weekly_apt-show-upgradeable --which=news
+
+
+
+

If there are updates available and no NEWS items, you will only get +the installation instructions.

+
+
+

You can use this second form of running the script to check for +updates initially, if you do not need to review which updates are +available (you will still get warnings about kernel updates). As +usual, you will see no output at all if there are no updates +available.

+
+
+
+

B.5. End of security updates

+
+

When support for bullseye ends, currently expected in May 2024, +there will be no more security updates. At that time, the existing +packages will be migrated to the Debian archive site. This will be +visible in the output from the weekly cron job script as errors that +the packages files can’t be found. Two steps are needed at that time:

+
+
+
    +
  1. +

    If you have been using the weekly cron job, it should be deleted:

    +
    +
    +
    rm /etc/cron.weekly/apt-show-upgradeable
    +
    +
    +
    +

    (you may need to answer y to confirm)

    +
    +
    2. +
  2. +

    Change the /etc/apt/sources.list file to point to the archive +site. Although there will be no more security updates, this will enable +downloading of additional packages if they are needed. The new lines that +should replace the corresponding existing lines are:

    +
    +
    +
    deb http://archive.debian.org/debian/ bullseye main contrib non-free
+deb http://archive.debian.org/debian-security bullseye/updates main contrib non-free
    +
    +
    +
    +

    And if you are using deb-src lines:

    +
    +
    +
    +
    deb-src http://archive.debian.org/debian/ bullseye main contrib non-free
+deb-src http://archive.debian.org/debian-security bullseye/updates main contrib non-free
    +
    +
    +
    +

    Otherwise the deb-src lines can be commented out (with a leading #). Any other deb or +deb-src lines relating to updates, proposed-updates etc. should likewise be commented out.

    +
    +
    +

    In addition, if you want to install packages from more recent +distributions that have been backported to bullseye you can add:

    +
    +
    +
    +
    deb http://archive.debian.org/debian-backports bullseye-backports main contrib non-free
    +
    +
    +
    +

    However, the “backports” are not normally needed.

    +
    +
    +

    Lastly, update the index files:

    +
    +
    +
    +
    apt-get update
    +
    +
    +
    +

    This may generate an error about a Release file having expired, but that is benign.

    +
    +
    3. +
+
+
+ + + + + +
+
Note
+		 +When bullseye becomes no longer supported, it is strongly advised that you move +your FS machine behind a firewall or upgrade it to a more recent FS Linux release. +
+
+
+
+
+
+

Appendix C: Other Maintenance Procedures

+
+
+

This appendix covers additional procedures for maintaining your +system.

+
+
+

C.1. Update IP address, hostname, FQDN, and other network information

+
+

This is useful if the computer is physically moved to a different +site, its IP address changes, or its network information needs to be +updated for a different reason. This is typically not needed if you +use DHCP, though that may still require some of the changes in the +Modify other system files step below (please let us know +if you gain experience).

+
+
+

This subsection requires using nm-connection-editor on a graphic +display (nmtui may be an option on a text terminal, but it has not +been fully verified). You may need to be root or desktop to do +this. This subsection assumes you are in the program and have +sufficient permissions.

+
+
+ + + + + +
+
Note
+		 +If you move the disks to a computer with a different mainboard +model, the device names of the network interfaces may change. In that +case, you will need to reselect the names as described in the +sub-steps of the Stabilize network configuration section of the +Additional Setup Items appendix. This should not be necessary if +the origin and destination computers have the same mainboard. +
+
+
+
    +
  1. +

    Select your connection and click the “gear” icon.

    +
    2. +
  2. +

    Select the IPv4 Settings (or IPv6 Settings if you are using +IPv6) tab.

    +
    3. +
  3. +

    Adjust your Manual Method configuration: Addresses, DNS +Servers (comma separated), and Search domains.

    +
    4. +
  4. +

    Click Save.

    +
    5. +
  5. +

    Close the window by pressing Esc (while the focus is on that +window).

    +
    6. +
  6. +

    Modify other system files:

    +
    +

    Update the information as appropriate. The system may have initially +been installed with the default hostname debian and no domain name.

    +
    +
    +
    /etc/hostname
    +

    Change your hostname

    +
    +
    +
    /etc/hosts
    +

    Update your IP address, FQDN (canonical name), and alias (typically +the hostname, but multiple aliases/nicknames are allowed).

    +
    +
    +

    If you moved your computer to a new LAN environment, you may also want +to update the nodes and aliases listed, see also Setup /etc/hosts.

    +
    +
    +
    /etc/networks
    +

    Use your local subnet (class A, B, or C) for the localnet line.

    +
    +
    +
    /etc/mailname
    +

    Use fully qualified node name.

    +
    +
    + + + + + +
    +
    Note
    +    		 +
    +

    If your system doesn’t have a FQDN or you don’t want to show it in +e-mail messages, you may be able to use a fake one. A FQDN may be +necessary to allow messages to be sent successfully to some remote +hosts and mailman mail lists. A possible strategy for this is to +append .net to the node name you use in this file and the next. The +node name in these two files can be different than the official +hostname. However, these two mail related files should be consistent. +You might consider fs1-<xx>.net (or fs2-<xx>.net), where <xx> is +your station two letter code (lower case).

    +
    +
    +
    +
    +
    /etc/exim4/update-exim4.conf.conf
    +

    Look for hostnames=, use fully qualified domain name.

    +
    +
    +

    Then execute:

    +
    +
    +
    +
    update-exim4.conf
    +
    +
    +
    +

    When finished, reboot.

    +
    +
    7. +
+
+
+
+

C.2. Increase the size of an LVM volume

+
+

It is possible to increase the size of an LVM volume if there is +additional room available in its volume group. These instructions +assume you will be resizing a logical volume for a typical +configuration. For example, for the logical volume mounted at /usr2, +on RAID device /dev/md0, which is using /dev/sda2 and /dev/sdb2. +Additionally, example pathnames are given in the instructions below +for adjusting the size of the logical volume for /usr2. All these +names may be different if you want to resize a different volume and/or +your disk configuration is different.

+
+
+
    +
  1. +

    Preparation

    +
    +
      +
    1. +

      Check that there is enough free space available.

      +
      +

      Examine the output of:

      +
      +
      +
      +
      vgs
      +
      +
      +
      +

      You can increase the size of a logical volume if the volume group +(under the VGS column heading) has enough free space (VFree +heading) for the increase. Typically, the volume group would be vg0.

      +
      +
      2. +
    2. +

      Determine the Path of the logical volume you want to extend.

      +
      +
        +
      1. +

        Get a listing to relate the internal device-mapper pathnames +(under the Filesystem column heading) and where the logical volumes +are mounted (Mounted on heading). For example, +/dev/mapper/vg0-usr2 would typically be mounted at /usr2.

        +
        +
        +
        df -h
        +
        +
        +
        2. +
      2. +

        Get a listing to relate the internal device-mapper pathname (under +the DMPath column heading) to the logical volume Path. For +example, for /dev/mapper/vg0-usr2, the Path would typically be +/dev/vg0/usr2.

        +
        +
        +
        lvdisplay -C -o lv_dm_path,lv_path
        +
        +
        +
        3. +
      3. +

        For the mount point of the logical volume you want to extend, +determine the Path using the internal device-mapper pathname from +the above two sub-steps. For example, the logical volume for /usr2 +would typically correspond to /dev/mapper/vg0-usr2 and the +corresponding Path would be /dev/vg0/usr2.

        +
        4. +
      +
      +
      3. +
    +
    +
    2. +
  2. +

    Pre-check (optional)

    +
    +

    This sub-step is not required but can be used, along with the +“Post-check” sub-step below, to check that the volume size changed as +expected and that no files were lost or changed +size/modification-time.

    +
    +
    +
      +
    1. +

      Get the size (under the 1G-block column heading) of the logical +volume (Mounted on heading) for the volume of interest:

      +
      +
      +
      df -BG
      +
      +
      +
      +

      Record the size to compare to the results in the “Post-check” +sub-step below.

      +
      +
      2. +
    2. +

      Make a listing of the files on the mount_point (include the +leading /) to be changed. For example, the mount_point might be +/usr2.

      +
      +
      +
      ls -ltR mount_point >/tmp/before.txt
      +
      +
      +
      3. +
    +
    +
    3. +
  3. +

    Make the change, using the Path you determined in the +“Preparation” sub-step above.

    +
    +
      +
    1. +

      Make a backup of your system.

      +
      + + + + + +
      +
      Note
      +      		 +This sub-step, and recovery in case of a problem, is much easier +if you using the FSL11 RAID system. If not, it is strongly recommended +that you make your own backup of your entire system. The remainder of +this sub-step assumes you are using a RAID, following the approach of +the Recoverable testing procedure +in the Raid Notes for FSL11 document. +
      +
      +
      +

      If you are using a RAID, you can drop the primary disk out of the +RAID to save as a backup:

      +
      +
      +
      +
      drop_primary
      +
      +
      +
      2. +
    2. +

      Extend Path

      +
      +

      For the logical volume (mount point) you want to extend, you can +either:

      +
      +
      +
        +
      1. +

        Incrementally increase the size. For example, to increase Path +by 4 GB:

        +
        +
        +
        lvextend -L+4G Path
        +
        +
        +
        2. +
      2. +

        Set the size to a new larger total size, say 8GB:

        +
        +
        +
        lvextend -L8G Path
        +
        +
        +
        3. +
      +
      +
      3. +
    3. +

      Resize Path

      +
      + + + + + +
      +
      Important
      +      		 +Do not interrupt the next command. If it is interrupted +and you are using the Recoverable +testing procedure in the Raid Notes for FSL11 +document, you will need to utilize the +If the update is +deemed to have failed subsection of that procedure. Otherwise, if +you are not using that procedure, you will need to use your own +recovery method. +
      +
      +
      +
      +
      resize2fs Path
      +
      +
      +
      4. +
    +
    +
    4. +
  4. +

    Post-check (optional)

    +
    +

    This sub-step is not required but can be used, if the “Pre-check” +sub-step above was used, to check that the new size is correct and no +files were lost or changed size/modification-time.

    +
    +
    +
      +
    1. +

      Check that the size of the logical volume (under the Mounted on +column heading) has the expected new size in the output of:

      +
      +
      +
      df -BG
      +
      +
      +
      +

      Compare the result to that in the “Pre-check” sub-step above.

      +
      +
      2. +
    2. +

      Make a listing of the files on the mount_point (include the +leading /) that was changed. For example, the mount_point might +be /usr2.

      +
      +
      +
      ls -ltR mount_point >/tmp/after.txt
      +
      +
      +
      3. +
    3. +

      Compare the before and after listings of the files

      +
      +
      +
      diff /tmp/before.txt /tmp/after.txt
      +
      +
      +
      4. +
    +
    +
    +

    There should be no differences in the listings except any changes that +can explained by other expected activity that occurred since the +“Pre-check” sub-step above. If there was no other activity on the +logical volume, there should be no differences.

    +
    +
    5. +
  5. +

    Cleanup

    +
    + + + + + +
    +
    Note
    +    		 +If you not are using the +Recoverable testing procedure in +the Raid Notes for FSL11 document, you will need to use +your own methods to restore the system if there was a problem. This +step describes how to proceed if you are using the referenced +procedure. +
    +
    +
    +

    There are two options:

    +
    +
    +
      +
    1. +

      If you are satisfied with the change, you can recover the RAID +with:

      +
      +
      +
      recover_raid
      +
      +
      +
      +

      This should only take a few minutes.

      +
      +
      + + + + + +
      +
      Note
      +      		 +The change in the volume size will not propagate to the shelf +disk until the next disk rotation. +
      +
      +
      2. +
    2. +

      If you are not satisfied with the change, you can try again if you +first restore the RAID using the +If the update is +deemed to have failed subsection of the +Recoverable testing procedure in +the Raid Notes for FSL11 document.

      +
      3. +
    +
    +
    6. +
+
+
+
+
+
+

Appendix D: Rescue Mode

+
+
+

Rescue mode is useful for repairing some problems that prevent booting +and/or logging in.

+
+
+ + + + + +
+
Note
+		 +If your computer’s setup utility is locked with a password, you +may need that password to select booting from your installation media. +
+
+
+ + + + + +
+
Note
+		 +You should provide suitable values for your system when a +specific value is required. Values that agree with the FSL11 install +described in this document (or reasonable defaults) are shown in parentheses. +
+
+
+
    +
  1. +

    Boot from installation media

    +
    2. +
  2. +

    Select Advanced options …​

    +
    3. +
  3. +

    Select …​ Rescue mode

    +
    + + + + + +
    +
    Note
    +    		 +
    +

    You could instead add parameters to the boot line (by entering e for UEFI or +Tab for BIOS on the …​ Rescue mode line instead), following the +directions in the Set boot options and boot installer section above. +This is not necessary nor usually helpful, but if you use this approach the +most useful parameters are probably netcfg/disable_dhcp=true and/or +time/zone=UTC. Use of added parameters will change the dialogue +below.

    +
    +
    +
    +
    4. +
  4. +

    Select Language (English)

    +
    5. +
  5. +

    Select Location (United States)

    +
    6. +
  6. +

    Select Keymap (American English)

    +
    7. +
  7. +

    Network configuration

    +
    +

    If no network is currently available (or you know that you do not need it +for the rescue), simply press Enter when DHCP autoconfiguration starts and +press Enter again for the resulting Network autoconfiguration failed +message. Thereafter select Do not configure the network at this time and +enter in the machine’s hostname when prompted before continuing below.

    +
    +
    +

    If the DHCP autoconfiguration succeeds before you can stop it, you may +as well confirm the hostname and domainname and continue with the +network anyway, since you never know when it might prove useful. +(However, if you want to make sure you don’t use the network, you can + select Go Back and press Enter for the resulting Network + autoconfiguration failed message. Thereafter select Do not + configure the network at this time and enter in the machine’s + hostname when prompted before continuing below.)

    +
    +
    +

    Otherwise if the DHCP autoconfiguration fails and you want to use the +network, press Enter for the resulting Network autoconfiguration +failed message. You can then select the appropriate option, most +likely Configure network manually and give appropriate responses to the +prompts, ultimately continuing below.

    +
    +
    8. +
  8. +

    Select time zone (Eastern)

    +
    + + + + + +
    +
    Note
    +    		 +The selected time zone will have no effect on the timestamps +stored on the disk for any changes you may make, but will affect the displayed times you see. +
    +
    +
    9. +
  9. +

    Unless you are not using Software RAID, select Assemble RAID array

    +
    +

    Press Space on Automatic and Enter to continue

    +
    +
    10. +
  10. +

    Select your root file system (/dev/vg0/root)

    +
    11. +
  11. +

    Select Yes to mount separate /boot partition (/boot), unless it is corrupt

    +
    +

    For UEFI boot also select Yes to mount separate /boot/efi partition (/boot/efi), +unless it is corrupt

    +
    +
    12. +
  12. +

    Select Execute a shell in /dev/vg0/root (or whatever your root file system is)

    +
    13. +
  13. +

    Select Continue to enter rescue mode

    +
    14. +
  14. +

    Use whatever commands are needed for your repair

    +
    + + + + + +
    +
    Note
    +    		 +
    +

    If you need to use the network, DNS does not appear to work by +default in recovery mode. Use of explicit IP addresses does work. If +you need to use DNS, you can make it functional by deleting the symbolic +link /etc/resolv.conf and creating it as a normal file with the +nameserver information you want, e.g.:

    +
    +
    +
    +
    rm /etc/resolv.conf
+cat >>/etc/resolv.conf <<EOF
+nameserver 8.8.8.8
+EOF
    +
    +
    +
    +
    +
    15. +
  15. +

    Use the exit command to exit when done

    +
    16. +
  16. +

    Select Reboot the system

    +
    17. +
  17. +

    “Bob’s your uncle” (i.e., you are done!)

    +
    18. +
+
+
+
+
+ + + \ No newline at end of file diff --git a/raid.html b/raid.html new file mode 100644 index 0000000..4275403 --- /dev/null +++ b/raid.html @@ -0,0 +1,2673 @@ + + + + + + + + +RAID Notes for FSL 11 + + + + + +
+
+
+
+
+
+
+

1. Introduction

+
+
+

These notes are intended to cover, albeit tersely, the major issues +for RAID operations with FSL11 (see the FS Linux +11 Installation Guide document). The scripts have been updated since +FSL10 to handle the occasional reversals of the assignment of sda +and sdb relative to the controller numbering. Some other minor +improvements are included as well.

+
+
+

All operations and scripts in this document require root privileges +unless otherwise indicated.

+
+
+
+
+

2. Guidelines for RAID operations

+
+
+

The FSL11 RAID configuration normally uses two disks configured +according to the FSL11 installation instructions (see the +FS Linux 11 Installation Guide document). +Mandatory and recommended guidelines are given below.

+
+
+

2.1. Mandatory practices

+
+

These practices are necessary for the procedures in this document.

+
+
+
    +
  1. +

    Make sure there are no SATA devices on lower numbered controllers +than the primary and secondary disks. The primary disk must be +on a lower numbered controller than the secondary. Putting the +primary disk on controller 0 and the secondary on 1 is usually +a good choice. This may require changing the internal cabling.

    +
    + + + + + +
    +
    Note
    +    		 +Which disk is primary disk and which secondary is determined +by which controllers they are on and so is fixed by the slots. +Although the primary will usually be sda and the secondary, +sdb, the designations may occasionally be reversed. The scripts used +in this document take that into account and will work correctly for +the primary and secondary slots. +
    +
    +
    2. +
  2. +

    Set your BIOS to allow hot swapping of disks for both the primary +and secondary controllers. This is necessary to use the RAID +procedures described in this document.

    +
    3. +
  3. +

    Never mix disks from different computers in one computer.

    +
    4. +
  4. +

    Never split up a RAID pair unless already synced. To enforce this, +use the rotation_shutdown command for shutdowns whenever two disks +are in use in the RAID.

    +
    +

    This command will only shutdown the system if the RAID is synced. This +script can also be useful in other cases when you are splitting the +disks and want to make sure they are synced first.

    +
    +
    +

    A RAID pair (kept together and in order) can be removed/reinserted or +moved between computers if need be. A disk rotation, recoverable +testing, and initializing a new disk are the only routine reasons to +split a pair.

    +
    +
    + + + + + +
    +
    Note
    +    		 +
    +

    When booting a disk from a RAID by itself, you may see ~20 volume +group not found/processed error message pairs and a mdadm: +/dev/md/0 assembled from 1 drive out of 2, but not started message +(typically after the second pair of the volume group messages), then +the machine will boot. These error messages only appear like this the +first time a disk from a RAID is booted without its partner.

    +
    +
    +

    When the single disk is booted subsequently (or booted normally with +its partner) there may be a couple of volume group not +found/processed error message pairs.

    +
    +
    +
    +
    5. +
+
+
+
+

2.2. Recommended practices

+
+

These recommendations are intended to provide consistent procedures +and make it easier to understand any problems, if they occur.

+
+
+
    +
  1. +

    Make the upper (or left) slot the primary, the lower (or right) +slot the secondary. This may require changing the internal cabling.

    +
    2. +
  2. +

    Label the slots as primary and secondary as appropriate.

    +
    3. +
  3. +

    Always boot for a refresh/blank with the primary slot turned on and the secondary slot turned off: so it is clear which is the active disk

    +
    4. +
  4. +

    Label the disks (so visible when in use) with the system name and number them 1, 2, and 3, …​

    +
    5. +
  5. +

    Label the disks (so visible when in use) with their serial numbers, determined either from mdstat when only one disk inserted or by examining the disk

    +
    6. +
  6. +

    For reference, place the disk serial numbers in a file with their corresponding numbers, e.g.:

    +
    +
    /root/DISKS.txt
    +
    +
    1=ZC1B1YCC
+2=ZC1A6WZ1
+3=ZC1AHENM
    +
    +
    +
    7. +
  7. +

    When rotating disks, keep the disks in cyclical order (primary, +secondary, shelf): 1, 2, 3; then 2, 3, 1;, then 3, 1, 2, then 1, +2, 3; and so on.

    +
    8. +
  8. +

    Rotate disks for a given computer at least once a month, and before any updates

    +
    9. +
  9. +

    If you have a spare computer (and/or additional systems), keep the +numbered order of the disks the same on all computers.

    +
    +

    Occasionally, extra rotations may be needed to re-sync the order of +the disks in the computers.

    +
    +
    10. +
  10. +

    Do not turn a disk off while the system is running. The only time a key switch state should be changed while the system is running is to add a disk for a blank or refresh operation.

    +
    11. +
+
+
+
+
+
+

3. Disk rotation

+
+
+

This section describes the disk rotation procedure. It is used to make +periodic updates of the shelf disk.

+
+
+ + + + + +
+
Note
+		 +Your BIOS must be set to allow hot swapping of disks, +particularly for the secondary controller (it should also be set for +the primary controller). +
+
+
+ + + + + +
+
Tip
+		 +If you do not have access to the root account, you may have +sudo access to privileged commands. If so, you may be able to run +rotation_shutdown and refresh_secondary commands with sudo, +e.g., sudo rotation_shutdown. This also applies to other +privileged commands used in this document. +
+
+
+
    +
  1. +

    Shut the system down with the rotation_shutdown command.

    +
    +

    This command will check the status of the RAID and proceed to shutting +down only if the RAID is synced. There are three errors that can +prevent shutting down: (i) if the FS is running, you should terminate +it before trying again; (ii) if the RAID is recovering, you will +need to wait until the recovery is finished before shutting down, you +can check the progress with the mdstat command; and (iii) if the +RAID is degraded, seek expert advice.

    +
    +
    2. +
  2. +

    Take the disk from primary slot, put it on the shelf

    +
    +

    We recommend that you label the disk immediately, including the date +(and possibly the time). In addition to getting the disk labeled +before it is put away, it will reduce the chances that it is confused +with the old shelf disk.

    +
    +
    3. +
  3. +

    Move the disk from the secondary slot to the primary slot, keyed-on

    +
    4. +
  4. +

    Move the old shelf disk to the secondary slot, keyed-off

    +
    5. +
  5. +

    Boot (primary keyed-on, secondary keyed-off)

    +
    6. +
  6. +

    Run refresh_secondary

    +
    7. +
  7. +

    Key-on the secondary slot when prompted

    +
    8. +
  8. +

    If the script rejects the disk (and stops with an error), seek +expert advice.

    +
    +

    Be sure to note any messages so they can be reported.

    +
    +
    9. +
  9. +

    If the disk is accepted, let the refresh run to completion.

    +
    +

    You can check its progress with mdstat. The system can be used for +operations while the refresh is in progress, but may be a little slow.

    +
    +
    10. +
+
+
+
+
+

4. Recoverable testing

+
+
+

Seek expert advice before using this method.

+
+
+

This section describes a method for testing updates in a way that provides a +relatively easy recovery option if a problem occurs. Should that recovery fail +for some reason, it is still possible to recover with the shelf disk as +described in the Recover from a shelf disk section below.

+
+
+

The basic plan is given in the three subsections below. The first +covers Setup and testing, the final two cover what to do +If the update is deemed successful or +If the update is deemed to have failed.

+
+
+

4.1. Setup and testing

+
+ + + + + +
+
Note
+		 +Your BIOS must be set to allow hot swapping of disks for both +the primary and secondary controllers. +
+
+
+
    +
  1. +

    If a rotation hasn’t just been completed, perform one (as an extra +backup) according to Disk rotation above.

    +
    2. +
  2. +

    Shut the system down with the rotation_shutdown command.

    +
    + + + + + +
    +
    Tip
    +    		 +
    +

    If an update is relatively minor or the envisaged testing is intended +to be of short duration and success is likely, expert users may wish +to make use of the drop_primary script to split the RAID pairing in +place of the reboot cycle method described here. Note that some +(hopefully minor) data loss is possible on the primary (backup) disk +as it is removed from the RAID whilst all the file systems are still +mounted read/write. Hence this script should only be used on a +unloaded or single-user system. The main advantage of using this +script is that, if the test is successful, no manipulation of the key +switches is required.

    +
    +
    + + + + + +
    +
    Warning
    +    		 +Do NOT use the drop_primary script for testing kernel +updates or any other testing that could affect grub and/or require +you to reboot in order to evaluate the success thereof. +
    +
    +
    +
    +
    3. +
  3. +

    Key-off the primary slot

    +
    4. +
  4. +

    Reboot (primary keyed-off, secondary keyed-on)

    +
    5. +
  5. +

    Install and test the update

    +
    +

    The update and testing will occur on the secondary disk only.

    +
    +
    6. +
  6. +

    Proceed to one of the two subsections below, +If the update is deemed successful or +If the update is deemed to have failed, as appropriate.

    +
    7. +
+
+
+
+

4.2. If the update is deemed successful

+
+

The other disk can be updated:

+
+
+
    +
  1. +

    Key-on the primary slot

    +
    2. +
  2. +

    Run recover_raid to add the primary slot disk back into the RAID.

    +
    +

    The recover_raid script will fail if the disk hasn’t spun up and been recognized by the kernel. It is perfectly +fine to try several times until it succeeds.

    +
    +
    3. +
  3. +

    Once the recovery completes (this may only take a few minutes), the +system has been successfully updated.

    +
    4. +
+
+
+
+

4.3. If the update is deemed to have failed

+
+

The system can be recovered as follows:

+
+
+
    +
  1. +

    Shutdown the system, e.g., shutdown -h now

    +
    2. +
  2. +

    Key-off the secondary slot

    +
    3. +
  3. +

    Key-on the primary slot

    +
    4. +
  4. +

    Reboot (primary keyed-on, secondary keyed-off)

    +
    5. +
  5. +

    Run blank_secondary

    +
    6. +
  6. +

    Key-on the secondary slot when prompted

    +
    7. +
  7. +

    Answer y to blank

    +
    8. +
  8. +

    Run refresh_secondary

    +
    9. +
  9. +

    Once the refresh is complete (this may take several hours), you have +recovered to the original state.

    +
    10. +
+
+
+
+
+
+

5. Recover from a shelf disk

+
+
+

The section describes how to recover from a good shelf disk. This +might be needed, e.g., if it is discovered that a problem has +developed on the RAID pair since the last disk rotation. This might be +due to a bad update of some type or some other problem.

+
+
+ + + + + +
+
Tip
+		 +Before using this procedure, it should be considered whether the +damage is extensive enough to require starting over from the shelf +disk or whether it can be reasonably repaired in place. +
+
+
+ + + + + +
+
Important
+		 +This will only produce a good result if the shelf disk is +a good copy. +
+
+
+ + + + + +
+
Warning
+		 +Do not use this procedure if a problem with the computer +caused the damage to the RAID. +
+
+
+ + + + + +
+
Note
+		 +Your BIOS must be set to allow hot swapping of disks, +particularly for the secondary controller (it should also be set for +the primary controller). +
+
+
+
    +
  1. +

    Shutdown the system, e.g., shutdown -h now

    +
    2. +
  2. +

    Take the disks from both the primary and secondary slots, set them aside.

    +
    3. +
  3. +

    Insert the good shelf disk in the primary slot, keyed-on.

    +
    4. +
  4. +

    Insert the disk that is next in cyclic order (from the ones set aside) in the secondary slot, keyed-off.

    +
    5. +
  5. +

    Reboot (primary keyed-on, secondary keyed-off)

    +
    6. +
  6. +

    Run blank_secondary

    +
    7. +
  7. +

    Key-on the secondary slot when prompted

    +
    8. +
  8. +

    Answer y to blank

    +
    9. +
  9. +

    Run refresh_secondary

    +
    +

    Once the refresh has entered the recovery phase, the system can be +used for operations, if need be. In that case, the rest of this +procedure can be completed when time allows.

    +
    +
    10. +
  10. +

    Wait until the RAID is not recovering, check with mdstat

    +
    11. +
  11. +

    Shut the system down with the rotation_shutdown command.

    +
    12. +
  12. +

    Take the disk from primary slot, put it back on the shelf

    +
    13. +
  13. +

    Move the disk from the secondary slot to the primary slot, keyed-on

    +
    14. +
  14. +

    Insert the remaining disk, that was set aside, in the secondary slot, keyed-off.

    +
    15. +
  15. +

    Reboot (primary keyed-on, secondary keyed-off)

    +
    16. +
  16. +

    Run blank_secondary

    +
    17. +
  17. +

    Key-on the secondary slot when prompted

    +
    18. +
  18. +

    Answer y to blank

    +
    19. +
  19. +

    Run refresh_secondary

    +
    20. +
  20. +

    When the refresh is complete, you have recovered to the state of the +previous good shelf disk.

    +
    21. +
+
+
+
+
+

6. Initialize a new disk

+
+
+

If one or more of the disks in the set for the RAID fails, you can +initialize new ones to replace them.

+
+
+ + + + + +
+
Important
+		 +The new disks should be at least +as large as the smallest of the remaining disks. +
+
+
+

The subsections below cover various scenarios for initializing one new +disk to complete a set of three, i.e., one of three disks in a set has +failed. It is assumed that you want to maintain the cyclic numbering +of the disks for rotations (but that is not required). It should be +straightforward to adapt the procedures for other cases.

+
+
+

If you need to initialize more than one disk, please follow the +instructions in the Setup +additional disks subsection of the FS Linux 11 +Installation Guide document.

+
+
+

6.1. Currently two disks are running in the RAID

+
+

This case corresponds to not having a good shelf disk.

+
+
+
    +
  1. +

    Shut the system down with the rotation_shutdown command.

    +
    2. +
+
+
+

If the disks are in cyclical order (i.e., primary, secondary are +numbered in order: 1, 2, or 2, 3, or 3, 1), you should:

+
+
+
    +
  1. +

    Take the disk from primary slot, put it on the shelf, labeled +with the date

    +
    2. +
  2. +

    Move the disk from the secondary slot to the primary slot, keyed-on

    +
    3. +
+
+
+

If the disks are not in cyclical order (i.e., primary, secondary +are numbered in order: 1, 3, or 2, 1, or 3, 2), you should:

+
+
+
    +
  1. +

    Take the disk from secondary slot, put it on the shelf

    +
    2. +
+
+
+

In either case, finish with:

+
+
+
    +
  1. +

    Put the new disk in the secondary slot, key-off.

    +
    2. +
  2. +

    Boot (primary keyed-on, secondary keyed-off)

    +
    3. +
  3. +

    Run blank_secondary

    +
    4. +
  4. +

    Key-on the secondary slot when prompted

    +
    5. +
  5. +

    Answer y to blank

    +
    6. +
  6. +

    Run refresh_secondary

    +
    7. +
  7. +

    Once the refresh is complete, the disk can be used normally.

    +
    8. +
  8. +

    Label the new disk with its system name, number, and serial number.

    +
    9. +
+
+
+
+

6.2. Currently one disk is running in the RAID, but two are installed

+
+

In this case, there is a good shelf disk. The strategy used avoids overwriting it until there are three functional disks again.

+
+
+
    +
  1. +

    Use mdstat to determine which disk is running, compare the serial number to those shown on the labels or inspect the disks to determine their serial numbers.

    +
    2. +
  2. +

    Shutdown the system, e.g., shutdown -h now

    +
    3. +
  3. +

    Remove the non-working disk.

    +
    4. +
  4. +

    Move the working disk to the primary slot, if it isn’t already there, keyed-on.

    +
    5. +
  5. +

    Put the new disk in the secondary slot, keyed-off.

    +
    6. +
  6. +

    Boot (primary keyed-on, secondary keyed-off)

    +
    7. +
  7. +

    Run blank_secondary

    +
    8. +
  8. +

    Key-on the secondary slot when prompted

    +
    9. +
  9. +

    Answer y to blank

    +
    10. +
  10. +

    Run refresh_secondary

    +
    11. +
  11. +

    Once the refresh is complete, the disk can be used normally.

    +
    12. +
  12. +

    Label the new disk with its system name, number, and serial number.

    +
    13. +
+
+
+

If the disks are not in cyclical order (i.e., primary, secondary +are numbered in order: 1, 3, or 2, 1, or 3, 2), then on the next disk +rotation you should move the secondary disk to the shelf instead of +moving the primary.

+
+
+
+

6.3. Currently one disk is installed and running

+
+

In this case, the shelf disk is assumed to be healthy, but older. + Again, the strategy is to avoid overwriting it until there is a full + complement of disks available.

+
+
+

If the working disk is not in the primary slot:

+
+
+
    +
  1. +

    Shutdown the system, e.g., shutdown -h now

    +
    2. +
  2. +

    Move the working disk to the primary slot, keyed-on.

    +
    3. +
  3. +

    Boot (primary keyed-on, secondary empty)

    +
    4. +
+
+
+

Then in any event:

+
+
+
    +
  1. +

    Put the new disk in the secondary slot, keyed-off.

    +
    2. +
  2. +

    Run blank_secondary

    +
    3. +
  3. +

    Key-on the secondary slot when prompted

    +
    4. +
  4. +

    Answer y to blank

    +
    5. +
  5. +

    Run refresh_secondary

    +
    6. +
  6. +

    Once the refresh is complete, the disk can be used normally.

    +
    7. +
  7. +

    Label the new disk with its system name, number, and serial number.

    +
    8. +
+
+
+

If the disks are not in cyclical order (i.e., primary, secondary +are numbered in order: 1, 3, or 2, 1, or 3, 2), then on the next disk +rotation you should move the secondary to the shelf instead of the +primary.

+
+
+
+
+
+

7. Script descriptions

+
+
+

This section describes the various scripts that are used for RAID maintenance.

+
+
+

7.1. mdstat

+
+

This script can be used by any user (not just root) to check the +status of the RAID. It is most useful for checking whether a recovery +is in process or has ended, but is also useful for showing the current +state of the RAID, including any anomalies.

+
+
+

The script also lists various useful details for all block devices +(such as disks) that are currently connected, including: the controller +they are on, their model, and serial numbers, where applicable.

+
+
+
+

7.2. rotation_shutdown

+
+

This script can be used to shut the system down if the RAID is in a +state that allows a disk rotation to be performed, i.e., synced. The +RAID must not be recovering and not be degraded. Otherwise, an +appropriate error message is printed. If the RAID is recovering, you +will need to wait until the recovery is finished before shutting down; +you can check the progress with the mdstat command. If it is +degraded, seek expert advice.

+
+
+

The script will also not shutdown the system if the FS is in use. To +override this, the -F option can be used, but is not recommended. It +is better to terminate the FS.

+
+
+

The script includes a -p option to display a progress meter for a +recovery if one is active. Whether there is an active recovery or not, +there will not be a shutdown if -p is used. This makes the command +useful for starting a progress meter after a recovery had been +started.

+
+
+
+

7.3. refresh_secondary

+
+

This can be used to refresh a shelf disk for the RAID as a new +secondary disk as part of a standard three (or more) disk rotation.

+
+
+

Initially, the script performs some sanity checks to confirm that the +RAID /dev/md0:

+
+
+
    +
  1. +

    Exists.

    +
    2. +
  2. +

    Is not a clean state, i.e., it needs recovery.

    +
    3. +
  3. +

    Is not already recovering, i.e., is in a recoverable state.

    +
    4. +
+
+
+

Additional checks are performed to confirm that the content the script +intends to copy is where it expects it to be and has the right form. +Any primary disk will be rejected that:

+
+
+
    +
  1. +

    Is not part of the RAID (md0)

    +
    2. +
  2. +

    Has a boot scheme other than the BIOS or UEFI set up as described in the FSL11 Installation Document.

    +
    3. +
+
+
+

To ensure that only an old shelf disk for this system is +overwritten, any secondary disk will be rejected that:

+
+
+
    +
  1. +

    Was loaded (slot keyed-on) before starting the script

    +
    +

    Unless overridden by -A or previously loaded by this or the +blank_secondary script (see below).

    +
    +
    2. +
  2. +

    Is already part of RAID md0

    +
    +

    Which should only happen if run incorrectly with -A (or other +interfering commands have been executed) or the disk has +fallen out of the RAID due to failure.

    +
    +
    3. +
  3. +

    Has a RAID from a different computer, i.e., foreign

    +
    +

    Technically this could also be another RAID from the same computer, but not of a +properly set up FSL11 computer, which should have only the one RAID

    +
    +
    4. +
  4. +

    Has any part already mounted

    +
    +

    Again catching misuse of the -A option.

    +
    +
    5. +
  5. +

    Has a different boot scheme than the primary

    +
    +

    And hence is probably from a different computer.

    +
    +
    6. +
  6. +

    Has a different RAID UUID

    +
    +

    This would be a disk from a different computer. Though whether this +check can actually trigger after the test for a foreign RAID above +remains to be seen.

    +
    +
    7. +
  7. +

    Was last booted at a future TIME (possibly due to a mis-set clock or clocks)

    +
    8. +
  8. +

    Has a higher EVENT count, i.e., is newer

    +
    + + + + + +
    +
    Warning
    +    		 +The check on the EVENT counter is intended to prevent +accidentally using the shelf disk to overwrite a newer disk from the +RAID. This check can be over-run if the primary has run for a +considerable period of time before the refresh is attempted. This +should not be an issue if the refresh is attempted promptly after the +shelf disk is booted for the first time by itself and the RAID was +run on the other disks for more than a trivial amount of time +beforehand. +
    +
    +
    9. +
  9. +

    Has been used (booted) separately by itself

    +
    10. +
  10. +

    Has a different partition layout from the primary

    +
    11. +
  11. +

    Is smaller than the size of the RAID on the primary disk.

    +
    12. +
+
+
+

If any of the checks reject the disk, we recommend you seek expert +advice; please record the error so it can be reported.

+
+
+

The checks are included to make the refresh process as safe as +possible, particular at a station with more than one FSLx computer. +We believe all the most common errors are trapped, but the script +should still be used with care.

+
+
+

If the disk being refreshed is from the same computer and has just +been on the shelf unused since it was last rotated, it is safe to +refresh and should be accepted by all the checks. In other words, + normal disk rotation should work with no problems.

+
+
+

If the primary and/or secondary disks are removable, the user will +be provided with some information about the disks and given an +opportunity to continue with Enter or abort with Ctrl+C. +Typically, if a USB disk is identified as the primary or +secondary, one would not want to continue. However for some +machines, the SATA disks that are the primary and/or secondary may +be marked removable if they are hot swappable, but would still be +appropriate to use.

+
+
+

This script requires the secondary disk to not be loaded, i.e., the +slot turned off, when the script is started. However, it has an +option, -A (use only with expert advice), to “Allow” an already +loaded disk to be used. It is intended to make remote operation +possible and must be used with extra care.

+
+
+

If the disk is turned on (when prompted) during the script, it will +automatically be “Allowed” by both this script and +blank_secondary, which also supports this feature. This allows +(expert use only), after a failed refresh_secondary, running +blank_secondary then rerunning refresh_secondary, all without +having to shutdown, turn the disk off, reboot, start the script, and +turn the disk on for each script.

+
+
+

The refresh will take several hours. You can check the progress with +mdstat. If you prefer, you can run the script with the -p option +to display a progress meter. The system can be used normally while it +refreshing, but it may be a little slow.

+
+
+

The system can rebooted while the refresh is still active, as long as +the neither disk is removed until it is finished. The refresh will +resume automatically after the reboot.

+
+
+ + + + + +
+
Note
+		 +
+

If the primary disk has a larger capacity than the secondary and +the latter is new or has been blanked (typically with +blank_secondary), you may see a warning like:

+
+
+
+
Caution! Secondary header was placed beyond the disk's limits! Moving the
+header, but other problems may occur!
+
+
+
+

In this case, the message is benign and can be ignored if the +primary disk has a partition layout that will fit on the smaller +disk. This should be the case if the system was setup initially as +described in the FS Linux 11 Installation Guide +document. This situation can occur if one (or more) of the disks is +larger than the smallest one, perhaps because it was obtained as a +replacement for a failed disk.

+
+
+
+
+
+

7.4. blank_secondary

+
+

This script should only be used with expert advice.

+
+
+

It can be used to make any secondary disk refreshable, if it is +big enough. It must be used with care and only on a secondary disk +that you know is safe to erase. Generally speaking you don’t want to +use it with a disk from a different FSLx computer, except for very +unusual circumstances; see the Recovery scenarios section below +for some example cases. It will ask you to confirm before blanking.

+
+
+

It will reject any secondary disk that:

+
+
+
    +
  1. +

    Was loaded (slot keyed-on) before starting the script

    +
    +

    Unless you have just loaded it through refresh_secondary's auspices or used +the -A option to “Allow” it (see below).

    +
    +
    2. +
  2. +

    Is still part of the RAID md0

    +
    +

    Which should only happen if run incorrectly with -A (or other +interfering commands have been executed).

    +
    +
    3. +
  3. +

    Has any partition already mounted

    +
    +

    Again catching misuse of the -A option.

    +
    +
    4. +
  4. +

    Has a partition that is in RAID md0

    +
    +

    This is essentially redundant with the “Is still part of the RAID +md0” check above, but is included out of an abundance of caution.

    +
    +
    5. +
  5. +

    Has a partition that is included in any RAID.

    +
    6. +
  6. +

    Is smaller in size than the primary disk

    +
    +

    This may be relaxed with the -A option, if the script is being used +to blank a disk that will not be used in this RAID.

    +
    +
    7. +
+
+
+

If the secondary disk is removable, the user will be provided with some +information about the disk and given an opportunity to continue with +Enter or abort with Ctrl+C. Typically, if a USB disk is +identified as the secondary, one would not want to continue. However +for some machines the SATA disk that is the secondary may be marked +removable if it is hot swappable, but would still be appropriate to +use.

+
+
+

This script requires the secondary disk to not be loaded, i.e., the +slot turned off, when the script is started. However, it has an +option, -A (use only with expert advice), to “Allow” an already +loaded disk to be used. It is intended to make remote operation +possible and must be used with extra care.

+
+
+

If the disk is turned on (when prompted) during the script, it will +automatically be “Allowed” by both this script and +refresh_secondary, which also supports this feature. This allows you +to then run refresh_secondary immediately without having to +shutdown, turn the disk off, reboot, start the script, and turn the +disk on.

+
+
+

The -A will also allow blanking of a disk that is too small to +support the current RAID. This might be used to initialize a disk that +will not be used in the current RAID. As before, use the -A option +only will expert advice.

+
+
+

The -Z option (for expert use only) will “zap” the partition table +and the start of each individual partition with 1 MiB of zeros. Each +additional -Z specified will double the number of zeros written to +the individual partitions. This option may be useful to force a disk +into a state that the installer can handle.

+
+
+
+

7.5. drop_primary

+
+

This script is only for use with expert advice.

+
+
+

This script can be used to drop a primary disk out of a RAID pair +(by marking it as failed) so that it can act as a safety backup during +testing of upgrades or other significant changes.

+
+
+

Initially, the script performs some sanity checks to confirm that the +RAID /dev/md0:

+
+
+
    +
  1. +

    Exists.

    +
    2. +
  2. +

    Is in a clean state, i.e., both disks are present and no recovery is +currently in progress.

    +
    3. +
  3. +

    Contains the primary disk as a member.

    +
    4. +
+
+
+

If the primary disk is removable, the user will be provided with some +information about the disk and given an opportunity to continue with +Enter or abort with Ctrl+C. Typically, if a USB disk is +identified as the primary, one would not want to continue. However +for some machines the SATA disk that is the primary may be marked +removable if it is hot swappable, but would still be appropriate to +use.

+
+
+ + + + + +
+
Note
+		 +This script is non-destructive in nature and its effect can +easily be reversed by running the recover_raid script mentioned +below. +
+
+
+
+

7.6. recover_raid

+
+

This script is only for use with expert advice.

+
+
+

This script can be used to recover a disk, (primary or secondary) +that has fallen out of the RAID array, becoming inactive. (The disk +the system is then running on is referred to as the active disk.) A +disk can fall out of the array for several possible reasons, +including:

+
+
+
    +
  1. +

    A real disk fault of some sort, including one caused by turning it off +whilst it is still in use.

    +
    2. +
  2. +

    Using the mdadm command with -f option to mark it as faulty.

    +
    + + + + + +
    +
    Caution
    +    		 +Using -f is risky and is for experts only. Using it on a +disk that is being refreshed (or is synced) should be relatively easy +to recover from with recover_raid. Using it on the disk that is +being recovered from can cause problems (including possibly crashing +the system). If -f has been used in that way, the system should be +rebooted. At which point, it should restart recovering the RAID. This +is in contrast to having a hard failure of the disk being recovered +from. In that case, you will need to use the +Recover from a shelf disk procedure with the remaining working +disk. +
    +
    +
    3. +
  3. +

    Turning it off whilst the system is shutdown and booting without it.

    +
    4. +
  4. +

    Using the drop_primary script.

    +
    5. +
+
+
+

This script is designed to be used only with a set of disks that were +most recently used together in an active RAID. It is recommended +only to use this script if the key switches for the disks have not +been manipulated since the inactive disk fell out of the RAID; in +this case it should always be safe. The script normally works on +md0, but a different md device can be specified as the first +argument.

+
+
+ + + + + +
+
Important
+		 +This script must NOT be used if the inactive disk has +been changed in any way e.g., by being used (booted) separately (which +is caught by the script) or refreshed against some other disk, or if +the active disk has been used to refresh any other disk in the +interim. In particular, this script must NOT be used to refresh a +shelf disk — only use refresh_secondary for that purpose. +
+
+
+ + + + + +
+
Note
+		 +The inactive disk is either failed or missing. It is +failed if it was either marked failed by hand or dropped out of the RAID due to disk errors. +It is missing if either the system was rebooted with the disk +failed or physically missing or it was manually marked removed. You +can check which state an inactive disk is in with +mdadm --detail /dev/md0 — which lists failed as +faulty but a missing disk will not appear at all. +
+
+
+ + + + + +
+
Tip
+		 +It is okay to use this script even if the inactive disk fell +out the RAID a (long) long time ago (in a galaxy far, far away) and/or +there have been extensive changes to the active disk. It is also +okay to use if the system was rebooted (even multiple times) or the +active disk was used (booted) separately by itself since the +inactive disk fell out of the RAID. +
+
+
+ + + + + +
+
Note
+		 +In extreme cases, the changes since the inactive disk fell out +of the RAID may be too extensive to allow for a recovery with this +script. You may get a message similar to mdadm: --re-add for …​ to +device /dev/md0 is not possible. If this happens, seek expert advice. +It should be possible to recover by blanking and then refreshing the +inactive disk. (If the inactive disk is in the primary slot, it +will be necessary to reboot with the active disk installed in the +primary slot then run blank_secondary and refresh_secondary, and +finally shutdown and, reverse the disks between the slots and reboot.) +Alternatively, it should be possible to use the --add option of the +mdadm command to add the inactive disk to the RAID; this will +take as long as a refresh_secondary. +
+
+
+

The script will refuse to recover the RAID if the RAID:

+
+
+
    +
  1. +

    Does not need recovery

    +
    2. +
  2. +

    Is not in a recoverable state, e.g., is already recovering

    +
    3. +
+
+
+

or if any missing disk:

+
+
+
    +
  1. +

    Has a later modification TIME than the active disk

    +
    2. +
  2. +

    Has a higher EVENT count, i.e., is newer, than the active disk

    +
    3. +
  3. +

    Has been used (booted) separately (as mentioned above in the +IMPORTANT item)

    +
    4. +
+
+
+

or if no matching missing disk can be found.

+
+
+

The recovery may be fairly quick, as short as a few minutes, if the +inactive disk is relatively fresh. You can check the progress with +mdstat. If you prefer, you can run the script with the -p option +to display a progress meter. The system can be used normally while it +recovering, but it may be a little slow.

+
+
+
+

7.7. raid-events

+
+

The mdmonitor service can be configured to use the raid-events +script to send email reports on RAID rebuilds and checks. This is most +useful for getting reports for the start and end of a RAID build +triggered by refresh_secondary. The script will also report on the +start and end of any other RAID rebuilds, including those triggered by +the recover_raid script. Checks are triggered periodically to verify +the integrity of the RAIDs.

+
+
+

The emails are sent to root, then typically redirected to oper,` +and then forwarded to off-system accounts that may have their email +read more frequently. There are four different possible subject lines +used in the emails:

+
+
+
    +
  • +

    Rebuild Running on device

    +
    + + + + + +
    +
    Note
    +    		 +Sometimes for a rebuild started by refresh_secondary, this +message may be sent about 20 minutes after the rebuild has started. +The cause of this is not entirely understood, but the message is +eventually sent. +
    +
    +
    • +
  • +

    Rebuild Ended state on device

    +
    • +
  • +

    Check Running on device

    +
    • +
  • +

    Check Ended state on device

    +
    • +
+
+
+

where:

+
+
+
    +
  • +

    device is the RAID device, e.g., /dev/md/0

    +
    • +
  • +

    state is OKAY if the final state was not degraded; DEGRADED, +if it was degraded.

    +
    • +
+
+
+

The body of each email is the output of the mdstat script at the +time the message was sent.

+
+
+

7.7.1. Checks

+
+

The checking process is triggered by /etc/cron.d/mdadm on +the first Sunday of each month. It uses the +/usr/share/mdadm/checkarray script and takes a similar amount of time +as a rebuild of the RAID triggered by refresh_secondary.

+
+
+
+

7.7.2. Installing raid-events

+
+

To install the script, use the following commands as root:

+
+
+
+
cd /usr/local/sbin
+cp ~/fsl11/RAID/raid-events .
+chmod u+x raid-events
+cat <<EOF >>/etc/mdadm/mdadm.conf
+
+PROGRAM /usr/local/sbin/raid-events
+EOF
+
+
+
+

And then reboot.

+
+
+
+

7.7.3. Disabling checking

+
+

If the checking process causes performance problems at inconvenient +times, there are at least three options for dealing with it:

+
+
+
    +
  • +

    Disable the AUTOCHECK option in /etc/default/mdadm

    +
    +

    This is suitable if the RAID is rebuilt monthly using +refresh_secondary. In this case, the check is superfluous.

    +
    +
    • +
  • +

    Change the time at which it runs as configured in +/etc/cron.d/mdadm

    +
    • +
  • +

    Cancel a running check, with:

    +
    +
    +
    /usr/share/mdadm/checkarray --cancel --all
    +
    +
    +
    • +
+
+
+
+
+

7.8. refresh_spare_usr2

+
+

This script is not part of RAID operations per se, but is included in +this document for completeness. In a two system configuration +(operational and spare), it is used to make a copy of the +operational system’s /usr2 partition on the spare system. +Normally this partition holds all the operational FS programs and +data.

+
+
+

A full description of the features of the script are available from +the refresh_spare_usr2 -h output.

+
+
+ + + + + +
+
Important
+		 +This script should be installed on the spare system only. +
+
+
+ + + + + +
+
Tip
+		 +
+

A recommended monthly backup strategy is to do a disk rotation on both +systems. Once the RAIDs on both systems are recovering you can +log-out of both systems and then login into the spare system again +to start refresh_spare_usr2.

+
+
+

While a refresh_spare_usr2 with two nearly synchronized /usr2 +partitions is fairly fast, the recovery of the RAIDs may increase the +amount of time required by about a factor of three.

+
+
+

Once refresh_spare_usr2 completes, it is safe to reboot, even if a +recovery is still ongoing. The only requirement is to reboot the +spare system before the FS is run on it again.

+
+
+

A feature of this approach is that it will make the spare system +shelf disk a deeper back-up than the spare system RAID disks.

+
+
+
+
+

7.8.1. Installing refresh_spare_usr2

+
+ + + + + +
+
Warning
+		 +For this script to work most usefully, the operational and +spare systems should have the same set-up including particularly the +same user accounts with the same UIDs and GIDs in parallel for all +accounts that have home directories on /usr2, as well as other OS +set-up information the FS may depend on such as /etc/hosts and +/etc/ntp.conf. +
+
+
+ + + + + +
+
Tip
+		 +If you don’t want to or are unable to use the forced command +approach below for the root account, you may find the approach of +using sudo in a regular account a usable alternative. For details on +that approach, please see the +Installing +refresh_spare_usr2 with CIS hardening subsection of the +CIS hardening for FSL11 document. +
+
+
+

All the steps below must be performed as root on the specified +system. You should read all of each step and sub-step before following +it.

+
+
+
    +
  1. +

    On the operational system:

    +
    +
      +
    1. +

      Temporarily set sshd to allow root login:

      +
      +
        +
      1. +

        Edit /etc/ssh/sshd_config

        +
        +

        Add an uncommented line (or change an existing line) for +PermitRootLogin to set it to yes

        +
        +
        2. +
      2. +

        Restart sshd. Execute:

        +
        +
        +
        systemclt restart sshd
        +
        +
        +
        3. +
      +
      +
      2. +
    +
    +
    2. +
  2. +

    On the spare system:

    +
    +
      +
    1. +

      Make sure the operational system is represented in the +/etc/hosts file.

      +
      +

      If it is not already there, add it. It is recommended that it be given +a simple alias for routine use.

      +
      +
      2. +
    2. +

      Install refresh_spare_usr2. Execute:

      +
      +
      +
      cd /usr/local/sbin
+cp -a /root/fsl11/RAID/refresh_spare_usr2 refresh_spare_usr2
+chown root.root refresh_spare_usr2
+chmod a+r,u+wx,go-wx refresh_spare_usr2
      +
      +
      +
      3. +
    3. +

      Customize refresh_spare_usr2, following the directions in the +comments in the script:

      +
      +
        +
      1. +

        Comment-out the lines (add leading #s):

        +
        +
        +
        echo "This script must be customized before use.  See script for details."
+exit 1
        +
        +
        +
        2. +
      2. +

        Change the operational in the line:

        +
        +
        +
        remote_node=operational
        +
        +
        +
        +

        to the alias (preferred), FQDN, or IP address of your operational +system.

        +
        +
        3. +
      +
      +
      4. +
    4. +

      Create and copy a key for root. Execute:

      +
      + + + + + +
      +
      Caution
      +      		 +If root already has a key, you only use the second command +below, to copy it to the spare system. +
      +
      +
      + + + + + +
      +
      Tip
      +      		 +It is recommended to not set a passphrase. +
      +
      +
      +
      +
      ssh-keygen
+ssh-copy-id root@operational
      +
      +
      +
      5. +
    +
    +
    +

    where operational is the alias, name, or IP of your operational +system.

    +
    +
    3. +
  3. +

    On the operational system:

    +
    +
      +
    1. +

      Set the root account to only allow a forced command with ssh:

      +
      +
        +
      1. +

        Replace the ssh-rsa at the start of the line (probably the only +one) in ~root/.ssh/authorized_keys for the root account on the +spare system with:

        +
        +

        command="rrsync -ro /usr2" ssh-rsa

        +
        +
        + + + + + +
        +
        Tip
        +        		 +If your spare system is registered with DNS, you can provide +some additional security by adding from="node"  (note +the trailing space) at the start of the line, where node is the +FQDN or IP address of the spare system. It may be necessary to +provide the FQDN, IP address, and/or alias of the spare system in a +comma separated list in place of node to get reliable +operation. +
        +
        +
        2. +
      2. +

        Set sshd to only allowed forced commands for root by replacing +yes with forced-commands-only on the uncommented PermitRootLogin +line.

        +
        3. +
      3. +

        Restart `sshd. Execute:

        +
        +
        +
        systemctl restart sshd
        +
        +
        +
        4. +
      +
      +
      2. +
    +
    +
    4. +
+
+
+
+

7.8.2. Using refresh_spare_usr2

+
+
    +
  1. +

    As part of a monthly backup, you would usually start a disk rotation +on both the operational and spare systems first. Once both systems +are recovering, you should log out of both systems.

    +
    + + + + + +
    +
    Important
    +    		 +Before proceeding, make sure that no one is logged into +either system and that no processes are running on /usr2 on either +system, particularly the FS. +
    +
    +
    2. +
  2. +

    Login on the spare system. The best choice for this is as root +on a local virtual console text terminal.

    +
    + + + + + +
    +
    Tip
    +    		 +
    +

    Logging in as a non-root user will also work. Any available means +can be used: a text console, ssh from another system (preferably not +the operational system), or the graphics X11 display. You must then +promote to root using su (for CIS hardened systems: +root_account, or execute the script with sudo).

    +
    +
    + + + + + +
    +
    Caution
    +    		 +If you use the -I option (which would not normally be +used), you must change your working directory to be somewhere off of +/usr2, e.g., /tmp, before using su (or root_account or +sudo). We have made an effort to make this reliable, but there still +may be a chance that the script will fail with the error +umount: /usr2: target is busy.. If this happens, +you can try to recover by simply rerunning the script. This should +work because although the error happens in the critical phase (see +refresh_spare_usr2 -h), the /usr2 partition does not get +unmounted when it occurs. It might take more than one try of rerunning +to achieve success. +
    +
    +
    +
    +
    3. +
  3. +

    Execute the script:

    +
    +
    +
    refresh_spare_usr2
    +
    +
    +
    + + + + + +
    +
    Note
    +    		 +
    +

    On a CIS hardened system (see CIS hardening for +FSL11), you may be able to use:

    +
    +
    +
    +
    sudo refresh_spare_usr2
    +
    +
    +
    +

    from an AUID account

    +
    +
    +
    +
    +

    Answer the question y if it is safe to proceed.

    +
    +
    4. +
  4. +

    Log out of the system.

    +
    5. +
  5. +

    Wait until the script has finished before logging in again and +resuming other activities on the systems.

    +
    +

    An email will be sent to root when the script finishes. If your +email to root is being forwarded to a mailbox off the system, you +can use receipt of that message (and that it shows no errors) as the +indication that it finished successfully.

    +
    +
    +

    Alternatively, you can examine the logs (before starting the script) +in /root/refresh_spare_usr2_logs to see how long previous script +uses took. When at least that much time has elapsed, you can login +and can check the log for the current script use to verify that it has +finished.

    +
    +
    + + + + + +
    +
    Caution
    +    		 +
    +

    Generally speaking, it is best to not login to either the spare or +operational system while the script is running. Under normal +circumstances the script should run quickly enough that this does not +cause a significant burden. If it is necessary to login to either +system, the following paragraphs in this CAUTION cover the relevant +considerations.

    +
    +
    +

    If you do login to the spare system, it is best to not use an +account with a home directory on the /usr2 partition (logging in as +root on a text console is okay) or otherwise access that partition +while the script is running. In any event, activity on /usr2 should +be minimized.

    +
    +
    +

    It is possible to use the operational system while the script is +running if necessary, but this should be avoided if possible and +activity on the /usr2 partition should be minimized. You should not +expect any changes on the operational system /usr2 that occur +after the script starts to be propagated to the spare system. If any +files are deleted before they can be transferred, there will be a +warning file has vanished: "file", for each such file, and +there will be a summary warning that starts with rsync warning: some +files vanished before they could be transferred, but without +additional warnings or errors, the transfer should otherwise be +successful.

    +
    +
    +

    In case you have logged into either system while the script is +running, you can touch-up the copy on the spare system, by rerunning +the script after logging out.

    +
    +
    +
    +
    6. +
  6. +

    If the script finished with no problems, you can reboot the spare +system as soon as is convenient. You may reboot even if the RAID is +recovering, but you can wait until the recovery is complete. The only +requirement is to reboot before the FS is run again on the spare +system.

    +
    7. +
+
+
+
+
+
+
+

8. Multiple computer set-up

+
+
+

You may have more than one FSL11 computer at a site, either an +operational and spare for one system and/or additional computers for a +additional systems. In this case, we recommend that you do a full setup of +each computer from scratch from FSL11 installation notes. The main, but not only, +reason for this is to make sure each RAID has a unique UUID, so the +refresh_secondary script will be able to help you avoid accidentally +mixing disks while doing a refresh. While in principle is it possible +to do one set-up and clone the configuration to more disks and then +customize for each computer, we are not providing detailed +instructions on how to do that at this time.

+
+
+

It is recommended that the network configuration on each machine be +made independent of the MAC address of the hardware. This will make it +possible to move a RAID pair to a different computer and have it work +on the network. Please note that the IP address and host name is tied +to the disks and not the computers. For information on how to +configure this, please see the (optional) +Stabilize network +configuration section of the FS Linux 11 +Installation Guide document.

+
+
+

The configuration of the system outside of the /usr2 partition +between operational and spare computers should be maintained in +parallel so that the same capabilities are available on both. In +particular, any packages installed on one should also be installed +on the other. In addition, it is important that the user and group +IDs of all users on the operational and spare computers be same. It +should not be necessary to maintain parallelism with OS updates, but that +is recommended as well. It is recommended to maintain maintenance parallelism +with other independent operational/spare systems at a site as well (this may + enable additional recovery options in extreme cases).

+
+
+
+
+

9. Recovery scenarios

+
+
+

The setup provided by FSL11 provides several layers of recovery in +case of problems with the computers or the disks. Each system has a +shelf disk, which can serve as a back-up. Additionally if there is a +spare computer for each operational computer, there are additional +recovery options. If there are other FSL11 computers at the site, it +may be possible in extreme cases to press those computers and/or disks into +service, particularly if they have been maintained in parallel.

+
+
+

A few example recovery scenarios are described below in rough order of +likelihood of being needed. None of them are very likely to be needed, +particularly those beyond the first two.

+
+
+ + + + + +
+
Important
+		 +In any scenario, if disks and/or a computer have failed, + they should be repaired or replaced as soon as feasible. +
+
+
+

9.1. Operational computer failure

+
+

This might be caused by a power supply or other hardware failure. +If the contents of the operational RAID are not damaged, the RAID pair +can be moved to the spare computer until the operational computer is +repaired. Once the RAID has been moved, whether the contents have +been damaged can be assessed. It will be necessary to move +connections for any serial/GPIB devices to the spare computer as well.

+
+
+ + + + + +
+
Tip
+		 +
+

If the disks do not connect to network after first +booting in a different computer:

+
+
+
    +
  1. +

    Shut the system down.

    +
    2. +
  2. +

    Remove the power cord.

    +
    3. +
  3. +

    Press and hold the power button for 15 or more seconds.

    +
    +

    The goal is drain any residual energy in the computer in order to completely +reset the NIC.

    +
    +
    4. +
  4. +

    Reboot and try again.

    +
    5. +
+
+
+

This has been seen to solve the problem, perhaps because it forces the +NIC to re-register with ARP. Waiting longer may also solve the problem.

+
+
+
+
+
+

9.2. One disk in the operational computer RAID fails

+
+

This should not interrupt operations. The computer should continue to +run seamlessly on the remaining disk. If the system is rebooted in +this state, it should use the working disk. At the first opportunity, +usually after operations, the recover_raid script can be tried to +restore the disk to the RAID. If that doesn’t work, the disk may have +failed and may need to be replaced (it may worthwhile to try blanking +and refreshing it first). If the disk has failed, it should be removed +and a disk rotation should be performed (with the still good disk in +the primary slot) to refresh the shelf disk and make a working +RAID. The failed disk should be repaired or replaced with a new disk +that is at least as large. The blank_secondary script should be used +to erase the new disk before it is introduced into the rotation +sequence. See the Initialize a new disk section above for full +details on initializing a new disk.

+
+
+
+

9.3. Operational computer RAID corrupted

+
+

As well as a large scale corruption, this can include recovery from +accidental loss of important non-volatile files. This would generally +not include .skd, .snp, and .prc files; those can be more easily +restored by generating them again. It also can be used to recover +from a bad OS patch (which is extremely unlikely). That is easier to +manage if the patches were applied just after a disk rotation (see +also the Recoverable testing section).

+
+
+

In this case, the shelf disk can be used to restore the system to +the state at the time of the most recent rotation. To do this, follow +the procedure in Recover from a shelf disk section above. The +system can be used for operations once the RAID is recovering for the +first refresh in the procedure. All needed volatile operational files that were +created/modified after the last disk rotation will need to be +recreated. Then as time allows, the other disk can recovered by +finishing the procedure in Recover from a shelf disk section.

+
+
+

If the first disk that is tried for blanking and recovery doesn’t work, the +other one can be tried. If neither works, it should be possible to run on just +what was the shelf disk until a fuller recovery is possible, probably with +replacements for the malfunctioning disks.

+
+
+

This approach could also be used for a similar problem with the +spare computer and using its shelf disk for recovery.

+
+
+

This approach of this section should not be used if a problem with the +operational computer caused the damage to its RAID. In that case, +follow the +Operational computer RAID corrupted and operational computer failure +subsection below.

+
+
+
+

9.4. Operational computer RAID corrupted and operational computer failure

+
+

This might happen if the operational computer is exposed to fire +and/or water. In this case, there are two options. One is switching to +using the spare computer as in the +Loss of operational computer and all its disks subsection below. +The other is to use the operational computer’s shelf disk in the +spare computer, either by itself or by making a ersatz RAID by +blanking the spare computer’s shelf disk and refreshing it from +the operational computer’s shelf disk.

+
+
+

In the latter scenario, be sure to preserve the original working RAID +from the spare computer. All needed volatile operational files that +were created/modified after the last operational computer disk +rotation will need to be recreated. It will be necessary to move +connections for any serial/GPIB devices to the spare computer as well. +However, it will not be necessary to enable any daemon’s like +metserver and metclient as it would be in the former scenario; this +may be a significant time saver.

+
+
+
+

9.5. Loss of all operational computer disks

+
+

If the RAID and shelf disk on the operational computer are beyond +recovery, the RAID pair from the spare computer can be moved to the +operational computer. All needed volatile operational files that +were created/modified after the last refresh_spare_usr2 will need to be +recreated. If daemons like metserver and metclient are needed, + they will need to be enabled.

+
+
+

This approach should not be used if a problem with the operational +computer caused the damage to its RAID. In that case, follow the +Operational computer RAID corrupted and operational computer failure +subsection above.

+
+
+
+

9.6. Loss of operational computer and all its disks

+
+

In this case, operations should be moved to the spare computer until +the operational computer is repaired or replaced. It will be +necessary to move connections for any serial/GPIB devices to the +spare computer as well. If daemons like metserver and +metclient are needed, they will need to be enabled. All needed +volatile operational files that were created/modified after the last +refresh_spare_usr2 will need to be recreated.

+
+
+
+
+
+ + + \ No newline at end of file