Fix usage of FAI's shell.log vs. scripts.log We check for scripts.log, but still use shell.log, d'oh :)
Rework memtest handling, incl. usage of latest available memtest file If we try to copy the memtest86+.bin file as shipped with memtest86+ versions <=5.01-3.1, we either copy such an old file from the build directory (grml_chroot), or if that doesn't exist (e.g. because the memtest86+ package isn't installed), we might end up copying an old memtest file from the build host system instead. AS ${BUILD_OUTPUT}/boot/addons/memtest exists already then, we don't update the file any longer (e.g. from a more recent memtest86+ package), and therefore end up with an old and outdated memtest version in BIOS boot, while EFI boot provides a more recent memtest version. This is inconsistent and unexpected behavior. So instead try to use the most recent version of memtest86* files first, and only then fall back to the old memtest86+ <=5.01-3.1 file behavior. While looking into this, I also noticed that until memtest86+ versions <=6.10-2 it used to be named memtest86+x32.bin, while as of memtest86+ versions >=6.10-3 it's memtest86+ia32.bin instead. As we have version 6.10-4 in bookworm/stable and only pre-6 version 5.01-3.1 in e.g. bullseye/oldstable, let's skip any backwards compatibility for memtest86+x32.bin. Furthermore while at it, drop duplicate execution of `copy_addon_file memtest86+x64.bin /boot addons` and `copy_addon_file memtest86+x32.bin /boot addons` to avoid ending up with further duplicate files on the ISO. So related to https://github.com/grml/grml-live/issues/128, we now ship /boot/addons/memtest on each of grml64 + grml32, and respectively only /boot/addons/memtest86+x64.efi on grml64 only and /boot/addons/memtest86+ia32.efi on grml32 only. To rename the files into FAT16/8.3 compatible "/boot/addons/memtest", let's provide a proper return code from within copy_addon_file() if we couldn't find any matching file. Related to https://github.com/grml/grml-live/issues/128 Closes: https://github.com/grml/grml/issues/178
Initial arm64 / aarch64 support Relevant changes: * arm64 (ARM64 in terms of FAI class) is considered a valid and supported architecture within grml-live now; updated documentation, configuration files and zsh completion accordingly * arm64 was enabled as supported architecture in grml repositories * the architecture specific grml-scripts + grml-scripts-core packages got built for arm64 and uploaded to the grml-testing repository * memtest86+ + syslinux aren't available on arm64, therefore move memtest86+ to Recommends to avoid ending up with an `Arch: any` package, also depend on either syslinux or syslinux-efi * Software related changes in GRML* (other than the obvious linux-image-arm64 and further arm64 specific packages): * GRMLBASE: - grml2usb is I386 + AMD64 specific for now (if we want to use grml2usb on arm64 as well, we'd need to port it) - grub-efi-amd64-bin, grub-efi-ia32-bin + grub-pc as well as syslinux, syslinux-common + syslinux-utils all are also I386 + AMD64 specific * GRML_FULL: - cciss-vol-status, cmospwd, cpuid, grub-pc-bin + ltrace all are I386 + AMD64 specific (so not available for arm64) - grml-terminalserver is I386 + AMD64 specific for now (at least until we ported grml2usb for arm64) * /etc/grml/fai/config/scripts/GRMLBASE/44-grub: invocation of grub-mkimage is I386 + AMD64 specific (at least until we identify whether there's an actual use-case for our generated /boot/grub/core.img) * /etc/grml/fai/config/scripts/GRMLBASE/45-grub-images: we use /boot/bootaa64.efi for the EFI boot image name for arm64, though need to skip the unavailable efi_uga module and also skipt the i386-pc architecture, since grub on arm64 doesn't provide /usr/lib/grub/i386-efi Known TODOs: * decide official naming for Grml on arm64 (also keep e.g. `update-grml-rescueboot -t small -a 64` in mind!) * grml2usb isn't available for arm64 yet, and therefore also grml-terminalserver isn't available for arm64 * when booting the resulting ISO inside an arm64 / aarch64, the default terminal seems to start up as /dev/ttyAMA0 (instead of /dev/tty1, as expected and handled via our getty@tty1.service) * look into SecureBoot support Development sponsored by netcup GmbH
Support FAI's newer scripts.log, as used with FAI versions >=6.0 Older FAI versions used to log the output of the scripts to shell.log, now as of FAI versions >=6.0 they end up in scripts.log instead, so also check those to identify failures during builds.
Deprecate FAI's make-fai-nfsroot.conf Since FAI v4.0 make-fai-nfsroot.conf is no longer relevant, even in Debian oldoldstable AKA buster we have FAI v5.8.4, so there's no point in further supporting this. Sadly we still need the temporary /etc/grml/fai/nfsroot.conf for FAI's dirinstall usage: | fai -C /etc/grml/fai -s file:////etc/grml/fai/config -cDEBIAN_BOOKWORM,GRMLBASE,GRML_SMALL,AMD64 - u grml dirinstall .... FAI relies on this file, otherwise failing with: | /etc/grml/fai/nfsroot.conf not found. | You may want to install the package fai-server
SW: switch from isc-dhcp-client to dhcpcd ISC DHCP (AKA isc-dhcp-client in Debian) is no longer maintained by ISC, see https://kb.isc.org/docs/isc-dhcp-eol-dates and also https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.en.html#deprecated-components Note that this also needs adjustments in FAI_DEBOOTSTRAP_OPTS, as debootstrap by default pulls ins isc-dhcp-client and isc-dhcp-common via ifupdown, due to its `Recommends: isc-dhcp-client | dhcp-client`. Thanks: András Korn for the suggestion
No longer bootstrap with --no-merged-usr The transition to usrmerge was done in Debian, see https://lists.debian.org/debian-devel-announce/2022/09/msg00001.html Debian/bookworm AKA v12 will only support the merged-/usr layout. Systemd is also dropping support for unmerged-usr systems (see https://lists.freedesktop.org/archives/systemd-devel/2022-September/048352.html). Also debootstrap complains about this nowadays: | W: Upgrading non-merged-/usr environments post-bookworm is unsupported. Only do this for CI/QA infrastructure that will be re-bootstrapped rather than upgraded. So no longer enable --no-merged-usr by default, if anyone should still need it, it can be set via custom $FAI_DEBOOTSTRAP_OPTS. Thanks: Csillag Tamas for reporting
Increase EFI image size when using Secure Boot on amd64 Now as of git commit 721a473 the disk usage of the Secure Boot files increased and 4MB are no longer enough. We don't support Secure Boot for i386, so when either building for i386 or amd64 without Secure Boot then keep the default 4MB size. But if Secure Boot is not disabled and when not building for i386 (:= amd64) then increase EFI size to 8MB. Thanks: János Pásztor for bugreport and initial patch
Support Memtest86+ with UEFI Memtest86+ up and until v5.01-3.1 didn't provide (U)EFI support. Starting with v6.00, the file memtest86+.bin no longer exists, but instead there are: * /boot/memtest86+x32.bin * /boot/memtest86+x32.efi * /boot/memtest86+x64.bin * /boot/memtest86+x64.efi So let's start adding support for memtest86 with (U)EFI. Hopefully we don't need FAT16/8.3 compatibility any longer, so let's try using the files as-is. Closes: https://github.com/grml/grml-live/issues/109
Replace egrep usage with grep -E grep 3.8 deprecated support for egrep + fgrep, and now prints a warning on stderr: | egrep: warning: egrep is obsolescent; using grep -E | fgrep: warning: fgrep is obsolescent; using grep -F Development time sponsored by Sipwise GmbH
Redesign RELEASE_INFO handling + fix variable replacements within templates There's no point in setting "$GRML_NAME $VERSION - $RELEASENAME" at different places, while it's supposed to be identical to $RELEASE_INFO, so let's unify it. Also ensure, that $fixed_release_info is set at the appropriate place. Otherwise it might be missing, when adjust_boot_files() is invoked from within grub_setup(). This has been observed to fail for example with SecureBoot enabled, and using grml-live's `-b` option: [*] Secure Boot is enabled [mode: debian] [x] Variable fixed_squashfs_name is unset, can't adjust %SQUASHFS_NAME% in templates. [x] Variable fixed_release_info is unset, can't adjust %RELEASE_INFO% in templates. [*] Generated 64-bit Secure Boot (debian) EFI image /srv/grml-live/grml64-forensic_2022.03-1/grml_chroot//boot/efi.img Furthermore, $fixed_squashfs_name shouldn't be used in adjust_boot_files(), as the boot templates usually include "toram=%SQUASHFS_NAME%", while this shouldn't be stripped of (as it would break the actual toram feature then). Instead, the $fixed_squashfs_name should be used only with specific isolinux/syslinux config files, where the length matters for the appropriate line length for its look'n'feel. This work was funded by Grml-Forensic.
Use grubx64.efi file from grml_chroot, instead of relying on host system We don't want to depend on the host system, but instead use the grubnetx64.efi.signed or grubnetx64.efi files from the grml_chroot as grubx64.efi for PXE boot. Related to commit 871fc96fc This work was funded by Grml-Forensic.
buildinfo generation: avoid error message with older versions of jo We use the jo(1) tool for generating /conf/buildinfo.json. To avoid syntax errors when sorting the list of keys/values, we also end the last entry (wayback_date=...) with a "\", so the next line is considered, and reordering the list is simple. The usage of ":" isn't supported by older versions of jo, though: | % jo -v | jo 1.0 | % jo -p foo=bar : | Argument `:' is neither k=v nor k@v | { | "foo": "bar" | } FTR, newer versions don't cause any failure messages: | % jo -v | jo 1.3 | % jo -p foo=bar : | { | "foo": "bar" | } Since there's no strong need to use ":", the error message might look like the /conf/buildinfo.json isn't generated (while it still is), and I'd like to keep sorting the list easy and error free, let's fix this. Usage of "--" usually ends list of command line options, let's use this pattern instead.
isolinux: fix toram=... variable usage within isolinux configs Fixup for commit 9453222c0, where we introduced usage of $fixed_squashfs_name. This variable was set up only *after* adjust_boot_files function got invoked for the isolinux configuration files. As a result the isolinux configuration files ended up with their template placeholder settings included ("toram=%SQUASHFS_NAME%"). Fix ordering, but also warn in adjust_boot_files() if the the $fixed_squashfs_name + $fixed_release_info variables are unset. Thanks: Chris Hofstaedtler for the bug report Closes: https://github.com/grml/grml/issues/179
Immediately bail out on errors when generating the ISO fails For example when the file system is (close to) full and trying to generate the ISO, the $MKISOFS command line fails with exit code 32, but we continue to operate on the ISO filename with the isohybrid related actions and might overwrite the ISO by resetting it.
Provide information how ISO was generated in file conf/buildinfo.json Relevant changes: * Depend on "jo" tool, used for generating the json output in conf/buildinfo.json * Do not rewrite $SQUASHFS_NAME content, instead handle its fixed output format via $fixed_squashfs_name * Same for $RELEASE_INFO content, handled via $fixed_squashfs_name now * Always set $SQUASHFS_BINARY if unset, to be able to log a proper command line, even if mksquashfs stage gets skipped FTR: the generated file is accessible on the live system under /run/live/medium/conf/buildinfo.json Closes: https://github.com/grml/grml-live/issues/44
netboot creation: no longer compress the tarball + only generate sha256 checksum file Compressing the tarball takes several seconds, while there's usually no change in file size, as pretty much all its files are already compressed. By no longer compressing the tarball we reduce build time. Also no longer generated sha1 and sha512 checksum files, we agreed to only rely on sha256 checksum files nowadays. Related to https://github.com/grml/grml/issues/120
No longer produce md5, sha1 + sha512 checksums, but only sha256 We no longer rely on those checksum files and it increases build time. It should be enough to generate just the sha256 checksum file. Related to https://github.com/grml/grml/issues/120
Revert "grml-live: clean up MIRROR_DIRECTORY if provided." This reverts commit c78f58bfee79af61276e3e91457a1b0faa55bbf5. While deleting the bind-mount destination of mirror directories is fine in theory, we end up deleting empty base directories as well, which is unexpected and unwanted behavior. See https://github.com/grml/grml-live/pull/104 for the related discussion, this issue needs further attention yet before we can include this in a new release of grml-live.
grml-live: clean up MIRROR_DIRECTORY if provided. Be like a ninja.