From: Michael Prokop Date: Wed, 13 Mar 2024 11:36:30 +0000 (+0100) Subject: grml2usb docs: drop references to deprecated grml-medium X-Git-Url: https://git.grml.org/?p=grml2usb.git;a=commitdiff_plain;h=HEAD;hp=7656c188b383de809123505c6fd9d598b7cb7a7d grml2usb docs: drop references to deprecated grml-medium grml-medium is gone since many years, so there's no point in further referring to it Thanks to Christoph Biedl for spotting --- diff --git a/.github/workflows/check-full.yml b/.github/workflows/check-full.yml new file mode 100644 index 0000000..4536d0f --- /dev/null +++ b/.github/workflows/check-full.yml @@ -0,0 +1,39 @@ +name: Code Testing + +on: + push: + pull_request: + schedule: + - cron: '42 1 * * *' + +jobs: + codecheck: + runs-on: ubuntu-latest + name: Run codecheck + + steps: + - name: Checkout source + uses: actions/checkout@v2 + + - name: pip install wheel (to make install black work) + run: pip3 install wheel + + - name: pip install flake8, isort + black, vulture + run: pip3 install flake8 isort black vulture + + - name: Codecheck execution + run: make codecheck + + unittests: + runs-on: ubuntu-latest + name: Run unit tests + + steps: + - name: Checkout source + uses: actions/checkout@v2 + + - name: Install pytest + run: pip3 install pytest + + - name: Run Pytest + run: pytest diff --git a/Makefile b/Makefile index 7a3f516..dd12e74 100644 --- a/Makefile +++ b/Makefile @@ -42,6 +42,10 @@ codecheck: flake8 grml2usb isort --check-only grml2usb black --check grml2usb + vulture grml2usb test/grml2usb_test.py + +test: + pytest # graph: # sudo pycallgraph grml2usb /grml/isos/grml-small_2008.11.iso /dev/sdb1 diff --git a/debian/changelog b/debian/changelog index bde4e0a..f818fc7 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,58 @@ +grml2usb (0.19.2) unstable; urgency=medium + + * [2955aaf] codecheck: reformat with black, version 23.1.0 + (Closes: #1031466) + + -- Michael Prokop Mon, 20 Feb 2023 11:33:34 +0100 + +grml2usb (0.19.1) unstable; urgency=medium + + * [27eba4a] Replace egrep usage with grep -E + * [ec28a50] Bump Standards-Version to 4.6.2 + + -- Michael Prokop Wed, 08 Feb 2023 08:27:50 +0100 + +grml2usb (0.19.0) unstable; urgency=medium + + [ Manuel Rom ] + * [40221eb] Add Github action workflows for CI/CD + + [ Michael Prokop ] + * [3f77679] Fix vulture usage and add vulture to Build-Depends + * [a6cce22] Github action: do not install virtualenv + + python3-setuptools. Thanks to Chris Hofstaedtler + * [78ae858] grml2iso: support parallel execution + * [8126bbf] grml2iso: execute under pipefail + + -- Michael Prokop Mon, 21 Mar 2022 16:19:54 +0100 + +grml2usb (0.18.5) unstable; urgency=medium + + * [01ed11d] Fix --grub and --syslinux handling. Thanks to Ralf Moll for + the bugreport + + -- Michael Prokop Fri, 22 Jan 2021 10:32:43 +0100 + +grml2usb (0.18.4) unstable; urgency=medium + + [ Michael Prokop ] + * [3664e4a] Code cleanups. Thanks to Chris Hofstaedtler for review + + feedback + * [1dfee7d] Use "GRML" as FAT label when creating the file system + * [272b66e] Fix race condition with blockdev/BLKRRPART due to lack of + fsync. Thanks to dann frazier for bug report and + patch (Closes: #975015) + * [100a957] Bump Standards-Version to 4.5.1 + + [ Darshaka Pathirana ] + * [7656c18] Use consistent case of 'USB' and 'Grml' + + [ Manuel Rom ] + * [6f3eb52] Add unit testing capabilities and basic tests for + check_for_usbdevice + + -- Michael Prokop Fri, 27 Nov 2020 17:56:02 +0100 + grml2usb (0.18.3) unstable; urgency=medium * [9e7dd96] codecheck: fix flake8 issues with versions >=3.8.3. Thanks diff --git a/debian/control b/debian/control index 05d2c41..c60f8fb 100644 --- a/debian/control +++ b/debian/control @@ -14,8 +14,9 @@ Build-Depends: docbook-xsl, flake8, isort, + vulture, xsltproc, -Standards-Version: 4.5.0 +Standards-Version: 4.6.2 Homepage: https://grml.org/grml2usb/ Vcs-git: git://git.grml.org/grml2usb.git Vcs-Browser: https://git.grml.org/?p=grml2usb.git diff --git a/debian/rules b/debian/rules index f83d809..7d696b5 100755 --- a/debian/rules +++ b/debian/rules @@ -10,7 +10,7 @@ VERSION:=$(shell dpkg-parsechangelog | awk '/Version: / { print $$2 }') override_dh_auto_build: dh_testdir - egrep -q '^PROG_VERSION = "\*\*\*UNKNOWN\*\*\*"' grml2usb || (echo "PROG_VERSION in grml2usb wrong." && exit 2) + grep -qE '^PROG_VERSION = "\*\*\*UNKNOWN\*\*\*"' grml2usb || (echo "PROG_VERSION in grml2usb wrong." && exit 2) $(MAKE) -C mbr $(MAKE) diff --git a/grml2iso b/grml2iso index 5f32804..3e48688 100755 --- a/grml2iso +++ b/grml2iso @@ -6,15 +6,15 @@ # License: This file is licensed under the GPL v2 or any later version. ################################################################################ +set -e -o pipefail + # make sure we have the sbin directories in our PATH to find grml2usb ootb PATH="${PATH}:/sbin:/usr/local/sbin:/usr/sbin" # adjust variables if necessary through environment {{{ # path to the grml2usb script you'd like to use - [ -n "$GRML2USB" ] || GRML2USB='grml2usb' -# work directory for creating the filesystem - [ -n "$TMPDIR" ] && WRKDIR="${TMPDIR}/grml2iso.tmp" - [ -n "$WRKDIR" ] || WRKDIR='/tmp/grml2iso.tmp' +[ -n "$GRML2USB" ] || GRML2USB='grml2usb' + # support mkisofs as well as genisoimage if which xorriso >/dev/null 2>&1 ; then MKISOFS='xorriso -as mkisofs' @@ -34,8 +34,6 @@ fi # }}} # helper stuff {{{ - set -e - usage() { echo >&2 "Usage: $0 [OPTIONS] -o target.iso source1.iso [source2.iso ...]" echo >&2 " @@ -54,10 +52,11 @@ Options: restrictions in the bootprocess only IPs are allowed. Supported protocols are: http and ftp -t Directory Directory that should be used for temporary files - during build. Defaults to /tmp/grml2iso.tmp if unset. + during build, instead of using a temporary directory + created by mktemp(1). Examples: - $0 -s http://192.168.23.42:8000/grml/ -o small.iso grml64-small_2018.12.iso + $0 -s http://192.168.23.42:8000/grml/ -o small.iso grml64-small_2021.07.iso Will generate a file small.iso which tries to download the squashfs file from http://192.168.23.42:8000/grml/ - the squashfs file is placed in the same @@ -95,10 +94,6 @@ Options: GRML2USB_OPTS+=(--bootoptions="fetch=$URI") fi - if [ -n "$WRKDIR" ] ; then - GRML2USB_OPTS+=(--tmpdir="$WRKDIR") - fi - # make sure -o is specified [ -n "$ISOFILE" ] || usage 1 @@ -143,8 +138,16 @@ Options: esac # }}} -# create necessary stuff under WRKDIR {{{ - [ -d "$WRKDIR" ] && WRKDIR_EXISTED='true' || WRKDIR_EXISTED='false' +# ensure to properly set up working directory {{{ + WRKDIR_EXISTED='false' + if [ -z "$WRKDIR" ] ; then + WRKDIR="$(mktemp -d)" + else + [ -d "$WRKDIR" ] && WRKDIR_EXISTED='true' + fi + + GRML2USB_OPTS+=(--tmpdir="$WRKDIR") + rm -rf "$WRKDIR/cddir" "$WRKDIR/grub_tmp" mkdir -p "$WRKDIR/cddir" # }}}} diff --git a/grml2usb b/grml2usb index 7afeb48..5ac6e76 100755 --- a/grml2usb +++ b/grml2usb @@ -64,9 +64,8 @@ RE_P_PARTITION = re.compile(r"(.*?\d+)p(\d+)$") RE_LOOP_DEVICE = re.compile(r"/dev/loop\d+$") -def syslinux_warning(option, opt, opt_parser): - """A helper function for printing a warning about deprecated option - """ +def syslinux_warning(option, opt, _value, opt_parser): + """A helper function for printing a warning about deprecated option""" # pylint: disable-msg=W0613 sys.stderr.write( "Note: the --syslinux option is deprecated as syslinux " @@ -76,9 +75,8 @@ def syslinux_warning(option, opt, opt_parser): # if grub option is set, unset syslinux option -def grub_option(option, opt, opt_parser): - """A helper function adjusting other option values - """ +def grub_option(option, opt, _value, opt_parser): + """A helper function adjusting other option values""" # pylint: disable-msg=W0613 setattr(opt_parser.values, option.dest, True) setattr(opt_parser.values, "syslinux", False) @@ -267,8 +265,7 @@ class VerifyException(Exception): def cleanup(): - """Cleanup function to make sure there aren't any mounted devices left behind. - """ + """Cleanup function to make sure there aren't any mounted devices left behind.""" def del_failed(_fn, path, _exc): logging.warning("Deletion of %s failed in temporary folder", path) @@ -403,8 +400,7 @@ def which(program): def get_defaults_file(iso_mount, flavour, name): - """get the default file for syslinux - """ + """get the default file for syslinux""" bootloader_dirs = ["/boot/isolinux/", "/boot/syslinux/"] for directory in bootloader_dirs: for name in name, "%s_%s" % (get_flavour_filename(flavour), name): @@ -520,7 +516,7 @@ def mkfs_fat16(device): logging.info("Formating partition with fat16 filesystem") logging.debug("mkfs.vfat -F 16 %s", device) - proc = subprocess.Popen(["mkfs.vfat", "-F", "16", device]) + proc = subprocess.Popen(["mkfs.vfat", "-n", "GRML", "-F", "16", device]) proc.wait() if proc.returncode != 0: raise CriticalException("error executing mkfs.vfat") @@ -810,7 +806,9 @@ def install_mbr(mbrtemplate, device, partition, ismirbsdmbr=True): set_rw(device) logging.debug( - "executing: dd if='%s' of='%s' bs=512 count=1 conv=notrunc", tmpf.name, device + "executing: dd if='%s' of='%s' bs=512 count=1 conv=notrunc,fsync", + tmpf.name, + device, ) proc = subprocess.Popen( [ @@ -819,7 +817,7 @@ def install_mbr(mbrtemplate, device, partition, ismirbsdmbr=True): "of=%s" % device, "bs=512", "count=1", - "conv=notrunc", + "conv=notrunc,fsync", ], stderr=open(os.devnull, "r+"), ) @@ -828,11 +826,6 @@ def install_mbr(mbrtemplate, device, partition, ismirbsdmbr=True): raise Exception("error executing dd (third run)") del tmpf - # make sure we sync filesystems before returning - logging.debug("executing: sync") - proc = subprocess.Popen(["sync"]) - proc.wait() - logging.debug("Probing device via 'blockdev --rereadpt %s'", device) proc = subprocess.Popen(["blockdev", "--rereadpt", device]) proc.wait() @@ -906,13 +899,21 @@ def unmount(target, unmount_options): unregister_mountpoint(target) +def extract_device_name(device): + """Extract the device name of a given path + + @device: device name, like /dev/sda1 or /dev/sda + """ + return re.match(r"/dev/(.*?)\d*$", device).group(1) + + def check_for_usbdevice(device): """Check whether the specified device is a removable USB device @device: device name, like /dev/sda1 or /dev/sda """ - usbdevice = re.match(r"/dev/(.*?)\d*$", device).group(1) + usbdevice = extract_device_name(device) # newer systems: usbdev = os.path.realpath("/sys/class/block/" + usbdevice + "/removable") if not os.path.isfile(usbdev): @@ -1322,8 +1323,7 @@ def get_device_from_partition(partition): def get_flavour(flavour_str): - """Returns the flavour of a grml version string - """ + """Returns the flavour of a grml version string""" return re.match(r"[\w-]*", flavour_str).group() @@ -1614,7 +1614,6 @@ def handle_syslinux_config(grml_flavour, target): new_grml_cfg = "%s/%s_grml.cfg" % (syslinux_target, flavour_filename) if os.path.isfile(defaults_file): - # remove default menu entry in menu remove_default_entry(new_default_with_path) @@ -1799,8 +1798,7 @@ def install_grml(mountpoint, device): def remove_mountpoint(mountpoint): - """remove a registered mountpoint - """ + """remove a registered mountpoint""" try: unmount(mountpoint, "") diff --git a/grml2usb.8.txt b/grml2usb.8.txt index 8e966ad..ec8a197 100644 --- a/grml2usb.8.txt +++ b/grml2usb.8.txt @@ -214,12 +214,6 @@ Directory layout on USB device | |-- grml64 | | |-- linux26 [Kernel] | | |-- initrd.gz [initramfs] - | |-- grml-medium - | | |-- linux26 [...] - | | |-- initrd.gz - | |-- grml64-medium - | | |-- linux26 - | | |-- initrd.gz | |-- grml-small | | |-- linux26 | | |-- initrd.gz @@ -256,11 +250,8 @@ Directory layout on USB device |-- grml/ | |-- filesystem.module [module specifying which squashfs should be used for grml] | `-- grml.squashfs [squashfs file for grml] - |-- grml-medium/ - | |-- filesystem.module [module specifying which squashfs should be used for grml-medium] - | `-- grml-medium.squashfs [squashfs file for grml-medium] |-- grml-small/ - | |-- filesystem.module [module specifying which squashfs should be used for grml-medium] + | |-- filesystem.module [module specifying which squashfs should be used for grml-small] | `-- grml-small.squashfs [squashfs file for grml-small] `-- ... diff --git a/pytest.ini b/pytest.ini new file mode 100644 index 0000000..6bba0a2 --- /dev/null +++ b/pytest.ini @@ -0,0 +1,3 @@ +[pytest] +markers = + check_for_usbdevice diff --git a/test/grml2usb.py b/test/grml2usb.py new file mode 120000 index 0000000..07ca4fc --- /dev/null +++ b/test/grml2usb.py @@ -0,0 +1 @@ +../grml2usb \ No newline at end of file diff --git a/test/grml2usb_test.py b/test/grml2usb_test.py new file mode 100644 index 0000000..56b2085 --- /dev/null +++ b/test/grml2usb_test.py @@ -0,0 +1,40 @@ +""" +grml2usb basic pytests +~~~~~~~~~~~~~~~~~~~~~~ + +This script contains basic "unit" tests, implemented for and executed with pytest. + +Requirements: +pytest (pip install pytest) + +Runwith: +$ pytest [-m {basic}] + +:copyright: (c) 2020 by Manuel Rom +:license: GPL v2 or any later version +:bugreports: http://grml.org/bugs/ +""" + + +import importlib + +import pytest + +grml2usb = importlib.import_module("grml2usb", ".") + + +@pytest.mark.check_for_usbdevice +def test_extract_device_name(): + """Assert, that 'extract_device_name' returns a device name for a given path""" + assert grml2usb.extract_device_name("/dev/sda") == "sda" + assert grml2usb.extract_device_name("/dev/sdb") == "sdb" + assert grml2usb.extract_device_name("/dev/sdb4") == "sdb" + + +@pytest.mark.check_for_usbdevice +def test_extract_device_name_invalid(): + """Assert, that 'extract_device_name' raises an Error, when given an incorrect string""" + with pytest.raises(AttributeError): + assert grml2usb.extract_device_name("/dev") + with pytest.raises(AttributeError): + assert grml2usb.extract_device_name("foobar")