grml2usb.git
27 hours agoRelease new version 0.18.4 master v0.18.4
Michael Prokop [Fri, 27 Nov 2020 16:56:40 +0000 (17:56 +0100)]
Release new version 0.18.4

27 hours agoBump Standards-Version to 4.5.1
Michael Prokop [Fri, 27 Nov 2020 16:55:56 +0000 (17:55 +0100)]
Bump Standards-Version to 4.5.1

10 days agoFix race condition with blockdev/BLKRRPART due to lack of fsync
Michael Prokop [Tue, 17 Nov 2020 22:45:23 +0000 (23:45 +0100)]
Fix race condition with blockdev/BLKRRPART due to lack of fsync

Quoting dann frazier from Debian's #975015:

| grml2usb autopkgtest frequently fail in Ubuntu's CI.
| [...]
| 2020-11-17 14:14:49,433 Probing device via 'blockdev --rereadpt /dev/loop0'
| blockdev: ioctl error on BLKRRPART: Device or resource busy
| 2020-11-17 14:14:49,452 Execution failed: ("Couldn't execute blockdev on '%s' (install util-linux?)", '/dev/loop0')
|
| I am able to reproduce this on an OpenStack instance with a failure rate of
| 33% (36 failures, 110 passes). My theory is that the sync is not sufficient,
| and that we really need to do a targeted fsync of the file. With the
| attached patch, I've yet to see a failure in 42 iterations.

Thanks: dann frazier <dannf@dannf.org> for bug report and patch
Closes: #975015

2 weeks agocodecheck: reformat with black, version 20.8b1
Michael Prokop [Fri, 13 Nov 2020 15:29:11 +0000 (16:29 +0100)]
codecheck: reformat with black, version 20.8b1

The formatting of grml2usb that was done with black, version 18.9b0
triggers changes with black 20.8b1-3 as present in current
Debian/unstable:

| black --check grml2usb
| would reformat grml2usb
| Oh no! 💥 💔 💥
| 1 file would be reformatted.
| make[2]: *** [Makefile:44: codecheck] Error 1

Applying the change via modern black though doesn't lead to any
changes with black, version 18.9b0, so let's use this.

Gbp-Dch: Ignore

8 weeks agoUse "GRML" as FAT label when creating the file system
Michael Prokop [Fri, 2 Oct 2020 13:41:13 +0000 (15:41 +0200)]
Use "GRML" as FAT label when creating the file system

Closes: grml/grml2usb#32

8 weeks agoAdd unit testing capabilities and basic tests for check_for_usbdevice
Manuel Rom [Fri, 11 Sep 2020 11:52:18 +0000 (13:52 +0200)]
Add unit testing capabilities and basic tests for check_for_usbdevice

* Add a test directory and create file
  'test/grml2usb_test.py'
* Provide basic test for extract_device_name()
* Extract regex match into separate function for
  better readability and testing options
* Provide pytest.ini configuration to be able to use
  custom markers
* Add symlink in test directory pointing to grml2usb
  for usage with importlib

3 months agoMerge remote-tracking branch 'origin/pr/42'
Michael Prokop [Wed, 12 Aug 2020 22:07:41 +0000 (00:07 +0200)]
Merge remote-tracking branch 'origin/pr/42'

4 months agoUse consistent case of 'USB' and 'Grml'
Darshaka Pathirana [Sun, 26 Jul 2020 17:07:33 +0000 (19:07 +0200)]
Use consistent case of 'USB' and 'Grml'

While at it also use the more common term 'USB flash drive' instead of 'USB pen'.

4 months agoCode cleanups
Michael Prokop [Thu, 25 Jun 2020 16:16:51 +0000 (18:16 +0200)]
Code cleanups

* Drop unused (*arg) usage from generate_main_syslinux_config()
* Drop global variable DATESTAMP, leftover from commit 100193b24,
  also remove corresponding `import datetime`
* Drop global variable GPT_HEADER, leftover from commit 705a96b84
* Drop unused `import time`
* Drop unused value variable from syslinux_warning() + grub_option()
* Drop unused function update_grml_versions(), leftover from commit 35feaad92af2e
* Drop unused array2string + string2array functions (introduced in commit 68df4a8c90),
  also remove corresponding `import struct`
* Drop unused is_writeable() function
* Drop unused generate_isolinux_splash() function
* Mark unused variables in search_file() as such
  (Vulture will ignore these variables if they start with an underscore)
* Mark unused function and excinfo parameters in del_failed() as unused
  The path and exception information is reported as unused by
  vulture, though shutil.rmtree needs to invoked with three parameters,
  quoting from https://docs.python.org/3/library/shutil.html:

  | If onerror is provided, it must be a callable that accepts three parameters: function, path, and excinfo.
* The 'warn' function is deprecated, using 'warning' instead

Thanks to vulture tool (https://pypi.org/project/vulture/).

Thanks: Chris Hofstaedtler for review + feedback

5 months agoRelease new version 0.18.3 v0.18.3
Michael Prokop [Wed, 24 Jun 2020 09:28:32 +0000 (11:28 +0200)]
Release new version 0.18.3

5 months agodocs: rework grml2iso docs, including note about order of ISOs
Michael Prokop [Wed, 24 Jun 2020 08:00:12 +0000 (10:00 +0200)]
docs: rework grml2iso docs, including note about order of ISOs

* let's point users towards https://github.com/grml/grml2usb/
* use most recent Grml version instead of ISO versions from 2009 :)
* add note regarding order of ISOs

5 months agodocs: drop no longer needed strong->bold workaround from asciidoc/xsltproc toolchain
Michael Prokop [Wed, 24 Jun 2020 07:54:53 +0000 (09:54 +0200)]
docs: drop no longer needed strong->bold workaround from asciidoc/xsltproc toolchain

asciidoc/xsltproc seem to no longer need this strong->bold change,
drop therefore.

5 months agogrml2iso: switch from isohybrid to xorriso/isohybrid to properly support EFI boot
Michael Prokop [Tue, 23 Jun 2020 15:17:59 +0000 (17:17 +0200)]
grml2iso: switch from isohybrid to xorriso/isohybrid to properly support EFI boot

This is the related change for grml-live's commit 6725bfa1ffc4, where we
introduced EFI support on 32bit systems and switched from isohybrid to
xorriso/isohybrid combination.

The isohybrid binary doesn't support 32-bit FI systems and fails hard
when using `--uefi` on a 32-bit ISO.  But xorriso with appropriate
options for EFI usage (see $BOOT_ARGS) and
/usr/lib/ISOLINUX/isohdpfx.bin from the isolinux package seems to
provide everything we need.

JFTR: To properly support 64bit EFI boot with our grml96 ISOs, we also
had to adjust the order of the provided ISOs in the grml2iso command
line in our release process.

Closes: grml/grml#149

5 months agoDo not fail if ISO doesn't have grub.cfg inside efi.img
Michael Prokop [Tue, 23 Jun 2020 13:48:08 +0000 (15:48 +0200)]
Do not fail if ISO doesn't have grub.cfg inside efi.img

If an ISO doesn't support Secure Boot then the grub.cfg file doesn't
exist inside the efi.img, so we shouldn't continue with its execution.
Use search_file() instead which is unset if the file doesn't exist and
therefore execution gets skipped.

Follow up fix for commit 92ffc08bb28f73c

5 months agoSupport Grml's new Secure Boot approach
Michael Prokop [Fri, 19 Jun 2020 13:13:36 +0000 (15:13 +0200)]
Support Grml's new Secure Boot approach

Secure Boot support was kind of broken and in grml-live commit 518eb395d
we reworked the layout and handling of the configuration.
The main change is the new GRUB prefix /boot/grub/grub.cfg instead
of /EFI/ubuntu. We need to adopt this accordingly, though it's probably
not worth being backwards compatible (given that we never released
official Grml ISOs with Secure Boot).

NOTE: the configuration file /boot/grub/grub.cfg *inside* the efi.img
doesn't get adjusted via handle_grub_config() yet, so if we should ever
add custom boot entries directly into this grub configuration file
(which is known as the grml-live template file
templates/secureboot/grub.cfg), we'd have to adjust handle_grub_config()
or invoke handle_grub_config() from inside handle_secure_boot().

Also we install the grub.cfg from inside EFI as /boot/grub/x86_64-efi/grub.cfg.
Looking at GRUB's default configuration file (see `cat
(memdisk)/grub.cfg`) shows that if /boot/grub/x86_64-efi/grub.cfg exists
it's getting sourced before /boot/grub/grub.cfg.  Since our *actual*
GRUB configuration of the Grml ISO is residing as /boot/grub/grub.cfg,
we can use /boot/grub/x86_64-efi/grub.cfg to control behavior in Secure
Boot mode.

Also ensure we take over file /conf/bootfile_*, which we
rely on from with grml-live's templates/secureboot/grub.cfg.

This work was funded by Grml-Forensic.

5 months agosetup.cfg: adjust flake8 section so invoking it without arguments works
Michael Prokop [Thu, 18 Jun 2020 15:56:49 +0000 (17:56 +0200)]
setup.cfg: adjust flake8 section so invoking it without arguments works

Instead of having to invoke `flake8 grml2usb` let's add the
files to the flake8 section in setup.cfg, so it's enough to
invoke just `flake8`.

Thanks: Florian Apolloner

5 months agocodecheck: fix flake8 issues with versions >=3.8.3
Michael Prokop [Thu, 18 Jun 2020 15:44:11 +0000 (17:44 +0200)]
codecheck: fix flake8 issues with versions >=3.8.3

Fixes:

| grml2usb:920:17: F523 '...'.format(...) has unused arguments at position(s): 0
| grml2usb:1954:57: E741 ambiguous variable name 'l'

Thanks: Florian Apolloner for feedback

5 months agoRelease new version 0.18.2 v0.18.2
Michael Prokop [Sat, 6 Jun 2020 13:09:51 +0000 (15:09 +0200)]
Release new version 0.18.2

5 months agoSkip boot flag check when installing to directory
Michael Prokop [Sat, 6 Jun 2020 10:26:15 +0000 (12:26 +0200)]
Skip boot flag check when installing to directory

In commit 8b59cb0b5c0cfa the check_boot_flag was moved from
install_grml() to main(), though inside install_grml() we
had the special casing whether the target is a directory or not.

To avoid having to add this check before any possible check_boot_flag()
invocation (and possibly forget about it), add this check to the
implementation of check_boot_flag().

Fixes:

| % sudo grml2usb --tmpdir=/tmp/grml2iso.tmp grml64-small_2020.06-rc1.iso grml32-small_2020.06-rc1.iso /tmp/grml2iso.tmp/cddir
| Executing grml2usb version 0.18.1
| Checking for boot flag
| Fatal: /tmp/grml2iso.tmp/cddir: unrecognised disk label

5 months agoRelease new version 0.18.1 v0.18.1
Michael Prokop [Wed, 3 Jun 2020 14:55:14 +0000 (16:55 +0200)]
Release new version 0.18.1

5 months agoMerge remote-tracking branch 'origin/pr/35'
Michael Prokop [Wed, 3 Jun 2020 13:11:37 +0000 (15:11 +0200)]
Merge remote-tracking branch 'origin/pr/35'

5 months agoMerge remote-tracking branch 'origin/pr/37'
Michael Prokop [Wed, 3 Jun 2020 13:05:05 +0000 (15:05 +0200)]
Merge remote-tracking branch 'origin/pr/37'

5 months agoCheck for boot flag before possibly creating FAT16 on the partition
Darshaka Pathirana [Wed, 3 Jun 2020 12:44:11 +0000 (14:44 +0200)]
Check for boot flag before possibly creating FAT16 on the partition

Checks should be performed before actually modifying the USB device,
if the user specified `--fat16` then the boot flag should be checked
*before* the FAT filesystem gets created.

check_for_fat(device) is called in handle_vfat(device) (if device is
present and not a directory and if option 'syslinux' (default) is set)
so it should be safe to remove check_for_fat(device) call in
install_grml().

The same applies to check_boot_flag() which should happen before
actually creating the FAT filesystem (in handle_vfat()).

Closes: grml/grml2usb#30

5 months agoReread partition table after installing MBR to ensure devices exist
Michael Prokop [Wed, 3 Jun 2020 11:45:54 +0000 (13:45 +0200)]
Reread partition table after installing MBR to ensure devices exist

We need to execute blockdev after installing the MBR, otherwise
the devices might not exist yet as expected and executing syslinux
can fail therefore.

While at it remove duplicate mbrtemplate python docstring in install_mbr

Closes: grml/grml2usb#33
Thanks: Darshaka Pathirana for debugging

5 months agoUse "boot flag" instead of "bootflag" also in docs + exceptions
Michael Prokop [Wed, 3 Jun 2020 11:36:04 +0000 (13:36 +0200)]
Use "boot flag" instead of "bootflag" also in docs + exceptions

Closes: grml/grml2usb#31

6 months agoMerge remote-tracking branch 'origin/pr/29'
Michael Prokop [Fri, 29 May 2020 16:08:19 +0000 (18:08 +0200)]
Merge remote-tracking branch 'origin/pr/29'

6 months agoDrop old bootflag detection + be more verbose about its execution
Michael Prokop [Thu, 28 May 2020 07:40:22 +0000 (09:40 +0200)]
Drop old bootflag detection + be more verbose about its execution

Our custom boot flag detection isn't reliable, so there's no
point in using it, instead let's rely on python3-parted only.

While at it, rename "bootflag" into "boot flag" in human readable
strings.

While at it, also be more verbose about the execution (esp. the
"sync" step takes a while and might look like something is
hanging).

Thanks: Darshaka Pathirana for bug report and debugging
Closes: grml/grml2usb#19

6 months agoRelease new version 0.18.0 v0.18.0
Michael Prokop [Tue, 12 May 2020 15:17:26 +0000 (17:17 +0200)]
Release new version 0.18.0

7 months agoGet rid of docbook-xsl workaround regarding duplicate empty lines
Michael Prokop [Thu, 2 Apr 2020 16:34:14 +0000 (18:34 +0200)]
Get rid of docbook-xsl workaround regarding duplicate empty lines

This doesn't seem to be no longer a problem

7 months agodebian: switch to debhelper minimal style
Michael Prokop [Thu, 2 Apr 2020 15:26:18 +0000 (17:26 +0200)]
debian: switch to debhelper minimal style

7 months agoautopkgtests: drop python2 support and depend on python3 only
Michael Prokop [Thu, 2 Apr 2020 15:13:33 +0000 (17:13 +0200)]
autopkgtests: drop python2 support and depend on python3 only

The smoke-grml2usb-py3 test was referring to python2.
python2 is no longer supported within Debian, let's drop
support for it.

Closes: #936663

7 months agodebian: execute `wrap-and-sort -a -t -s`
Michael Prokop [Thu, 2 Apr 2020 15:12:11 +0000 (17:12 +0200)]
debian: execute `wrap-and-sort -a -t -s`

Let's get this properly sorted...

7 months agoBump Standards-Version to 4.5.0
Michael Prokop [Thu, 2 Apr 2020 15:11:54 +0000 (17:11 +0200)]
Bump Standards-Version to 4.5.0

7 months agocodecheck: rely on flake8, isort + black during build time
Michael Prokop [Thu, 2 Apr 2020 14:45:12 +0000 (16:45 +0200)]
codecheck: rely on flake8, isort + black during build time

Switch from pyflakes, pylint + pep8 to flake8, isort + black,
and include it in the default build target to get checks during
build time. Add relevant packages to Build-Depends.

Thanks: Florian Apolloner for the feedback

7 months agoCoding style: execute black + isort and fix undefined 'path'
Michael Prokop [Thu, 2 Apr 2020 14:40:47 +0000 (16:40 +0200)]
Coding style: execute black + isort and fix undefined 'path'

Avoid discussions around pep8, pylint etc, but instead
use what black(1) (see https://github.com/psf/black) provides.

Fix:

| grml2usb:299:39: F821 undefined name 'path'

7 months agoCoding style: fix pep8 + flake8 issues
Michael Prokop [Thu, 2 Apr 2020 13:45:54 +0000 (15:45 +0200)]
Coding style: fix pep8 + flake8 issues

Fix:

| W605 invalid escape sequence '\s'

by using re.compile(r'...') instead of re.compile("...")

Fix:

| E261 at least two spaces before inline comment

introduced in commit ed5b633be961ef6b3

Fix flake8 issues:

| grml2usb:409:5: F841 local variable 'e' is assigned to but never used
| grml2usb:557:34: W504 line break after binary operator
| grml2usb:559:34: W504 line break after binary operator
| grml2usb:737:37: W504 line break after binary operator
| grml2usb:1615:13: F841 local variable 'error' is assigned to but never used
| grml2usb:1641:9: F841 local variable 'error' is assigned to but never used
| grml2usb:1665:5: F841 local variable 'error' is assigned to but never used
| grml2usb:1826:30: W504 line break after binary operator
| grml2usb:1832:30: W504 line break after binary operator

The https://www.flake8rules.com/rules/W503.html vs
https://www.flake8rules.com/rules/W504.html is strange,
though let's use what black(1) (see https://github.com/psf/black)
uses.

8 months agoMerge remote-tracking branch 'origin/pr/26'
Michael Prokop [Wed, 4 Mar 2020 13:38:39 +0000 (14:38 +0100)]
Merge remote-tracking branch 'origin/pr/26'

8 months agoMerge remote-tracking branch 'origin/pr/27'
Michael Prokop [Mon, 2 Mar 2020 07:50:37 +0000 (08:50 +0100)]
Merge remote-tracking branch 'origin/pr/27'

8 months agotarball.sh: spaces in pattern for PROG_VERSION
Alexei Colin [Sun, 1 Mar 2020 00:39:21 +0000 (19:39 -0500)]
tarball.sh: spaces in pattern for PROG_VERSION

9 months agoFix duplicate Syslinux entries (2)
Tomáš Virtus [Wed, 19 Feb 2020 14:14:30 +0000 (15:14 +0100)]
Fix duplicate Syslinux entries (2)

Commit f0e3c936 changed open mode to "r+", but this mode does not create
missing files.

Revert open mode back to "a+" and add seek to beginning of the file.

9 months agoAdd mbr/ output files to gitignore
Tomáš Virtus [Wed, 19 Feb 2020 13:15:07 +0000 (14:15 +0100)]
Add mbr/ output files to gitignore

9 months agoFix duplicate Syslinux entries
Tomáš Virtus [Mon, 3 Feb 2020 12:31:27 +0000 (13:31 +0100)]
Fix duplicate Syslinux entries

Documentation of open()[1] says:

  Other common values ... and 'a' for appending (which on some Unix
  systems, means that all writes append to the end of the file
  regardless of the current seek position).

On my system (Linux 5.4), writes to a file opened as 'a+' go at the end.

[1] https://docs.python.org/3/library/functions.html#open

9 months agoSupport more Syslinux module locations
Tomáš Virtus [Mon, 3 Feb 2020 11:44:35 +0000 (12:44 +0100)]
Support more Syslinux module locations

The current hardcoded directory of Syslinux modules,
/usr/lib/syslinux/modules/bios/, does not exist on ArchLinux. Since
modules are copied with copy_if_exist(), the installer doesn't notify
user when Syslinux modules are missing on the host. Furthermore, when
modules in Grml image are incompatible with Syslinux binary on the host,
the bootloader fails to load graphical menu as described in this bug:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=943878

  Undef symbol FAIL: x86_init_fpu
  Failed to load libcom32.c32
  Failed to load COM32 file vesamenu.c32
  boot:

This change makes grml2usb look for more possible locations on the host
and adds --syslinux-libs option to add another location.

9 months agoFix Python 3 syntax warning (is vs ==)
Tomáš Virtus [Mon, 3 Feb 2020 11:01:42 +0000 (12:01 +0100)]
Fix Python 3 syntax warning (is vs ==)

Fixes the following warning:

  ./grml2usb:1691: SyntaxWarning: "is" with a literal. Did you mean "=="?
    if mbrcode is "":

12 months agoRelease new version 0.17.0 v0.17.0
Michael Prokop [Thu, 31 Oct 2019 11:00:25 +0000 (12:00 +0100)]
Release new version 0.17.0

12 months agocheck_for_fat(): use subprocess.check_output to avoid dependency on Python 3.7
Michael Prokop [Thu, 31 Oct 2019 10:46:55 +0000 (11:46 +0100)]
check_for_fat(): use subprocess.check_output to avoid dependency on Python 3.7

The text=... option for subprocess.Popen was added in Python 3.7 only
("text was added as a more readable alias for universal_newlines"), and
also "encoding" and "errors" were introduced in Python 3.6 only.
We don't want to stick to a specific py3k version, so let's try to
be as backwards compatible as possible.

Since we only need stdout of blkid let's switch to
subprocess.check_output instead.

Thanks: Florian Apolloner for review and feedback

12 months agoNo longer explicitly list addon files, instead copy all files from /boot/addons/
Michael Prokop [Wed, 30 Oct 2019 14:34:39 +0000 (15:34 +0100)]
No longer explicitly list addon files, instead copy all files from /boot/addons/

It's annoying to have to manually adjust the list of expected addons
whenever we add/remove files (like adding EFI-capable addon files,
what I'm currently working on). Especially since the grml2usb version
as present in Debian/Grml repositories might not match the Grml ISO
the user wants to use.

12 months agoProvide git-describe based version information when running from within git
Michael Prokop [Wed, 30 Oct 2019 14:34:34 +0000 (15:34 +0100)]
Provide git-describe based version information when running from within git

This is useful when executing grml2usb from inside grml2usb.git itself

Thanks: Florian Apolloner for review and feedback

12 months agoMerge remote-tracking branch 'origin/pr/25'
Michael Prokop [Thu, 31 Oct 2019 09:45:17 +0000 (10:45 +0100)]
Merge remote-tracking branch 'origin/pr/25'

12 months agoFix path to syslinux *.c32 files
Sven Joachim [Thu, 31 Oct 2019 08:55:15 +0000 (09:55 +0100)]
Fix path to syslinux *.c32 files

In syslinux version 6.00 their path has changed, and so grml2usb
failed to find the files, using the ones from the grml iso instead.
The result was that the bootloader and its files had mismatched
versions which could lead to errors on boot, see
https://bugs.debian.org/943845.

Since syslinux 6.00 is already over six years old, I did not bother to
check for the *.c32 in the /usr/lib/syslinux/ directory.

12 months agoMerge remote-tracking branch 'origin/pr/24'
Michael Prokop [Wed, 30 Oct 2019 17:45:56 +0000 (18:45 +0100)]
Merge remote-tracking branch 'origin/pr/24'

12 months agocheck_boot_flag(): Open device in binary mode
Sven Joachim [Wed, 30 Oct 2019 16:35:03 +0000 (17:35 +0100)]
check_boot_flag(): Open device in binary mode

By default open() reads in text which is not going to fly for a boot
sector.

Fatal: 'utf-8' codec can't decode byte 0x90 in position 6: invalid start byte
Exception:
Traceback (most recent call last):
  File "/usr/sbin/grml2usb", line 1913, in main
    install(iso, device)
  File "/usr/sbin/grml2usb", line 1626, in install
    install_grml(iso_mountpoint, device)
  File "/usr/sbin/grml2usb", line 1653, in install_grml
    check_boot_flag(device)
  File "/usr/sbin/grml2usb", line 393, in check_boot_flag
    data = image.read(520)
  File "/usr/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 6: invalid start byte

12 months agocheck_for_fat: Avoid comparing strings and bytes
Sven Joachim [Wed, 30 Oct 2019 16:28:07 +0000 (17:28 +0100)]
check_for_fat: Avoid comparing strings and bytes

By default Subprocess.open() opens file objects in binary mode, so the
"filesystem" variable is an array of bytes, and comparing it to a
string always yields false.  Fix that by explicitly opening the stream
in text mode.

13 months agoRelease new version 0.16.7 v0.16.7
Michael Prokop [Tue, 22 Oct 2019 13:18:27 +0000 (15:18 +0200)]
Release new version 0.16.7

13 months agoDrop debian/compat and switch to debhelper-compat instead
Michael Prokop [Tue, 22 Oct 2019 13:16:07 +0000 (15:16 +0200)]
Drop debian/compat and switch to debhelper-compat instead

Fixes lintian warning package-uses-old-debhelper-compat-version

13 months agoBump Standards-Version to 4.4.1
Michael Prokop [Tue, 22 Oct 2019 13:13:24 +0000 (15:13 +0200)]
Bump Standards-Version to 4.4.1

13 months agoMerge remote-tracking branch 'origin/pr/23'
Michael Prokop [Tue, 15 Oct 2019 12:58:08 +0000 (14:58 +0200)]
Merge remote-tracking branch 'origin/pr/23'

13 months agoMerge remote-tracking branch 'origin/pr/22'
Michael Prokop [Tue, 15 Oct 2019 12:58:05 +0000 (14:58 +0200)]
Merge remote-tracking branch 'origin/pr/22'

13 months agoAvoid subprocess.Popen with stdout/stderr=PIPE and wait() usage
Michael Prokop [Tue, 15 Oct 2019 10:07:07 +0000 (12:07 +0200)]
Avoid subprocess.Popen with stdout/stderr=PIPE and wait() usage

Quoting from https://docs.python.org/2/library/subprocess.html#popen-objects:

| Popen.wait()
|   Wait for child process to terminate. Set and return returncode attribute.
|
|   Warning
|   This will deadlock when using stdout=PIPE and/or stderr=PIPE and the child
|   process generates enough output to a pipe such that it blocks waiting for the
|   OS pipe buffer to accept more data. Use communicate() to avoid that.

While modprobe isn't expected to ever output enough output to be
relevant, let's better be safe than sorry.

While at it fix typo (it's -> its)

Closes grml/grml2usb#21

13 months agoPort to py3k mika/py3k
Michael Prokop [Tue, 15 Oct 2019 08:28:47 +0000 (10:28 +0200)]
Port to py3k

Quoting from #936663:

Python2 becomes end-of-live upstream, and Debian aims to remove
Python2 from the distribution, as discussed in
https://lists.debian.org/debian-python/2019/07/msg00080.html

Closes: #936663

13 months agoMerge remote-tracking branch 'origin/pr/20'
Michael Prokop [Tue, 15 Oct 2019 08:24:47 +0000 (10:24 +0200)]
Merge remote-tracking branch 'origin/pr/20'

13 months agogrml2usb: add python3 test variant
Chris Hofstaedtler [Sat, 31 Aug 2019 13:17:22 +0000 (13:17 +0000)]
grml2usb: add python3 test variant

13 months agogrml2usb: Add smoke autopkgtest for python2
Chris Hofstaedtler [Sat, 31 Aug 2019 10:41:47 +0000 (10:41 +0000)]
grml2usb: Add smoke autopkgtest for python2

Adds an autopkgtest for the python2 variant of grml2usb -- currently
the only variant. The test unfortunately needs to be run inside a full
machine isolation context, e.g. autopkgtest-virt-qemu.

This builds a minimal stub ISO and does not attempt to test full
functionality.

13 months agogrml2usb: print backtrace when --verbose is given
Chris Hofstaedtler [Sat, 31 Aug 2019 13:11:30 +0000 (13:11 +0000)]
grml2usb: print backtrace when --verbose is given

Useful for debugging.

13 months agogrml2usb: abort if required logo.16 file is missing
Chris Hofstaedtler [Sat, 31 Aug 2019 13:10:52 +0000 (13:10 +0000)]
grml2usb: abort if required logo.16 file is missing

Instead of crashing with a type coercion error in popen (because
the logo file is None, then).

16 months agoRelease new version 0.16.6 v0.16.6
Michael Prokop [Thu, 11 Jul 2019 15:26:40 +0000 (17:26 +0200)]
Release new version 0.16.6

17 months agoAvoid custom boot options getting duplicated when used with multiple ISOs
Michael Prokop [Tue, 4 Jun 2019 08:39:28 +0000 (10:39 +0200)]
Avoid custom boot options getting duplicated when used with multiple ISOs

grml2usb can be invoked with multiple ISOs *and* custom boot options, like:

  % sudo grml2usb --bootoption="ssh=foobar mikawashere" grml32-small_2018.12.iso grml64-small_2018.12.iso /dev/sdb1

Then grml2usb appends the custom boot options in the GRUB
(boot/grub/*.cfg) + isolinux/syslinux (boot/syslinux/*.cfg)
configuration files of files that have been installed by the underlying
grml32-small ISO.

But when hitting the grml64-small ISO then, it continues to replace the
files with the boot options once again. Therefore we end up with
duplicate boot options in the grml32-small_2018.12.iso specific files
(boot/grub/grml32small_default.cfg, boot/grub/grml32small_options.cfg,
boot/syslinux/grml32_small_default.cfg, boot/syslinux/grml32_small_grml.cfg +
boot/syslinux/hidden.cfg).

We could actually ensure to only touch the ISO specific files, but
we don't have version information included in the file names and also
have shared files like boot/syslinux/hidden.cfg, so this might unexpected
side effects as well. So instead let's explicitly check for the provided
custom boot options.

Notes regarding the implementation:

* If boot options are whitespace-only or empty, nothing will happen. This
  prevents str.replace to do unintended things.

* otherwise consider string "PRE some-boot-options POST". Then we should find
  'some-boot-options' and remove them. Why the delimiter handling? Consider
  some-boot-options is 'ssh'. If we replace line.replace(bootopt, '') then some
  fictitious boot option like 'service.ssh=autostart' would be broken. Thus the
  space delimiters are necessary.

* However, the space delimiters might not exist if the boot options occur at the
  end of the string. Thus special handling is required for this case.

Thanks: Ralf Moll for the bug report and Lukas Prokop for providing the bug fix

20 months agoRelease new version 0.16.5 v0.16.5
Michael Prokop [Thu, 21 Mar 2019 11:24:14 +0000 (12:24 +0100)]
Release new version 0.16.5

20 months agoSwitch default mount point from /lib/live/mount/medium to /run/live/medium
Michael Prokop [Thu, 21 Mar 2019 09:14:48 +0000 (10:14 +0100)]
Switch default mount point from /lib/live/mount/medium to /run/live/medium

In commit 0d878d3a679 of live-boot(-grml)
("Simplify mount point handling by using /run/live instead of /lib/live/mount")
the mountpath of /lib/live/mount/medium was moved towards /run/live/medium.

Commit c6a17c7b41b of live-boot(-grml) provides a backward compatibility
rbind mount, but occasionally there seems to be a regression somewhere
during boot (see https://github.com/grml/live-boot-grml/issues/10), and
the rbind mount will be deprecated and removed before the bullseye
(Debian 11) release.

Layout changes over time:

* /cdrom for old linuxrc approach
* /live/image for initramfs layout until December 2012
* /lib/live/mount/medium for initramfs layout since December 2012
* /run/live/medium for initramfs layout since December 2018

21 months agoRelease new version 0.16.4 v0.16.4
Michael Prokop [Thu, 28 Feb 2019 15:02:58 +0000 (16:02 +0100)]
Release new version 0.16.4

21 months agogrml2iso: update ISO filename in usage example
Michael Prokop [Thu, 28 Feb 2019 15:00:45 +0000 (16:00 +0100)]
grml2iso: update ISO filename in usage example

21 months agoUpdate copyright year information
Michael Prokop [Thu, 28 Feb 2019 15:00:08 +0000 (16:00 +0100)]
Update copyright year information

21 months agodocs: update ISO filenames, referring to a more recent stable version
Michael Prokop [Thu, 28 Feb 2019 14:58:56 +0000 (15:58 +0100)]
docs: update ISO filenames, referring to a more recent stable version

21 months agoForce usage with python2
Michael Prokop [Thu, 28 Feb 2019 14:49:19 +0000 (15:49 +0100)]
Force usage with python2

grml2usb isn't py3k-ready yet, so until we're there
ensure we don't run under python3, even by accident.

Closes: #921327

21 months agoRelease new version 0.16.3 v0.16.3
Michael Prokop [Wed, 20 Feb 2019 15:35:25 +0000 (16:35 +0100)]
Release new version 0.16.3

21 months agoNo longer depend on coreutils or realpath
Michael Prokop [Wed, 20 Feb 2019 14:59:46 +0000 (15:59 +0100)]
No longer depend on coreutils or realpath

The transition of realpath to coreutils happened for the jessie release,
so it shouldn't be necessary any longer.

Closes: #877554
Thanks: Michael Stone for the bug report

21 months agoDepend on either syslinux or grub2-common + grub-pc-bin
Michael Prokop [Wed, 20 Feb 2019 14:52:21 +0000 (15:52 +0100)]
Depend on either syslinux or grub2-common + grub-pc-bin

grml2usb depends on grub-pc(-bin) files and grub2-common's grub-install,
but if someone has grub-efi-amd64 installed, then the grub-pc(-bin)
files are missing. The grub-efi-amd64 package shouldn't matter at all,
since we explicitly run grub with `--target=i386-pc`. Adjust depends
accordingly.

Closes: #796801
Thanks: Albert Dengg for bugreport and suggested solution + Axel Beckert for assistance in getting a proper dependency resolution

21 months agoBump Standards-Version to 4.3.0
Michael Prokop [Wed, 20 Feb 2019 14:52:14 +0000 (15:52 +0100)]
Bump Standards-Version to 4.3.0

2 years agoRelease new version 0.16.2 v0.16.2
Michael Prokop [Thu, 15 Nov 2018 09:03:38 +0000 (10:03 +0100)]
Release new version 0.16.2

2 years agoSwitch Homepage header from http to https
Michael Prokop [Thu, 15 Nov 2018 09:01:14 +0000 (10:01 +0100)]
Switch Homepage header from http to https

2 years agoBump Standards-Version to 4.2.1
Michael Prokop [Thu, 15 Nov 2018 09:01:06 +0000 (10:01 +0100)]
Bump Standards-Version to 4.2.1

2 years agoMake recent versions of pep8 happy
Michael Prokop [Fri, 1 Jun 2018 14:08:57 +0000 (16:08 +0200)]
Make recent versions of pep8 happy

Fixes:

| grml2usb:71:1: E305 expected 2 blank lines after class or function definition, found 1

as reported by pep8 v2.2.0 (though not yet with v1.6.2)
in our source-test job on Jenkins.

2 years agoMerge remote-tracking branch 'origin/pr/16'
Michael Prokop [Fri, 25 May 2018 18:39:40 +0000 (20:39 +0200)]
Merge remote-tracking branch 'origin/pr/16'

2 years agoRelease new version 0.16.1 v0.16.1
Michael Prokop [Fri, 25 May 2018 11:55:29 +0000 (13:55 +0200)]
Release new version 0.16.1

2 years agoBump Standards-Version to 4.1.4
Michael Prokop [Fri, 25 May 2018 11:54:42 +0000 (13:54 +0200)]
Bump Standards-Version to 4.1.4

3 years agoFix a bunch of typos
Michael Prokop [Sat, 23 Sep 2017 11:50:30 +0000 (13:50 +0200)]
Fix a bunch of typos

s/prefered/preferred/
s/Ressources/Resources/
s/retrive/retrieve/
s/intial/initial/
s/compability/compatibility/
s/ressources/resources/
s/Ressources/Resources/
s/been been/been/

3 years agoRelease new version 0.16.0 v0.16.0
Michael Prokop [Thu, 7 Sep 2017 07:41:51 +0000 (09:41 +0200)]
Release new version 0.16.0

3 years agoProvide Secure Boot support
Michael Prokop [Thu, 7 Sep 2017 07:38:14 +0000 (09:38 +0200)]
Provide Secure Boot support

This is pretty nice since we can support Secure Boot
straight without any further/extra command line options.
If the ISO doesn't ship the files we expect to be there
then we don't need any special handling either, so we're
fully backwards and forwards compatible so far.

Note: only supported with Grml ISOs generated with grml-live >=0.31.0

3 years agoRelease new version 0.15.4 v0.15.4
Michael Prokop [Thu, 17 Aug 2017 12:51:00 +0000 (14:51 +0200)]
Release new version 0.15.4

3 years agoBump Standards-Version to 4.0.1
Michael Prokop [Thu, 17 Aug 2017 12:49:57 +0000 (14:49 +0200)]
Bump Standards-Version to 4.0.1

3 years agoPython3: Some improvements to increase support
joeran [Tue, 18 Jul 2017 19:27:10 +0000 (21:27 +0200)]
Python3: Some improvements to increase support

Hopefully python2 is still supported...

3 years agoRelease new version 0.15.3 v0.15.3
Michael Prokop [Wed, 31 May 2017 09:28:59 +0000 (11:28 +0200)]
Release new version 0.15.3

3 years agoEnsure that grub-install doesn't receive emtpy command line argument
Michael Prokop [Wed, 31 May 2017 09:20:26 +0000 (11:20 +0200)]
Ensure that grub-install doesn't receive emtpy command line argument

We run grub-install once without --force and if that fails
try rerunning with --force. On the first run we pass an
empty argument though that grub-install assumes as install
device and fails with:

  /usr/sbin/grub-install: error: More than one install device?.

By using "--" as option to signify the end of command line
options we avoid this failure in the run without passing the
--force option.

3 years agoOption --rw-blockdev: set r/w also in GRUB specific steps
Michael Prokop [Wed, 31 May 2017 09:19:49 +0000 (11:19 +0200)]
Option --rw-blockdev: set r/w also in GRUB specific steps

Thanks: Ralf Moll

3 years agoRelease new version 0.15.2 v0.15.2
Michael Prokop [Fri, 26 May 2017 21:34:46 +0000 (23:34 +0200)]
Release new version 0.15.2

3 years agoSupport netboot.xyz.lkrn addon
Michael Prokop [Fri, 26 May 2017 21:32:17 +0000 (23:32 +0200)]
Support netboot.xyz.lkrn addon

For the grml96 flavours we use grml2iso, which itself
relies on grml2usb for copying the relevant files.
With the addition of the netboot.xyz.lkrn file we
forgot

Actually it would be nice to copy any present addon
from /boot/addons/, but close to the new stable Grml
release let's keep the change as small as possible.

Thanks: Michael Schierl for reporting the bug
Closes grml/grml#25

3 years agoRelease new version 0.15.1 v0.15.1
Michael Prokop [Wed, 24 May 2017 13:48:04 +0000 (15:48 +0200)]
Release new version 0.15.1

3 years agoOption --rw-blockdev: ensure to set device to rw between grub-install runs + don...
Michael Prokop [Wed, 24 May 2017 13:31:21 +0000 (15:31 +0200)]
Option --rw-blockdev: ensure to set device to rw between grub-install runs + don't set rw on source

If we're invoking set_rw on the source device this probably
points to the source ISO which will fail.

On the other side we invoke grub-intall with and without --force
in a loop and have to unblock the underlying block device in
between as well, otherwise failing to install.

3 years agoDon't fail hard if installing on partition number >4, instead warn user [Closes:...
Michael Prokop [Fri, 19 May 2017 14:49:54 +0000 (16:49 +0200)]
Don't fail hard if installing on partition number >4, instead warn user [Closes: issue1353]

3 years agoRelease new version 0.15.0 v0.15.0
Michael Prokop [Thu, 13 Apr 2017 21:57:37 +0000 (23:57 +0200)]
Release new version 0.15.0