Scientific Computing

For those dual-booting Windows and Linux, it’s useful to disable Windows Fast startup so that you can access the Windows partition from Linux. Disabling Fast Boot may add a few seconds to boot time. When you shutdown your PC, Fast Boot actually reboots and then hibernates. This locks the Windows partition so that Linux can’t access it.

Disable Fast Boot: ⊞+r powercfg.cpl → “Choose what the power button does”. We typically disable hibernate as well to save on SSD wear.

BitLocker and dual-booting: If BitLocker is enabled, Linux cannot access the Windows partition.

Fast Boot SSD wear: Fast Boot just like hibernate increases solid state drive (SSD) wear, since you are dumping an image ~ RAM size to disk on each Fast Boot shutdown.

Python pathlib is part of Python standard library. When using Python subprocess, in general, do str(mypath) where type(mypath) is pathlib.Path.

Example:

#!/usr/bin/env python
from pathlib import Path
import subprocess

P = Path('~').expanduser()

O = subprocess.check_output(['dir', str(P)])

print(O)

if you don’t cast pathlib.Path to str:

TypeError: argument of type ‘WindowsPath’ is not iterable.

Notes

You should rarely if ever need shell=True. On Windows in particular, give the full path to a non-system executable.

Related: Idiomatic pathlib Python

Find and replace text in files recursively on Linux, macOS and Windows Subsystem for Linux with recursive_sed.sh:

#!/usr/bin/env bash
#
# The quoted variables allow using backticks and other "nasty" characters in sed.
#
# usage:
# ./recursive_sed.sh  pathtofiles yuck yay
#
# References:
# https://stackoverflow.com/a/1585810
# https://unix.stackexchange.com/a/128758

set -o errexit
set -o nounset

path=$1
old=$2
new=$3

echo "${old} => ${new}"

find $path -not -path '*/.git*' -type f -execdir sed -i 's/'"${old}"'/'"${new}"'/g' {} +

Pdoc is vastly simpler to use than Sphinx for small projects, with good HTML formatting.

In distinction from Sphinx, pdoc does not parse RST docstrings, but Pdoc does allow the following docstring formats:

• Markdown
• numpydoc
• Google-style docstrings

At this time, pdoc3 doesn’t consider Python type hinting.

Install

pdoc is a light minimal install.

pip install pdoc3

Usage

We use the pymap3d project with auto-generated docs as an example.

From the top-level pymap3d/ directory, type

pdoc --html pymap3d

This creates several *.html files under html/pymap3d, including index.html.

Check the output by:

firefox html/pymap3d/index.html

If the output seems satisfactory, move the files to the gh-pages branch, or at least to a docs/ folder:

mv html/pymap3d/*.html docs/

Publish the docs to the web by:

1. In the Github repo Settings → Github Pages → Source: “gh-pages”
2. git commit and push. It takes a few seconds for Github Pages to deploy HTML.

Python users typically use a non-system Python distribution such as Miniconda. Python distributions typically allow easy switching of Python version. Where Linux system Python must be used, the default Python version can be switched persistently with update-alternatives.

Update-alternatives without sudo: a one-time setup.

Configure shell to use ~/.local/bin instead of system-wide /usr as follows:

mkdir ~/.local/bin

Add to ~/.profile:

export PATH="$HOME/.local/bin:$PATH"

Enable switching Python default between Python versions with these one-time commands:

update-alternatives --install $HOME/.local/bin/python python /usr/bin/python3 20

After the setup above, at any time select the default system Python version without sudo:

update-alternatives --config python

GNU Radio is trivially easy to install on Ubuntu, which includes the UHD driver.

apt install gnuradio

DO NOT use GNU Radio from a virtual machine in general with an SDR, as typically many packets will be lost and the data will not be useful. The typical best choice is to use a GNU Radio live USB image. Frequent GNU Radio users often dual boot into Ubuntu for working with SDR.

Build GNU Radio from source

Optionally compile GNU radio yourself

The HP-Setup program sets up HP printers on Linux and is easy to install and use.

Install HP setup program:

apt install hplip

Get the IP address of your networked HP printer from its front panel or the network address e.g. myprinter.local The CLI method is very quick and reliable.

CLI

Add the HP printer to the PC by:

hp-setup -i PrinterAddress

where PrinterAddress is the IP address e.g. 10.0.0.64 or a network name like myprinter.local

HP setup will try to select a PPD file, usually it finds a good one. Perhaps name the printer by the room name or department – it’s arbitrary.

After setup, the printer will show in:

• system “Printers” window
• CUPS under https://localhost:631/printers/ → Manage Printers

GUI

As noted above, this method may not work until HP updates their program to use a current Qt version.

Start GUI by typing hp-setup then select

1. Network
2. Manual Discovery
3. type in printer IP address

The Printer should be instantly discovered and you’re ready to print.

BIOS update via FreeDOS works across many PCs, including Dell and Gigabyte. The key is that the OEM provides BIOS as a DOS executable.

Setup DOS-bootable USB drive: download the DOS-flashable BIOS firmware from the OEM e.g. support.dell.com Determine USB drive device (e.g. /dev/mmcblk0) by issuing this command before and after USB drive insertion:

lsblk

Format (permanently erase) this USB drive with

umount /dev/mmcblk0

mkdosfs -F 32 /dev/mmcblk0

Download the FreeDOS “USB Full Image” and extract the FD*.img file. Write this image to the USB stick:

dd if=FD*.img of=/dev/mmcblk0 bs=4M

the image write will take a minute or two, depending on the write speed of the USB stick. To be sure it’s done, wait for this command to exit in a second Terminal window:

sync

Update BIOS: extract/copy ALL the BIOS self-extracted files to the root of the FreeDOS flash drive from Linux. Insert the USB flash drive into the PC to be flashed Upon powerup, choose to boot from the USB drive. This might require pressing F12 key. In the BIOS boot device menu, look under Legacy boot. If Legacy boot is not enabled, it may need to be enabled to see the FreeDOS USB drive. Do NOT install FreeDOS, just boot to DOS when prompted by FreeDOS. In FreeDOS prompt, type the name of the .exe or .bat file for the firmware update as per OEM instructions. The flashing process may take several minutes.

If the PC says

operating system not found

on this DOS USB drive, try it in another PC. If it still doesn’t work, try another model of USB flash drive. Some flash drives just are not amenable to being booted from.

If a new computer motherboard seems to be dead out of the box, be sure the power supply is working and connected correctly. It may have a CPU that’s too new for the motherboard BIOS.

Specific example: This happened for Gigabyte GA-Q87M-D2H with version F6 BIOS. Version F7 BIOS was necessary for Haswell Refresh CPUs. I had to put an “old” CPU in the motherboard to flash the motherboard BIOS!

Bash scripts by default ignore aliases, unless the command

shopt -s expand_aliases

has been used before the aliased command. This is typically a good thing, as if one has set in ~/.bash_aliases something like

alias mv="mv -v"

any script using mv could produce extremely lengthy and verbose output when installing a program for example.

However, sometimes a user has multiple versions of a program installed in directories in $PATH and for whatever reason cannot use update-alternatives to make the desired one the default. Use update-alternatives instead whenever possible as this method described here is not as robust. Because we are not sourceing the Bash script, the alias scope is only within the Bash script itself. That is, once the script is done, the alias disappears.

If a script needs CMake 3.x and update-alternatives is not available, do within the Bash script:

shopt -s expand_aliases
alias cmake=/usr/bin/cmake3

Again, we stress this is not general or robust, so only do this as a last resort. sudo is not required for update-alternatives. Put the softlinks under $HOME/.local/bin and put that on your $PATH as a much better choice.

Pytest 4.1 removed a deprecated API, resulting in errors like:

AttributeError: ‘Function’ object has no attribute ‘get_marker’

The fix requires update of affiliated pytest-* packages to at least:

• pytest-cov 2.6.1
• pytest-remotedata 0.3.1

At this time, conda doesn’t handle this automatically, so if you are a conda user:

conda install pytest-cov pytest-remotedata

Similarly for other affected packages–look at the last few lines of the error to get a hint as to what package is affected.