Scientific Computing

Linux package managers like Apt and DNF can be set to not install recommended packages by default, which can be useful for headless embedded systems or WSL. This can save 100s of MB or more, which can be important for embedded systems or WSL to save time and disk space. Saving of download/install time is often relevant to CI/CD systems. A caveat is that some expected features may be missing.

Apt

From the command line (without changing default):

apt install --no-install-recommends <package>

Create a file like “/etc/apt/apt.conf.d/95-no-install-recommends” with the following contents:

apt::install-recommends "false";

To override this setting at the command line, to install recommended packages add “–install-recommends” to the apt command line.

DNF

From the command line (without changing default):

dnf install --setopt=install_weak_deps=False <package>

Edit “/etc/dnf/dnf.conf” adding under [main]:

install_weak_deps=False

Git submodules don’t put their individual Git config file under .git/config. This Git command issued from the Git submodule directory tells the file path to the Git submodule config file:

git rev-parse --git-path config

Say a repository “alpha” has submodule “beta”. Find the Git config file location of “beta” from “alpha” like:

git -C beta rev-parse --git-path config

# OR

cd beta
git rev-parse --git-path config

NEXRAD preview PNGs are 12200 x 5400 pixels: example This takes over 650 MB of RAM to load each frame.

ImageMagick -limit option can be used to limit the amount of RAM and/or disk used by ImageMagick operations. For example, if processing images on an embedded systems like a Raspberry Pi, the limited RAM may lead to disk swapping, taking 1000x longer or failing anyway. Use combinations of -limit options to control RAM and disk usage.

magick -limit map 500MB -limit memory 500MB -limit disk 100MB in.png -scale 10% out.png

The -debug cache option show how ImageMagick is using memory for a magick command.

On Windows, Miniconda can be installed from the Command Prompt or PowerShell:

winget install --id=Anaconda.Miniconda3 -e

Setup the new shell support (PowerShell, Bash, Command Prompt, etc.) with

conda init

Reopen Terminal to see new conda environment.

If message upon opening PowerShell like:

Documents\WindowsPowerShell\profile.ps1 cannot be loaded because running scripts is disabled on this system.

Consider ExecutionPolicy like:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned

By design, CTest WILL_FAIL test property does not handle segmentation faults and other unexpected terminations. That is, a failing test will also show as failed in CTest, when the desire may have been to show an intended failure as success via WILL_FAIL. A simple workaround for this is to make a wrapper call for the intentionally segfaulting executable and catch that result, as in example project.

Visual Studio Code can compare folders via Extension Compare Folders. In Unix-like shells, an alias can be used to ease command line use:

alias vsdiff="COMPARE_FOLDERS=DIFF code"

Then, to compare two folders, use:

vsdiff folder1 folder2

vsdiff allows applying changes from the left to the right. To “swap sides” click the left-right arrow in the top right corner.

Git submodules can switch remotes. This is useful when making a pull request for a project that relies on another pull request submodule. Verify the change in the top project’s “.gitmodules” file.

Example: suppose the directory “subdir” is a Git submodule. In this command, do not put “./subdir” or “subdir/”, just “subdir” by itself. Suppose you also wish to change the branch to “feat1” in the submodule.

git submodule set-url -- subdir https://github.invalid/username/new

git submodule sync

git -C subdir pull

git -C subdir switch feat1

It’s straightforward to detect if running inside WSL or if WSL is available.

These functions use only Python standard library modules. Other languages are demonstrated as well for checking WSL.

For certain use cases, it’s feasible to run a Bash script from within Windows using Windows Subsystem for Linux (WSL). Another way to run Bash scripts from within Windows itself without WSL is the Bash shell installed with Git on Windows.

Start the Bash script you want to run from Linux or Windows with the shebang (first line of Bash script file):

#!/bin/bash; C:/Program\ Files/Git/git-bash.exe

This tells the shell (Linux or Windows) to first try /bin/bash which is a Unix-like path, and then try the Git Bash shell on Windows. If Python is on Windows Path, one can use Bash scripts that invoke Python scripts.

CMake outputs default compiler flags based on platform and project configuration, which can be overridden. This example shows how to override the default compiler flags by putting the user flags later in the command line