Scientific Computing

CTest WILL_FAIL segfault SIGABRT catch

By design, CTest WILL_FAIL test property does not handle segmentation faults and other unexpected terminations. That is, a failing test will also show as failed in CTest, when the desire may have been to show an intended failure as success via WILL_FAIL. A simple workaround for this is to make a wrapper call for the intentionally segfaulting executable and catch that result, as in example project.

Folder comparison with vscode

Visual Studio Code can compare folders via Extension Compare Folders. In Unix-like shells, an alias can be used to ease command line use:

alias vsdiff="COMPARE_FOLDERS=DIFF code"

Then, to compare two folders, use:

vsdiff folder1 folder2

vsdiff allows applying changes from the left to the right. To “swap sides” click the left-right arrow in the top right corner.

Git submodule change remote

Git submodules can switch remotes. This is useful when making a pull request for a project that relies on another pull request submodule. Verify the change in the top project’s “.gitmodules” file.

Example: suppose the directory “subdir” is a Git submodule. In this command, do not put “./subdir” or “subdir/”, just “subdir” by itself. Suppose you also wish to change the branch to “feat1” in the submodule.

git submodule set-url -- subdir https://github.invalid/username/new

git submodule sync

git -C subdir pull

git -C subdir switch feat1

Run Bash scripts from Windows

For certain use cases, it’s feasible to run a Bash script from within Windows using Windows Subsystem for Linux (WSL). Another way to run Bash scripts from within Windows itself without WSL is the Bash shell installed with Git on Windows.

Start the Bash script you want to run from Linux or Windows with the shebang (first line of Bash script file):

#!/bin/bash; C:/Program\ Files/Git/git-bash.exe

This tells the shell (Linux or Windows) to first try /bin/bash which is a Unix-like path, and then try the Git Bash shell on Windows. If Python is on Windows Path, one can use Bash scripts that invoke Python scripts.

CMake override default compiler flags

CMake outputs default compiler flags based on platform and project configuration, which can be overridden. This example shows how to override the default compiler flags by putting the user flags later in the command line

CMake FetchContent ignore build system

CMake FetchContent and ExternalProject bring remote projects into the top-level project to avoid making a monorepo or vendoring code into the top-level project. With FetchContent, the source code is retrieved at CMake configure time, allowing one to ignore the subproject build system and / or use only specific source files.

An example of this is using nRF5 SDK, which is a large project, but one may only wish to use a single source file and header as in this example:

Python collect images to HTML document

In many data analyses we may generate a large number of plots saved to disk. For convenience of sharing these plots, we have created a Python script that collects all images in a directory into a single HTML document that can be exported to PDF via the web browser “save as PDF” function.

HDF5 file version

When reading HDF5 files, the HDF5 file version bounds chosen by the writing program must be no newer than the reading program’s HDF5 library version. Usually this is not a problem unless using a very old program to read HDF5 files. When necessary to write older version HDF5 files, select the HDF5 file version from the writing program API. At the time of writing, there is no way to introspect the HDF5 library version required to read a specific HDF5 file. The only way to know is to try to read data from the HDF5 file and see if it works.

Python h5py: use libver= when necessary:

with h5py.File(libver='earliest') as f:
   ....

Matlab low-level HDF5 interface H5P.set_libver_bounds() set HDF5 file write library version bounds.

C / C++ / Fortran HDF5 library can set the HDF5 library version bounds by H5Fset_libver_bounds() and H5Pset_libver_bounds().


Related: Check HDF5 files for corruption