Scientific Computing

CTest resume testing

CTest runs can take an arbitrarily long time depending on the number and duration of tests configured in a CMake project. Whether one accidentally hit Ctrl+C in the Terminal window or intentionally stopped a CTest run with options like --stop-on-failure, it’s possible to resume from where CTest left off with ctest -F. If CTest ran all tests (whether or not there were failures of tests), then CTest will ignore “-F” and run all tests again or as specified by other CTest command line options.

Git rebase merge auto-accept changes

NOTE: using git rebase in general can wipe out a lot of work in difficult or impossible to recover ways. Before doing operations like this, create a copy of the Git branch and push it to recover from undesired outcomes.


Git rebase or merge operations on a feature branch with numerous commits can be fatiguing when it’s known that all of either the local or remote changes should be accepted in conflicts. This auto-accept of changes can be done on some or all changed files.

Git rebase auto-accept operations have the reverse sense of merge.

Rebase on “main”, auto-accept all local changes in conflicts:

git rebase -X theirs main

Rebase on “main”, auto-accept all remote changes in conflicts:

git rebase -X ours main

Git merge auto-accept operations have the reverse sense of rebase.

Merge “main”, auto-accept all local changes in conflicts:

git merge -X ours main

Merge “main”, auto-accept all remote changes in conflicts:

git merge -X theirs main

Reference

CMake FetchContent manual download

CMake FetchContent and ExternalProject bring remote projects into the top-level project to avoid making a monorepo or vendoring code into the top-level project. For those who desire more control over the remote project download process for FetchContent and ExternalProject, a method demonstrated in this script can be used:

CMake build files require CMake

CMake is a meta build system that generates build files such as Makefile, build.ninja etc. consumed by the build system such as GNU Make, Ninja, etc. corresponding to the CMake Generator selected. The input files for the build system generated by CMake themselves require CMake. They are not directory portable.

The most general approach uses CMake’s build command after configuration like:

cmake -B build
cmake --build build

An equivalent alternative is to directly invoke the build tool like:

cmake -B build -G "Unix Makefiles"
make -C build

or

cmake -B build -G Ninja
ninja -C build

Even when directory invoking the build tool, observe that editing the CMake script then invoking the build tool directory reconfigures CMake before commencing the build.

Example: Suppose

main.c:

int main(void) { return 0; }

CMakeLists.txt:

project(hello LANGUAGES C)

add_executable(main main.c)
  • cmake -G Ninja generates a 17 kB build.ninja.
  • cmake -G "Unix Makefiles" generates 15 kB of three distinct Makefiles.

Catch Numpy warnings as error

Numeric-focused libraries and programs may wise to “raise” and/or “catch” warnings generated by Numpy operations. By default for Numpy data types operations that raise with plain Python types may only warn with Numpy types.

Use numpy.errstate context manager or decorator to raise Numpy arithmetic warnings as errors. We do this INSTEAD of “numpy.seterr()” that changes the global Numpy settings. That is, scripts or packages that call this module would be affected by numpy.seterr() in surprising ways, while numpy.errstate as decorated or context manager limits the scope of the change to the function or context.

Make Numpy arithmetic warnings raise ArithmeticError as seen in this example.

Matlab / GNU Octave HTTP user agent

Some servers may block Matlab or GNU Octave download traffic from web operations like websave() or webread(). A web browser from the same computer may work–this is a symptom of server user agent blocking.

Get Matlab or GNU Octave HTTP user agent, with this script.

This script demonstrates setting a custom HTTP user agent using Matlab or GNU Octave factory function weboptions() to get around servers that block non-allow-listed user agents.


Python urllib.request can also set user agent.

CMake HTTP user agent

Programs using HTTP typically report a user agent to avoid being blocked. User agent metadata help the server know user client statistics.

CMake’s HTTP user agent is like

curl/<curl version>

as seen with CMake script.

Some servers may block CMake download traffic such as file(DOWNLOAD …). A web browser from the same computer may work–this is a symptom of server user agent blocking. This script demonstrates setting a custom HTTP user agent to get around servers that block non-allow-listed user agents.

PulseAudio on macOS

PulseAudio is available via Homebrew.

brew install pulseaudio

Start PulseAudio by:

brew services start pulseaudio

Check that PulseAudio is running by:

pactl list sinks

The proper “sink” may need to be selected to hear sound. Inspect the device list looking for say “Macbook Speakers” and set the default audio output device like:

pactl set-default-sink 1

if “Sink #1” is the Macbook Speakers and so on.


Related: macOS X11

Python pathlib iterdir subdirectories

Python pathlib.Path.iterdir only iterates over a single directory level. It does not recurse directories.

A common task is to iterate over each subdirectory of a top-level directory non-recursively. Given directory structure:

a/b
z
y/1/2

The Python iterdir example would return (in unspecified order):

a
z
y

Notice in the C++ and Python examples, iterators are used that emit one element at a time rather than building up an entire list at once. Since directory operations are often unordered, there is no advantage to retrieving every directory name in a greedy operation rather than the lazy generators shown, particularly if networked file systems are being used.

C11 Annex K safe functions

The C11 standard defines optional bounds-checking functions with an “_s” suffix in their names in Annex K. There are numerous reasons why these functions aren’t implemented in popular compilers / stdlib except MSVC. The most salient points are in the field experience note that observes that static analysis, dynamic analysis, address sanitization, etc. provide benefits that are largely beyond what the secure functions could provide, without the end user runtime penalty.

For totally new projects, one could consider coding languages that have inherently more secure memory access such as Rust. Or for a less dramatic change, using C++ for string-heavy portions of the project where the string class can be easier to use than C char.