Scientific Computing

On Windows with Python subprocess, an executable may be missing DLL from PATH. This environment error can be detected and a message given to the user.

The return code of the subprocess can be checked for the error code 3221225781, which corresponds to hex error code C0000135.

import subprocess
import os
import logging

ret = subprocess.run('myprog.exe')

if ret.returncode == 3221225781 and os.name == 'nt':
    # Windows 0xc0000135, missing DLL
    logging.error('missing DLL detected. Ensure DLL directory is in environment variable PATH')
    path = os.environ['PATH']
    print("PATH:", path)
    raise SystemExit(ret.returncode)

If putting DLLs on PATH is problematic, another possible approach is statically compiling the executable, but this may not be feasible for all programs.

Most shells (including Bash) don’t have a straightforward way to get the directory of the script being executed. Here’s a common one-liner to get the script directory:

sdir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

However, for some HPC batch systems, the above may not work as it copies this script to a temporary directory. This fallback put after the line above works if “qsub” batch command was run from this script’s directory–supposing a script file “gcc.sh” also exists in the same directory as the original script.

[[ ! -f ${sdir}/gcc.sh ]] && sdir=$PWD

PROJ.4 / GEOS-based CartoPy downloads and caches shape files as needed, avoiding a large install up front. CartoPy is an alternative to Matplotlib basemap. CartoPy uses easily available / automatically & seamlessly installed prereqs.

conda install cartopy

Note the zorder option of Matplotlib elements such as contourf. Higher number zorder is higher priority (on top). See PlotPrecip.py for an example of zorder. An example using CartoPy follows:

import cartopy
import cartopy.feature as cpf
from matplotlib.pyplot import figure, show
import numpy.random as npr


proj = cartopy.crs.PlateCarree()

fig = figure()
ax = fig.add_subplot(projection=proj)

ax.add_feature(cpf.LAND)  # type: ignore
ax.add_feature(cpf.OCEAN)  # type: ignore
ax.add_feature(cpf.COASTLINE)  # type: ignore
ax.add_feature(cpf.BORDERS, linestyle=':')  # type: ignore
# ax.add_feature(cpf.LAKES,   alpha=0.5)
# ax.add_feature(cpf.RIVERS)

N = 10

lat = (npr.random(N) - 0.5) * 180
lon = (npr.random(N) - 0.5) * 360

ax.scatter(lon, lat, transform=proj)

show()

The #type: ignore is to avoid mypy lint as CartoPy didn’t yet have type hinting support.

CartoPy can even make an auroral oval.

CMake environment variable CMAKE_BUILD_PARALLEL_LEVEL can be manually set to control default number of build parallel threads. Parallel builds are virtually always desired to save build and rebuild time. As a starting point, perhaps set CMAKE_BUILD_PARALLEL_LEVEL environment variable to be equal to the number of physical or logical CPU cores by setting it in the user profile:

#!/bin/bash

if [[ x"${CMAKE_BUILD_PARALLEL_LEVEL}" == x ]]; then
n=8;
case "$OSTYPE" in
linux*)
n=$(nproc);;
darwin*)
n=$(sysctl -n hw.physicalcpu);;
bsd*)
n=$(sysctl -n hw.ncpu);;
esac
export CMAKE_BUILD_PARALLEL_LEVEL=${n}
fi

Or for Windows, in environment variable settings:

CMAKE_BUILD_PARALLEL_LEVEL=%NUMBER_OF_PROCESSORS%

If the computer runs out of RAM, reduce the specific command parallelism with the cmake --build --parallel N command line option. For Ninja build systems, specific targets can control the number of workers with job pools.

The purpose of compiler flag checks is to test if a flag is supported. Metabuild system (such as CMake and Meson) compiler flag checks must not test the “-Wno-” form of a warning flag. This is because several compilers including Clang, GCC, Intel oneAPI emit a “success” return code 0 for the “-Wno-” form of an unsupported flag.

Incorrect

check_compiler_flag(C "-Wno-badflag" has_flag)

cc = meson.get_compiler('c')
has_flag = cc.has_argument('-Wno-badflag')

The incorrect CMake and Meson example scripts above will in general always set “has_flag = true” for the “-Wno-” form of a warning flag.

Correct way

check_compiler_flag(C "-Wbadflag" has_flag)

if(has_flag)
  target_compile_options(myexe PRIVATE -Wbadflag)
endif()

cc = meson.get_compiler('c')
has_flag = cc.has_argument('-Wbadflag')

if has_flag
  executable('myexe', 'myexe.c', c_args : '-Wbadflag')
endif()

From time to time, the topic of why meta-build systems like CMake and Meson are single-threaded sequentially executing processes is brought up. With desktop workstations (not to mention build farms) having 32, 64, and even 128+ CPU cores increasingly widespread, and the configure / generation step of meta-build systems taking tens of seconds to a few minutes on large projects, developers are understandably frustrated by the lack of parallelism.

A fundamental issue with CMake, Meson and equivalently for other meta-build systems is that the user’s CMake scripts would then have to be dataflow / declarative versus imperative. This would require reworking of script syntax and meta-build internal code radically.

In Meson, Python threading (single thread executes at one time) is used in subprojects, giving a speed boost in download time of subproject source code. There is no Python multiprocessing or ProcessPoolExecutor in Meson configure step. Meson parallel execution is for build (Ninja) and test. Both build and test are also done in parallel in CMake. For CMake, the ExternalProject steps can already be run in parallel (including download) via the underlying build system.

A way to speed up meta-build configure time–here specific to CMake–is to stuff the CMakeCache.txt file with precomputed values and/or use CMake Toolchain to do likewise, skipping configure tests when the host build system is static. CMakeCache.txt stuffing is a technique Meson uses to speed up configure time of CMake-based subprojects from Meson projects.

Flake8 is a Python code style checker that can detect unexecuted syntax errors. Python 3.12 with flake8 detects PEP8 formatting errors inside f-strings that are not yet handled by Black. Currently these formatting errors must be corrected by hand.

CI jobs should test with Python < 3.12 and Python >= 3.12 to ensure the f-string syntax is valid in older and newer Python versions.

Debugging C, C++, and Fortran code with GNU Debugger “gdb” is akin to debugging Python with pdb.

Start GDB Fortran debugger: assuming executable myprog with arguments hello and 3:

gdb --args ./myprog hello 3

Run program in gdb (perhaps after setting breakpoints) with

r

A typical debugging task uses breakpoints. Breakpoints are where the debugger stops running until you type

c

Set breakpoints by functionName:lineNumber. Example: Set a breakpoint in function myfun on line 32

b myfun:32

For breakpoints in Fortran modules in this example line 32 of a module named mymod in function myfun:

b mymod::myfun:32

List all scope variables as two separate steps.

1. local variables
2. arguments to the function.

Variable type, size (bytes/element), and shape (elements/dim) are available for each variable “var” by

whatis var

Example: a Fortran iso_fortran_env real64 3-D array of size 220 x 23 x 83:

var = REAL(8) (220,23,83)

If “var” is a Fortran derived type, get the same information about each record (akin to “property”) of the derived type by:

whatis var%prop

Local variables are variables used only within the scope of the function–excluding arguments to the function.

info locals

List the names and values of all arguments to the current function:

info args

Example: in integer function myfun(a,b) or subroutine mysub(a,b), upon info args you’d see perhaps

a = 1.5
b = 0.2

If a or b are arrays or structs, array values are printed as well.

CMake OS name flags like APPLE and UNIX are terse and are frequently used. A possible downside is their have broad, overlapping meanings.

In contrast, CMAKE_SYSTEM_NAME has more fine-grained values.

However, it is often more convenient (if using care) to use terse variables that are not as specific:

if(CMAKE_VERSION VERSION_LESS 3.25 AND CMAKE_SYSTEM_NAME STREQUAL "Linux")
  set(LINUX true)
endif()

if(APPLE)
  # macOS
elseif(BSD)
  # FreeBSD, NetBSD, OpenBSD, etc.
elseif(LINUX)
  # Linux
elseif(UNIX)
  # Linux, BSD, etc. (including macOS if not trapped above)
elseif(WIN32)
  # Windows
else()
  message(FATAL_ERROR "Unsupported system: ${CMAKE_SYSTEM_NAME}")
endif()

There is not one “right” or “wrong” way.

In North America and other areas of the world, license-free radio frequency bands used to communicate between vehicles include the 27 MHz CB band and 150 MHz MURS (or Canadian LADD). In the USA, 462 MHz GMRS requires a license to use an external antenna, and license-free FRS is limited to 2 watts with no external antenna allowed per 47 CFR 95.587 (b)1. MURS does allow an external antenna per 47 CFR 95.2741, but the maximum MURS transmit RF power is 2 watts per 47 CFR 95.2767.

In the USA, since the 1970s, CB channel 19 27.185 MHz AM has been a common ground for general communications and calling. For extended conversations, CB 19 users would switch to another channel to avoid tying up the calling channel. For FRS and MURS there generally aren’t enough users in a particular area to expect to get a random response while on the road.

For road convoy use, a radio communications priority is often to avoid annoyance from unwanted users and static while having adequate communication range. An example is charter buses, school field trips, sports groups, off-road vehicles, or other groups that travel together. In the USA the maximum power for FRS is 2 Watts with the antenna built into the radio. While FRS walkie talkies are available for about $15 each, they are not reliable car-to-car for more than about 1/4 mile due to interference and laying the walkie talkie down in the car. This is usually too short for convoy use, say if one person breaks down and the other person didn’t notice immediately, they may drive out of communication range before knowing it.

For CB radio, the maximum transmit power is 4 Watts. A mobile CB radio with FM and CTCSS (Radioddity CB-500) is about $94 currently. For CB mobile antenna, the Cobra HGA 1500, Cobra HGA 1000, K30, or Wilson Lil Wil antenna are each in the $35-$50 price range.

The limit of mobile-to-mobile communication range for CB, MURS, and FRS is primarily in the terrain and interference, not the 2 Watt vs. 4 Watt power. Pick a CB radio convoy channel that is not channel 6, 9, 11, 19, 26, or 28 to avoid interference from high-power users. For MURS, pick any free channel. For CB, MURS, and FRS use a CTCSS tone to avoid hearing other users on the same channel.

• CB radio: choose whether to communicate with truckers or those with inexpensive AM / FM radios.
• CB radio antenna length vs. range in open terrain and clear channel:
  • 28-36 inches length is about 3-5 miles.
  • 18-24 inches length is about 2-3 miles.
• FRS range is about a half-mile.
• MURS range is about 2-4 miles, but the radios are more expensive.
• GMRS range is about 5-7 miles, but the radios require a license.

Communication range reference