Scientific Computing

While CMake binaries can be downloaded for most platforms, there are certain cases where one wishes to build CMake from source. For example, when preparing a merge request to fix or enhance CMake. Usually the computer will have at least an older version of CMake that can be used. If so, we recommend using the existing CMake to build the newer CMake from the CMake source directory. We recommend using Ninja in general for faster build and rebuild for any CMake project.

cmake -B build -DBUILD_TESTING:BOOL=OFF -DCMAKE_BUILD_TYPE=Release --install-prefix=$HOME/cmake-dev -G Ninja

cmake --build build

cmake --install build

A one-step download-build script is the scripts/build_cmake.cmake.

If an old CMake isn’t available on the computer, then use CMake bootstrap:

./bootstrap --prefix=$HOME/cmake-dev --generator=Ninja -- -DBUILD_TESTING:BOOL=OFF -DCMAKE_BUILD_TYPE:STRING=Release

ninja -C Bootstrap.cmk/ install

This puts the compiled CMake under ~/cmake-dev, without disturbing the primary CMake install. Upon making any CMake code changes, simply recompile the minimum needed bits by:

ninja -C Bootstrap.cmk/ install

Long-running Terminal programs might not be detected by the operating system on a laptop or other device that has power-saving sleep or suspend modes. This can lead to disappointment as the long-running program hasn’t completed when the user checks back. This caffeinate Python script works on Windows, Linux, and macOS to keep the computer awake while a command line program is running, using features built into the operating system utilities.

Regardless of how an App is installed, whether through WinGet, the Microsoft Store, or a direct download, WinGet can be used to find the program install location on disk. For example, to find the install location of the Microsoft Edge browser:

winget list msedge --details

(one can also use the names Microsoft.Edge or Edge)

The results include “Installed Location” which for this example may be examined like:

ls ${Env:ProgramFiles(x86)}/microsoft/edge/application

with results including the Edge executable msedge.exe.

Matlab external language interfaces includes .NET on Windows, Linux, and macOS. This allows efficiently calling .NET assemblies and using .NET libraries directly from Matlab.

The Matlab function dotnetenv is used to set up and check the active .NET environment in Matlab. Environment variable DOTNET_ROOT is vital for Matlab to detect the .NET installation, particularly on Linux and macOS. If Matlab is having issues detecting the .NET installation NET.isNETSupported is false, determine the value for DOTNET_ROOT from system Terminal:

dotnet --info

If “dotnet” command is not found, install .NET SDK:

• macOS: brew install dotnet
• Windows: winget search Microsoft.DotNet.SDK
• Linux: from Microsoft
• Matlab Online (Linux): if .NET isn’t detected, use .NET install script to install .NET SDK.

On macOS, determine environment variable DOTNET_ROOT by $(brew --prefix dotnet)/libexec.

If this path is not defined, look for the Base Path, and pick the directory that contains the “dotnet” executable.

Set the DOTNET_ROOT environment variable in Matlab by adding to Matlab startup.m so that future Matlab sessions have the correct DOTNET_ROOT set.

edit(fullfile(userpath,'startup.m'))

Then add the following line to startup.m:

setenv('DOTNET_ROOT', '/opt/homebrew/opt/dotnet/libexec')

Where the path was obtained from $(brew --prefix dotnet)/libexec on macOS, or from the .NET install location on Linux.

Restart Matlab. Do the following one-time command to finish setting up .NET in Matlab:

dotnetenv("core", Version="10")

Where version number “10” must match the major version of .NET in DOTNET_ROOT as shown by dotnet --list-sdks in Terminal.

In future Matlab sessions, dotnetenv() will be empty until the first .NET command is run, for example

NET.isNETSupported

then (optionally) running dotnetenv() shows the current .NET environment.

Summary:

• stop (1956 Fortran I): return integer on stderr (recommendation)
• stop (Fortran 77): return integer or constant character, if character it may or may not be printed, but return code is 0 (no error)
• error stop (Fortran 2008): constant character with error code
• error stop (Fortran 2018): variable character with error code, also allowed inside pure procedure.

Fortran 2018 finally brought the needed behavior for convenient error messages and continuous integration.

CMake and Meson handle automatic detection of compiler supported features like error stop.

Fortran 2008 error stop with constant code:

• Gfortran ≥ 5
• NAG ≥ 6.0
• Intel oneAPI
• Nvidia HPC SDK

Fortran 2018 error stop with variable code

• Gfortran ≥ 7
• NAG ≥ 6.2
• Intel oneAPI
• Nvidia HPC SDK

Fortran 2018 error stop,QUIET=.true./.false.

• NAG ≥ 6.2

This feature was promoted by Steve Lionel, but has not yet been widely adopted. From the Fortran 2018 standard, the quiet= parameter not only suppresses any console output but also may suppress the error code? I would prefer to have the error return code, without the console text.

Since Fortran I in 1956, the stop statement has generally displayed a return code to indicate an error if an integer value was provided. Over time, stop statement behavior has changed to allow more refined signaling on program stop.

Since Fortran I in 1956, stop without return code to stop execution normally has been supported, along with stop with integer return code to indicate abnormal termination.

stop 1

The Fortran 2008 and 2018 standards recommend that the error code be returned on iso_fortran_env: error_unit, which was first defined in Fortran 2003 standard. The Fortran 77 standard defines the character string as “accessible” but doesn’t define where it goes. A best practice if desired to print a message when stopping a program is with explicit “print” or “write” statement. stop with integer code is still normal program termination in modern Fortran.

Since Fortran 77, stop may instead return a constant scalar character like “goodbye”. This generally sets return code to 0, that is, no error is indicated.

For continuous integration, having a reliable way to indicate error behavior is crucial. For HPC, indicating abnormal conditions to the shell is also vital to avoid taking resources on runs that suffered a computational error.

Fortran 2008 brought the long overdue error stop statement. stop still indicates normal program termination, and can for example stop individual images in a parallel executing program. Say an individual cell in a 3-D simulation did not find a stable solution. Depending on the simulation, that can be OK, perhaps set that value to NaN and stop with an error code on stderr, while letting the other images continue running.

However, in other types of simulations, an early failure to converge to a solution in a cell may invalidate the entire simulation taking a month of CPU time. Instead of writing cumbersome external checking code, the programmer can instead use error stop to reliably terminate all images when a critical failure is detected. Fortran 2008 error stop with constant string or integer code: both return non-zero exit status on stderr.

use, intrinsic:: iso_fortran_env, only: stderr=>error_unit

write(stderr,*) 'the failure was in ' // failedmod

error stop

Fortran 2018 added error stop with variable scalar string or variable integer code. A vital addition of Fortran 2018 is that error stop can be used within pure procedures, a very commonly needed use case. Fortran 2018 error stop variable character string allows for cleaner syntax, for example:

error stop 'the failure was in ' // failedmod

“conda” is a package manager commonly used for Python distributions that can manage both Python and non-Python packages. Conda channels are repositories of packages that conda can install from. Conda channels can provide CPU architecture-specific optimized packages. For systems with CPU emulations like Windows Prism, channels for distinct CPU architectures like “win-64” on a “win-arm64” system can allow using packages build for non-native architectures when native versions are unavailable. Conda prioritized channels can help resolve version conflicts, helping mitigate Python package dependency hell

Conda channel priority order is ordered by which channel appears first (highest) in .condarc. It’s generally recommended to add per-environment channels rather than modifying the global configuration to avoid corrupting multiple environments with incompatible packages. In general “strict” channel priority is recommended to mitigate compatibility problems.

conda config --set channel_priority strict

# check with
conda config --get channel_priority

Add a conda per-environment channel:

conda activate <env-name>

conda config --env --add channels <channel-name>

Get the current channel list for an environment:

conda activate <env-name>

conda config --env --show channels

Example of conda cross-architecture install

Windows ARM64 channels may not have older Python versions that are available on x86-64 channels. One might be able to install an older EOL (End of Life) Python version from the “win-64” channel on a “win-arm64” system.

conda search --subdir win-64 python

may show more and additional older Python versions than

conda search --subdir win-arm64 python

Suppose Python 3.9 is available on “win-64” but not on “win-arm64”. Use the “win-64” channel to install Python 3.9 on a “win-arm64” system:

conda create -n py39 --subdir win-64 python=3.9

This will be visible in the environment’s channel list:

conda env list

and as usual used by:

conda activate py39

Some patient medical image viewing systems (e.g. for X-ray, MRI, CT) only export JPEG. Other patient systems only allow uploading DICOM images, which contain metadata about the imaging study. If one needs to quickly convert JPEG images to DICOM, omitting the metadata, the following simple method can be used.

First, install DCMTK:

• macOS: brew install dcmtk
• Windows: Download and install from GitHub Releases *-win64.zip or use WSL
• Linux (or Windows Subsystem for Linux): apt install dcmtk

Convert a single JPEG file to DICOM using the img2dcm command:

img2dcm img1.jpg img1.dcm

This will create a DICOM file img1.dcm from the image data of img1.jpg, but without any metadata. The resulting DICOM file can be uploaded to the patient system that only accepts DICOM, and the image will be viewable, but without any metadata about the imaging study.

Convert multiple JPEG files in a folder with a simple loop in the Unix-like terminal:

for img in *.jpg; do
    img2dcm "$img" "${img%.jpg}.dcm"
done

Or in PowerShell:

Get-ChildItem -Filter *.jpg | ForEach-Object {
    $img = $_.FullName
    $dcm = [System.IO.Path]::ChangeExtension($img, ".dcm")
    img2dcm $img $dcm
}

If the input file isn’t JPEG, first convert it to JPEG using ImageMagick, which can be installed via:

• macOS: brew install imagemagick
• Windows: winget install --id=ImageMagick.ImageMagick
• Linux (or Windows Subsystem for Linux): apt install imagemagick

Then convert the image to JPEG:

magick img1.png img1.jpg

CMake file(ARCHIVE_EXTRACT), is more robust and easy to use than the command-line cmake -E tar syntax when within a CMakeLists.txt file. The option PATTERNS_EXCLUDE saves time and disk space by skipping the extraction of files that are not needed, such as documentation files.

cmake_minimum_required(VERSION 3.18)

if(CMAKE_VERSION VERSION_GREATER_EQUAL 4.5)
  set(_pat_excl_docs PATTERNS_EXCLUDE "docs/*")
endif()

file(ARCHIVE_EXTRACT
  INPUT ${archive}
  DESTINATION ${out_dir}
  ${_pat_excl_docs}
)

Zstd is an open file compression standard. Zstd has become widely used and is incorporated in the Linux kernel and GCC. We use Zstd for data archiving particularly for large files where size and speed are a concern. CMake supports Zstd compression throughout, including file(ARCHIVE_CREATE) and file(ARCHIVE_EXTRACT). Zstd is vendored into CMake, so there is no need to worry about system libraries for Zstd.

CMake can be a useful command line tool for Zstd compression and extraction, especially on platforms where Zstd is not available as a system library. For example, CMake can be used to extract .zst file archives like:

cmake -E tar x arc.zst

CMakeLists.txt Zstd example

• option file(ARCHIVE_CREATE ... WORKING_DIRECTORY ...) is necessary to avoid system-specific relative path issues.
• option file(ARCHIVE_CREATE ... PATTERNS_EXCLUDE ...) saves time and disk space by skipping archiving of files that are not needed, such as documentation files.

cmake_minimum_required(VERSION 3.31)

set(archive "example.zst")
set(in_dir "data/")

if(CMAKE_VERSION VERSION_GREATER_EQUAL 4.5)
  set(_pat_excl_docs PATTERNS_EXCLUDE "docs/*")
endif()

file(ARCHIVE_CREATE
  OUTPUT ${archive}
  PATHS ${in_dir}
  COMPRESSION Zstd
  COMPRESSION_LEVEL 3
  WORKING_DIRECTORY ${in_dir}
  ${_pat_excl_docs}
  )

COMPRESSION_LEVEL

arbitrary, bigger value is more compressed.

FORMAT

not used for Zstd.

If writing a custom Find*.cmake module, it’s important to avoid modifying the *_ROOT variable corresponding to the Find module’s package. For example, if writing FindMy.cmake, do not modify My_ROOT within FindMy.cmake as that value will be ignored. Instead of modifying the My_ROOT variable, set the output variable say my_root to the modified value. An example of when a My_ROOT variable might be modified is changing the file separator from \ to / on Windows with cmake_path(CONVERT My_ROOT TO_CMAKE_PATH_LIST my_root). Then, feed every “find_*()” command in FindMy.cmake a HINTS ${my_root} argument.