Scientific Computing

Regardless of how an App is installed, whether through WinGet, the Microsoft Store, or a direct download, WinGet can be used to find the program install location on disk. For example, to find the install location of the Microsoft Edge browser:

winget list msedge --details

(one can also use the names Microsoft.Edge or Edge)

The results include “Installed Location” which for this example may be examined like:

ls ${Env:ProgramFiles(x86)}/microsoft/edge/application

with results including the Edge executable msedge.exe.

Matlab external language interfaces includes .NET on Windows, Linux, and macOS. This allows efficiently calling .NET assemblies and using .NET libraries directly from Matlab.

The Matlab function dotnetenv is used to set up and check the active .NET environment in Matlab. Environment variable DOTNET_ROOT is vital for Matlab to detect the .NET installation, particularly on Linux and macOS. If Matlab is having issues detecting the .NET installation NET.isNETSupported is false, determine the value for DOTNET_ROOT from system Terminal:

dotnet --info

If “dotnet” command is not found, install .NET SDK:

• macOS: brew install dotnet
• Windows: winget search Microsoft.DotNet.SDK
• Linux: from Microsoft
• Matlab Online (Linux): if .NET isn’t detected, use .NET install script to install .NET SDK.

On macOS, determine environment variable DOTNET_ROOT by $(brew --prefix dotnet)/libexec.

If this path is not defined, look for the Base Path, and pick the directory that contains the “dotnet” executable.

Set the DOTNET_ROOT environment variable in Matlab by adding to Matlab startup.m so that future Matlab sessions have the correct DOTNET_ROOT set.

edit(fullfile(userpath,'startup.m'))

Then add the following line to startup.m:

setenv('DOTNET_ROOT', '/opt/homebrew/opt/dotnet/libexec')

Where the path was obtained from $(brew --prefix dotnet)/libexec on macOS, or from the .NET install location on Linux.

Restart Matlab. Do the following one-time command to finish setting up .NET in Matlab:

dotnetenv("core", Version="10")

Where version number “10” must match the major version of .NET in DOTNET_ROOT as shown by dotnet --list-sdks in Terminal.

In future Matlab sessions, dotnetenv() will be empty until the first .NET command is run, for example

NET.isNETSupported

then (optionally) running dotnetenv() shows the current .NET environment.

Summary:

• stop (1956 Fortran I): return integer on stderr (recommendation)
• stop (Fortran 77): return integer or constant character, if character it may or may not be printed, but return code is 0 (no error)
• error stop (Fortran 2008): constant character with error code
• error stop (Fortran 2018): variable character with error code, also allowed inside pure procedure.

Fortran 2018 finally brought the needed behavior for convenient error messages and continuous integration.

CMake and Meson handle automatic detection of compiler supported features like error stop.

Fortran 2008 error stop with constant code:

• Gfortran ≥ 5
• NAG ≥ 6.0
• Intel oneAPI
• Nvidia HPC SDK

Fortran 2018 error stop with variable code

• Gfortran ≥ 7
• NAG ≥ 6.2
• Intel oneAPI
• Nvidia HPC SDK

Fortran 2018 error stop,QUIET=.true./.false.

• NAG ≥ 6.2

This feature was promoted by Steve Lionel, but has not yet been widely adopted. From the Fortran 2018 standard, the quiet= parameter not only suppresses any console output but also may suppress the error code? I would prefer to have the error return code, without the console text.

Since Fortran I in 1956, the stop statement has generally displayed a return code to indicate an error if an integer value was provided. Over time, stop statement behavior has changed to allow more refined signaling on program stop.

Since Fortran I in 1956, stop without return code to stop execution normally has been supported, along with stop with integer return code to indicate abnormal termination.

stop 1

The Fortran 2008 and 2018 standards recommend that the error code be returned on iso_fortran_env: error_unit, which was first defined in Fortran 2003 standard. The Fortran 77 standard defines the character string as “accessible” but doesn’t define where it goes. A best practice if desired to print a message when stopping a program is with explicit “print” or “write” statement. stop with integer code is still normal program termination in modern Fortran.

Since Fortran 77, stop may instead return a constant scalar character like “goodbye”. This generally sets return code to 0, that is, no error is indicated.

For continuous integration, having a reliable way to indicate error behavior is crucial. For HPC, indicating abnormal conditions to the shell is also vital to avoid taking resources on runs that suffered a computational error.

Fortran 2008 brought the long overdue error stop statement. stop still indicates normal program termination, and can for example stop individual images in a parallel executing program. Say an individual cell in a 3-D simulation did not find a stable solution. Depending on the simulation, that can be OK, perhaps set that value to NaN and stop with an error code on stderr, while letting the other images continue running.

However, in other types of simulations, an early failure to converge to a solution in a cell may invalidate the entire simulation taking a month of CPU time. Instead of writing cumbersome external checking code, the programmer can instead use error stop to reliably terminate all images when a critical failure is detected. Fortran 2008 error stop with constant string or integer code: both return non-zero exit status on stderr.

use, intrinsic:: iso_fortran_env, only: stderr=>error_unit

write(stderr,*) 'the failure was in ' // failedmod

error stop

Fortran 2018 added error stop with variable scalar string or variable integer code. A vital addition of Fortran 2018 is that error stop can be used within pure procedures, a very commonly needed use case. Fortran 2018 error stop variable character string allows for cleaner syntax, for example:

error stop 'the failure was in ' // failedmod

“conda” is a package manager commonly used for Python distributions that can manage both Python and non-Python packages. Conda channels are repositories of packages that conda can install from. Conda channels can provide CPU architecture-specific optimized packages. For systems with CPU emulations like Windows Prism, channels for distinct CPU architectures like “win-64” on a “win-arm64” system can allow using packages build for non-native architectures when native versions are unavailable. Conda prioritized channels can help resolve version conflicts, helping mitigate Python package dependency hell

Conda channel priority order is ordered by which channel appears first (highest) in .condarc. It’s generally recommended to add per-environment channels rather than modifying the global configuration to avoid corrupting multiple environments with incompatible packages. In general “strict” channel priority is recommended to mitigate compatibility problems.

conda config --set channel_priority strict

# check with
conda config --get channel_priority

Add a conda per-environment channel:

conda activate <env-name>

conda config --env --add channels <channel-name>

Get the current channel list for an environment:

conda activate <env-name>

conda config --env --show channels

Example of conda cross-architecture install

Windows ARM64 channels may not have older Python versions that are available on x86-64 channels. One might be able to install an older EOL (End of Life) Python version from the “win-64” channel on a “win-arm64” system.

conda search --subdir win-64 python

may show more and additional older Python versions than

conda search --subdir win-arm64 python

Suppose Python 3.9 is available on “win-64” but not on “win-arm64”. Use the “win-64” channel to install Python 3.9 on a “win-arm64” system:

conda create -n py39 --subdir win-64 python=3.9

This will be visible in the environment’s channel list:

conda env list

and as usual used by:

conda activate py39

Some patient medical image viewing systems (e.g. for X-ray, MRI, CT) only export JPEG. Other patient systems only allow uploading DICOM images, which contain metadata about the imaging study. If one needs to quickly convert JPEG images to DICOM, omitting the metadata, the following simple method can be used.

First, install DCMTK:

• macOS: brew install dcmtk
• Windows: Download and install from GitHub Releases *-win64.zip or use WSL
• Linux (or Windows Subsystem for Linux): apt install dcmtk

Convert a single JPEG file to DICOM using the img2dcm command:

img2dcm img1.jpg img1.dcm

This will create a DICOM file img1.dcm from the image data of img1.jpg, but without any metadata. The resulting DICOM file can be uploaded to the patient system that only accepts DICOM, and the image will be viewable, but without any metadata about the imaging study.

Convert multiple JPEG files in a folder with a simple loop in the Unix-like terminal:

for img in *.jpg; do
    img2dcm "$img" "${img%.jpg}.dcm"
done

Or in PowerShell:

Get-ChildItem -Filter *.jpg | ForEach-Object {
    $img = $_.FullName
    $dcm = [System.IO.Path]::ChangeExtension($img, ".dcm")
    img2dcm $img $dcm
}

If the input file isn’t JPEG, first convert it to JPEG using ImageMagick, which can be installed via:

• macOS: brew install imagemagick
• Windows: winget install --id=ImageMagick.ImageMagick
• Linux (or Windows Subsystem for Linux): apt install imagemagick

Then convert the image to JPEG:

magick img1.png img1.jpg

CMake file(ARCHIVE_EXTRACT), is more robust and easy to use than the command-line cmake -E tar syntax when within a CMakeLists.txt file. The option PATTERNS_EXCLUDE saves time and disk space by skipping the extraction of files that are not needed, such as documentation files.

cmake_minimum_required(VERSION 3.18)

if(CMAKE_VERSION VERSION_GREATER_EQUAL 4.5)
  set(_pat_excl_docs PATTERNS_EXCLUDE "docs/*")
endif()

file(ARCHIVE_EXTRACT
  INPUT ${archive}
  DESTINATION ${out_dir}
  ${_pat_excl_docs}
)

Zstd is an open file compression standard. Zstd has become widely used and is incorporated in the Linux kernel and GCC. We use Zstd for data archiving particularly for large files where size and speed are a concern. CMake supports Zstd compression throughout, including file(ARCHIVE_CREATE) and file(ARCHIVE_EXTRACT). Zstd is vendored into CMake, so there is no need to worry about system libraries for Zstd.

CMake can be a useful command line tool for Zstd compression and extraction, especially on platforms where Zstd is not available as a system library. For example, CMake can be used to extract .zst file archives like:

cmake -E tar x arc.zst

CMakeLists.txt Zstd example

• option file(ARCHIVE_CREATE ... WORKING_DIRECTORY ...) is necessary to avoid system-specific relative path issues.
• option file(ARCHIVE_CREATE ... PATTERNS_EXCLUDE ...) saves time and disk space by skipping archiving of files that are not needed, such as documentation files.

cmake_minimum_required(VERSION 3.31)

set(archive "example.zst")
set(in_dir "data/")

if(CMAKE_VERSION VERSION_GREATER_EQUAL 4.5)
  set(_pat_excl_docs PATTERNS_EXCLUDE "docs/*")
endif()

file(ARCHIVE_CREATE
  OUTPUT ${archive}
  PATHS ${in_dir}
  COMPRESSION Zstd
  COMPRESSION_LEVEL 3
  WORKING_DIRECTORY ${in_dir}
  ${_pat_excl_docs}
  )

COMPRESSION_LEVEL

arbitrary, bigger value is more compressed.

FORMAT

not used for Zstd.

If writing a custom Find*.cmake module, it’s important to avoid modifying the *_ROOT variable corresponding to the Find module’s package. For example, if writing FindMy.cmake, do not modify My_ROOT within FindMy.cmake as that value will be ignored. Instead of modifying the My_ROOT variable, set the output variable say my_root to the modified value. An example of when a My_ROOT variable might be modified is changing the file separator from \ to / on Windows with cmake_path(CONVERT My_ROOT TO_CMAKE_PATH_LIST my_root). Then, feed every “find_*()” command in FindMy.cmake a HINTS ${my_root} argument.

The ubiquitous GNU coreutils has long been missing from Windows. We found ourselves invoking coreutils utilities via WSL using wsl <coreutils command> to get access to these utilities on Windows until now. Microsoft has enhanced Rust-based uutils coreutils to run natively on Windows, and has made it available via WinGet:

winget install --id=Microsoft.Coreutils -e

Close / reopen the Terminal windows to use Coreutils, which has distinct conflict and availability of tools when using ComSpec Command Prompt vs PowerShell. Some commands are so POSIX-intrinsic that they are not available or relevant in Microsoft coreutils. Other coreutils commands overlap so much or conflict with Windows intrinsic commands that they are omitted from Microsoft coreutils. There are distinctions in command parsing of Microsoft coreutils vs. standard coreutils to be aware of.

A general issue across systems that use “coreutils”, say on embedded or other minimal systems where not all coreutils are available, is that it’s up to the developer to handle cases where some coreutils tools isn’t available or overloaded by something else. Build systems like CMake also handle these problems, like what exactly is “gcc” when multiple compilers masquerade as “gcc” – CMake inspects the version string to formally ID the compiler vendor. To use a script consuming coreutils on Windows, the script needs to handle issues like the following, where MSVC “link” is overriding coreutils “link”:

where.exe link

C:\Program Files\Microsoft Visual Studio\18\Community\VC\Tools\MSVC\*\bin\HostARM64\arm64\link.exe
C:\Program Files\coreutils\bin\link.exe

Have a look in Cargo.toml to see the Microsoft coreutils commands.

We have long augmented CMake projects with Bash and PowerShell scripts to handle tasks too awkward for CMake. Python usually has enough built-in capability in “os”, “pathlib”, and “shutil” to avoid needing coreutils in Python scripts.

GNU / Windows ?

Could Windows with Microsoft coreutils be considered a GNU / Windows hybrid - no, because Microsoft coreutils is based on uutils coreutils, which is an MIT-licensed reimplementation of GNU coreutils in Rust. GNU / Linux is a common term for Linux distributions that include GNU utilities, and Microsoft coreutils brings many of those utilities to Windows. While it’s not a full GNU environment, it does provide a significant portion of the GNU toolset on Windows, making it a sort of hybrid in terms of command-line utilities.

As background, the core components of a typical GNU/Linux system include:

• Linux Kernel: Core of the system, handling hardware and process management
• GNU Utilities: Essential tools for file management, text processing, and system administration
• Display Server and Desktop Environment: X11 or Wayland for graphics, with desktop environments like GNOME, KDE Plasma, or Xfce
• Package Manager: Software installation and updates (e.g., APT, DNF, Pacman).
• Shell: Command-line interface for interacting with the system.

On Windows the core components include:

• Windows Kernel: Core of the system, handling hardware and process management
• Microsoft coreutils: Essential tools for file management, text processing
• Display Server and Desktop Environment: Windows GUI for graphics and user interface
• Package Manager: WinGet for software installation and updates
• PowerShell: Command-line interface for interacting with the system

On macOS the core components include:

• XNU Kernel: Core of the system, handling hardware and process management
• BSD Utilities: Essential tools for file management, text processing, and system administration
• Display Server and Desktop Environment: Quartz for graphics, with the macOS desktop environment
• Package Manager: Homebrew for software installation and updates
• Shell: Terminal with Zsh for command-line interface

Python has become a dominant language for scientific computing, data analysis, machine learning, and engineering workflows. Julia offers a modern high-performance syntax specifically designed for numerical and scientific computing. GNU Octave is an open-source MATLAB alternative with largely MATLAB compatibile syntax.

GNU Octave continues to be developed by John W. Eaton. Octave is a high-level interpreted language designed for numerical computations. The community continues to release major versions roughly yearly.

Octave shines when you need:

• Near drop-in compatibility with MATLAB .m files (as long as proprietary toolboxes aren’t required).
• A quick way to test whether it’s worth porting a MATLAB function or script to Python.
• Calling MATLAB/Octave functions directly from Python using Oct2Py.

Octave includes its own growing set of packages (toolboxes) that extend its capabilities in areas like signal processing, control systems, and optimization.

Julia

Julia is a modern, high-performance language designed specifically for scientific and numerical computing. It aims to combine the ease of use of Python/MATLAB with the speed of C/Fortran.

Julia excels when:

• You need high performance without dropping to lower-level languages (JIT compilation often delivers near-C speeds for numerical loops and linear algebra).
• Working on large-scale simulations, differential equations, optimization, or other compute-intensive scientific tasks.
• You want a clean, math-friendly syntax with advanced features like multiple dispatch, metaprogramming, and excellent built-in support for parallelism and distributed computing.
• Reproducibility and package management are priorities (via its built-in package manager).

Julia has strong libraries for data science, machine learning, visualization, and more, though its overall ecosystem is smaller than Python’s. It’s particularly appealing for researchers writing performance-critical code from scratch.

Python

Key Advantages of Python:

• Vast ecosystem: NumPy, SciPy, Pandas, Matplotlib, scikit-learn, PyTorch/TensorFlow, and thousands of other specialized libraries cover everything from microcontrollers to supercomputers.
• Scalability: The same language and core libraries work from embedded devices → Raspberry Pi → laptops → HPC clusters.
• Reproducibility: Open-source nature means anyone can run your code with pip install or conda environments—no license server or version-matching headaches.
• Embedded / IoT support: Since 2014, MicroPython has brought a capable subset of Python (including exception handling, coroutines, etc.) to low-cost hardware like the Raspberry Pi Pico and many other MCUs/SoCs.

Python’s general-purpose nature also makes it easier to integrate with web apps, databases, GUIs, automation scripts, and version control workflows—areas where Octave is weaker.

Comparison Table

With Python and Oct2Py, Octave can be a bridge for those transitioning away from MATLAB. While Python is often a default choice for new projects, Julia can be a compelling alternative for high-performance numerical work.

Other Mathematical Software

These systems generally have smaller user bases than Python or MATLAB/Octave, largely due to historical momentum and narrower focus.

• SageMath — Open-source computer algebra system with excellent symbolic math capabilities.
• Scilab — Another free MATLAB-like environment.
• GDL — Open-source IDL work-alike, common in astronomy and geophysics.
• Mathematica / Maple — Proprietary tools with strong symbolic mathematics focus.