Scientific Computing

Matlab CI services include Azure Pipelines. Matlab Azure Pipelines CI automatically tests Matlab code on each “git push” as with other coding languages. The azure-pipelines.yml file in the Git repo controls CI behavior.

GitHub Actions Matlab CI is often easier to use than Azure in general and runs atop Azure.

This example Matlab azure-pipelines.yml will test a Matlab project on Azure Pipelines CI.

Matlab system() discards the shell environment variables. Set environment variables in name-value pairs like:

if ispc
  system("echo %test_var%", test_var="hello")
else
  system("echo $test_var", test_var="hello")
end

Multiple environment variables can be set in a single command by adding more name-value pairs.

To set specific environment variables desired to set in the system command, do like this example for “CC”:

system("make", CC=getenv("CC"))

To set multiple environment variables:

env = namedargs2cell(struct(CC="gcc", CXX="g++"));

system("make", env{:});

For more advanced Python-like subprocess control from Matlab, use matlab-stdlib subprocess_run() that allows setting environment variables, cwd, stdin pipe, timeout, and more.

Environment variables in CMake set outside of CMake before a CMake process is started have global scope in CMake. Setting environment variables in a CMake script don’t have visibility outside the current CMake process. That includes CMake ExternalProject, which will NOT see environment variables set(ENV{name}) in the parent CMake project. Also, build and test processes will NOT see environment variables set in the CMake script.

To make environment variables take effect in a CMake script or CMake configure, set the environment variable before running CMake, or on a Unix-like OS on the command line. For example, to tell CMake the HDF5 library location, set environment variable HDF5_ROOT before the first CMake configure command:

# Unix-like
HDF5_ROOT=~/mylib cmake -Bbuild

# Windows PowerShell
$env:HDF5_ROOT="~/mylib"
cmake -B build

Of course, one could also set the CMake variable at configure like:

cmake -Bbuild -DHDF5_ROOT=~/mylib

To make environment variables present during a CMake build, for example for an ExternalProject, do as above for the configure.

To control environment variables present for CTest tests, including for individual tests, set test properties.

For modern Fortran development, “.f90” is the recommended filename suffix. For legacy Fortran 77 code, “.f” is the recommended filename suffix.

To preprocess Fortran code files (e.g. with #ifdef statements), capitalize the filename suffix e.g. “.F90” Failing to capitalize the suffix of code that requires preprocessing can break build systems such as CMake and Meson. The introspection of Fortran code by CMake or Meson may make an incorrect build graph if the preprocessing is not done. This can show up as random (or deterministic) build failures, whether in serial or parallel build.

We use the first “free-form” syntax Fortran 90 standard “.f90” suffix to indicate the current Fortran standard. This helps avoid too many ambiguous file suffixes.

C and C++ likewise do not indicate their language standard year in the source code file suffix.

Matlab CI services include GitHub Actions. Matlab GitHub Actions CI is useful to automatically test Matlab code on each “git push” as with other coding languages. A top-level file buildfile.m is also needed.

This Matlab .github/workflows/ci.yml tests a Matlab project on GitHub Actions CI across operating systems and Matlab versions.

Common among build systems is the use of environment variables CC, CXX, FC to signal user intent to use a compiler, regardless of what appears first in environment variable PATH. A contradiction can arise for a CMake project using ExternalProject in that CMAKE_C_COMPILER et al are not automatically passed to the ExternalProject. Thus if environment variable “CC=gcc” but the top-level project user specified “cmake -DCMAKE_C_COMPILER=icx”, the top level CMake project would use icx IntelLLVM but the subproject would use GCC, which can cause unintended results.

The fix for this issue is to explicitly pass CMAKE_C_COMPILER et al to the ExternalProject CMAKE_ARGS if the subproject is also a CMake project.

set(args
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_Fortran_COMPILER=${CMAKE_Fortran_COMPILER}
)

ExternalProject_Add(...
CMAKE_ARGS ${args}
)

For Autotools ExternalProject do like:

set(args
CC=${CMAKE_C_COMPILER}
CXX=${CMAKE_CXX_COMPILER}
FC=${CMAKE_Fortran_COMPILER}
)

ExternalProject_Add(...
CONFIGURE_COMMAND <SOURCE_DIR>/configure ${args}
)

For GNU Make subproject do like:

set(args
CC=${CMAKE_C_COMPILER}
CXX=${CMAKE_CXX_COMPILER}
FC=${CMAKE_Fortran_COMPILER}
)

ExternalProject_Add(...
CONFIGURE_COMMAND ""
BUILD_COMMAND ${MAKE_EXECUTABLE} -j ${args}
)

CMake file(ARCHIVE_EXTRACT), is more robust and easy to use than the prior syntax.

if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.18)
  file(ARCHIVE_EXTRACT ${archive} ${out_dir})
else()
  # older, less robust
  file(MAKE_DIRECTORY ${out_dir})

  execute_process(
  COMMAND ${CMAKE_COMMAND} -E tar xf ${archive}
  WORKING_DIRECTORY ${out_dir}
  RESULT_VARIABLE ret
  )
  if(NOT ret EQUAL 0)
    message(FATAL_ERROR "extract ${archive} => ${out_dir}    ${ret}")
  endif()
endif()

CMake can print to “stderr” pipe cleanly like:

message(NOTICE "this message is on stderr pipe")

However, printing cleanly without “–” leading message requires a workaround:

execute_process(COMMAND ${CMAKE_COMMAND} -E echo "this message is on stdout pipe")

NOTE: Another technique that does NOT work in general for stdout is to print the invisible carriage return character. This only works visibly on the Terminal but if piping CMake stdout to another shell command does NOT work in general.

# don't do this, only works for printed shell, doesn't work for stdout pipe

string(ASCII 13 cr)
message(STATUS "${cr}  ${cr}No dashes visible, but stdout pipe is messed up")

Creating a unique temporary directory is not completely trivial. Despite the POSIX standard, some CIs (e.g. GitHub Actions) and Unix-like operating systems (e.g. RHEL) do not define the environment variables “TMPDIR” or “TEMP” for temporary directory.

Caveat: virus scanners can false detect activity in the system temporary directory as malicious.

Create a temporary directory in CMake:

Ancient Fortran code readability is impacted by the restrictions on variable length and line length that could lead to inscrutable variable and procedure names. The Fortran 2003 standard raised many of these limits to lengths that might only be a factor for auto-generated code with internally used very long names. If going beyond the usual name lengths, it’s a good idea to test across the compilers of interest (including compiler versions) to ensure that the required compiler vendors and versions can support the proposed name lengths.

We provide code examples verifying that compilers can support 63 character syntax elements (names for modules, submodules, variables), which is the maximum set by Fortran 2003 standard. The maximum line length is officially 132, but can be much longer depending on the compiler and compiler options.