Scientific Computing

This small Python-based Markdown link-checking script is effective for large (thousands of pages, tens of thousands of links) Markdown-based websites/ It is immensely faster than the legacy HTML LinkChecker program of the next section. Alternatives exist for Go and JavaScript.

If using Netlify, consider a link-checking plugin that checks tens of thousands of links for each “git push” of the website Markdown in about two minutes.

The PyPI releases are out of date so instead of the usual

pip install linkchecker

we recommend using the development Linkchecker code

git clone --depth 1 https://github.com/linkchecker/linkchecker/

cd linkchecker

python -m pip install -e .

Internal/external links are tested recursively. This example is for a Jekyll website running on laptop:

linkchecker --check-extern localhost:4000

The checking process takes several minutes, perhaps even 20-30 minutes, depending on your website size (number of pages & links). Pipe to a file as below if you want to save the result (recommended).

List options for recursion depth, format output and much more:

linkchecker -h

Save the output to a text file:

linkchecker --check-extern localhost:4000 &> check.log

monitor progress with:

tail -f check.log

The benchmarks for Raspberry Pi 2 and Raspberry Pi 3 show far faster performance than the Raspberry Pi 1 / Zero. Using the graphical desktop and 4 GB of RAM, the Raspberry Pi 4 is useful for modest desktop use. The new CPU-direct USB and Ethernet of the Raspberry Pi 4 brings drastically faster IO.

The Raspberry Pi Zero has an ARMv6 single core CPU but much lower idle power for battery-powered applications.

Beaglebone deterministic IO due to the PRU is a key advantage for certain applications. The Beaglebone DDR3L RAM, and ARMv7 CPU is nearly twice as powerful as the Raspberry Pi Zero, native Ethernet, onboard SSD (eMMC), better onboard I/O (for local sensors), etc. However, CPU or GPU demanding tasks should consider the latest Raspberry Pi or Intel NUC, etc. instead.

Fortran coarrays are an abstract, higher-level construct than using MPI directly. Coarrays support multiple parallel processing libraries including MPI. The simpler Coarray syntax can make debugging easier in general. Gfortran, NAG, Intel oneAPI and Cray are among the compilers supporting Fortran 2018 coarray features so useful in high-performance computing.

Coarray Fortran as enabled in Fortran standard is available from multiple compilers. Compilers with built-in coarray support (not needing external libraries) include:

• Intel ifx
• NAG nagfor
• Cray ftn

The OpenCoarrays library is popular for enabling coarray features in GFortran. Install Fortran OpenCoarrays library for Gfortran by:

Ubuntu / Debian / Raspberry Pi (packages names may vary by Linux distro version)

apt install gfortran libcoarrays-dev libcoarrays-openmpi-dev

macOS: using Homebrew

brew install gcc opencoarrays

When manually compiling, add -fcoarray=lib and -lcaf_mpi. For example:

gfortran -fcoarray=lib myfun.f90 -lcaf_mpi

When manually running use cafrun like:

cafrun -np 4 ./myprog

Intel Fortran standard support includes coarrays.

ifx -coarray

enables the coarray features.

OpenCoarrays includes CMake scripts. CMake itself can recognize generic coarray support with FindCoarray.cmake

An example top-level CMakeLists.txt using Fortran coarray contains:

find_package(Coarray REQUIRED)

add_executable(coarray_pi pi.f90)
target_compile_options(coarray_pi PRIVATE ${Coarray_COMPILE_OPTIONS})
target_link_libraries(coarray_pi ${Coarray_LIBRARIES})

add_test(NAME CoarrayPi
COMMAND ${Coarray_EXECUTABLE} ${Coarray_NUMPROC_FLAG} ${Coarray_MAX_NUMPROCS} ./coarray_pi)

Fortran coarray examples includes CMake for Fortran coarrays.

• Intel Coarray basics
• coarray 32,000 worker example
• coarraylib
• coarray fortran intro slides

The GCC -march=native flag can cause unexpected and difficult to diagnose error messages on certain CPU arches, including high end Intel Xeon CPUs. The compiler error messages generally don’t give any clues as to what the real problem may be. While there can be a small to significant performance benefit to using the GCC flags enabling special features in a CPU arch, we feel it’s important to provide software that “just works” by default, even if not completely optimized for a given CPU. The end users that care most about performance probably have already learned the appropriate flags for their CPU, and are willing to fiddle with the CPU arch flags to get the program running at optimum.

End users can provide the appropriate CPU arch flags for their system at compile time if desired like:

gfortran -mtune=native foo.f90

or for using build systems like CMake or Meson:

FFLAGS=-mtune=native cmake -B build
cmake --build build

FFLAGS=-mtune=native meson setup build
meson compile -C build

GCC CPU flags: GCC will show a neat list of which flags are enabled and disabled like:

gfortran -Q --help=target foo.f90

Or to just show the raw flags being used

gfortran -v foo.f90

Reference: Gentoo safe Cflags and GCC optimization. ARM benchmarks of -mtune and -march.

Git contribution calendars are a key way a future client, employer or colleague gets a quick glance at your individual Git contribution level. Consider including private contributions on the profile. GitHub and GitLab contribution metrics include more than just Git commits and the corporate versions offer deeper insights to managers. GitHub depicts contribution types in the contribution rose:

Software is a common core skill necessary for many fields, and it’s important to show the distinction of your software work over time. Ensure your Git commits are counted over time with the correct Git commit email configuration.

Consider using a “noreply” email address in Git repositories to avoid spam. Git contribution credit is based on having a Git email associated with your Github account.

• GitHub
• GitLab

By default, the email address

username@users.noreply.github.com

is associated with the GitHub account. Be sure this email address is correct on each computer used to make Git commits to get Git contribution credit.

git config --get user.email

Correcting email Git commits is possible via a sequence of Git operations.

To avoid connection failures requiring reboot of a Windows PC upon resuming from sleep/standby, consider disabling Windows Modern Standby (formerly known as Connected Standby) during sleep. Modern Standby uses WiFi, Ethernet or other connections, periodically rousting the laptop from deep sleep to sync email and fetch updates. Only certain PCs are capable of having network connectivity during sleep. Some PCs lockout (make invisible) the Modern Standby option since there are firmware bugs that would keep the laptop awake (Dell XPS for example). The option to disable Modern Standby is only present if the laptop supports it and the OEM hasn’t disabled it.

Advantages of disabling Windows Modern Standby:

• save battery by not having your computer connected to Wifi while sleeping.
• not connecting to possibly rogue AP as you travel about, or being subject to tracking.
• some hardware (e.g. Microsoft Surface Pro) often fails to connect to Wifi on wakeup until manually going to Device Manager and disable/enable cycling the Wifi adapter.

Windows Wifi power setting is under the Control Panel → System → Power & Sleep. If present, uncheck under “Wi-Fi” the “stay connected…sleep” options.

Check configuration: from Command Prompt, type:

powercfg /a

# which is a shortcut for:

powercfg /availablesleepstates

if the response includes

Standby (S0 Low Power Idle) Network Disconnected

then the PC is set to disconnect from WiFi on sleep. This command works even if the OEM has hidden the option in the Control Panel (like Dell XPS).

SleepStudy tool

To help diagnose problems with Modern Standby, Windows provides from the factory the SleepStudy tool, invoked by:

powercfg /sleepstudy

This outputs an HTML report–copy and paste the filename from the Command Prompt into the web browser. The filename is like “%WINDIR%/System32/sleepstudy-report.html” The Windows SleepStudy report contains a great deal of nerdy but useful power system info, including battery life state.

EXFAT is a convenient format for sharing files between macOS / Linux / Windows / Android. However, ExFAT does not support symbolic / soft links. Operating systems have supported EXFAT for many years:

• Windows: XP
• macOS: 10.6.3
• Linux: Red Hat 5, Ubuntu 10.04, etc.
• Android: 4

However, continue to use FAT32 instead of EXFAT for test equipment (e.g. Agilent) and some older USB drive accepting printers (print direct from USB stick).

These file systems are commonly, but not universally supported across OS’s.

NTFS: macOS (including macOS 10.15 Catalina): NTFS is read-only by default. Android needs additional third-party software for NTFS

FAT32 is readable by Windows / macOS / Linux / Android, but is limited to 4 GB file size. With many USB HDDs over 2 TB, you would have to break up your HDD into multiple partitions for FAT32 – quite inconvenient. You could use FAT32 for a small USB flash drive. For old printers “USB Direct Print” it may be that only single partition FAT32 is supported. Keep a small USB drive formatted to FAT32 for this purpose.

ExFAT is generally the best choice for universal support on USB HDD. For USB flash thumb drives and SD cards, I format most to ExFAT. Keep a small flash drive formatted to FAT32 for the USB port on old USB direct print printers.

For Linux kernels older than 5.4, or for arches where the Linux distro didn’t include ExFAT in the kernel, you may need to use FUSE ExFAT support:

apt install exfat-utils

As always there are performance penalties with FUSE. It is possible to switch from FUSE to kernel ExFAT.

Hugo can have 100x faster build times than Jekyll. Hugo offers a more powerful design experience. We recommend Hugo in general for fast, HTML5 standard compliant and robust static site generation.

Download Hugo binary The Hugo Jekyll importer seemed broken. This Python-based Jekyll to Hugo converter can be used instead.

Select a Hugo theme and clone the theme into the site. We use “hugo-flex” theme–we have an example hugo-flex template.

cd ~/myHugoSite

git submodule add https://github.com/ldeso/hugo-flex themes/hugo-flex

Enable the theme by adding to hugo.yaml a line:

theme: hugo-flex

Hugo Markdown parsing may use different default options or parser than Jekyll. As a result, you may need to manually fix pages’ Markdown syntax. You may need to rewrite the internal page links to categories and tags, something perhaps like:

find content -type f -name "*.md" -execdir sed -i 's,](/category/#,](/categories/,g' {} +

find content -type f -name "*.md" -execdir sed -i 's,](/tag/#,](/tags/,g' {} +

To convert options you might see in examples between TOML and YaML etc. consider this web converter.

Sometimes minor updates to a blog post are made without wanting to change the main posted date of the article. To have this update time reflected in the article metadata, telling search engines the “freshness” of the article, enable the last updated metadata from Git commit time by adding to hugo.yaml:

enableGitInfo: true

It’s typically beneficial to set a limit to the number of posts in the site RSS feed. This can speed the site build by a factor a 10 for large sites, and improve the update speed of search engines. Check that the RSS limit is working by checking localhost:1313/index.xml from the site preview hugo serve -O.

In hugo.yaml:

rssLimit: 10

To disable RSS for taxonomies (tags, categories) that aren’t useful for many site, additionally add to hugo.yaml:

outputs:
  taxonomy:
  - HTML
  section:
  - HTML

A key design pattern in Hugo is overriding theme files without editing the theme files themselves that are in a Git submodule. Use the lookup order for partials and templates. This will keep your web design clean, minimal and DRY.

Redirect page URL with Hugo front matter tag aliases, equivalent to Jekyll redirect_from.

aliases:
- /old-url

The Surface Go should be considered against entry-level Surface Pro. Surface Pro 7 has proportionately more capability than the Surface Go 2 vs. cost. The benchmarks and initial reviews of the Core m3 2020 Surface Go 2 are favorable.

The Surface Go is adequate for video conferencing as long as you’re not using other programs at the same time. While headsets are better for any laptop, the built-in array microphones of the Surface Go and Surface Pro 7 are good for videoconferencing. The Surface Go generally seems to have a great array microphone. It is relatively straightforward to dictate using Windows Voice Commands to compose text with the Surface Go. This is accomplished in general by pressing ⊞+h in any application, bringing up a dictation bar. Windows voice dictation accuracy is OK, but is not nearly as good as Android’s voice typing system.

The Surface Go can be charged at least in three ways, which we find quite flexible:

• OEM 24 watt (Surface connector)
• USB-C PD to Surface connector adapter (need 15 volt USB-C PD adapter, works with other Surface devices also)
• USB-C PD 18 watt or higher charger

The Surface Go is fine to play video at the full screen resolution. The web browser is a bit slower on the 2018 Surface Go than on a mid-range laptop priced 3 times higher than the Surface Go. If one is building code and running simulations, those will feel slower on the Surface Go, like a 5 year old laptop would feel.

Since at least 2018, the creative work tablet market has had Windows and Linux devices as a great choice. Features to consider in a tablet for creative work include:

• Windows or Linux OS instead of iPadOS or Android
• cost $500-700
• microSD slots to expand storage cheaply
• 128 GB SSD instead of too-small 64 GB (unless you know a microSD card is fast enough for the work)

Inexpensive tablets in general suffer poor display resolution and slow CPUs.

Windows laptops with 5G wireless are available, but with mobile phone hotspot plans commonplace, it may be more feasible to not get specific 5G laptops. Whatever tablet one is considering for creative work, don’t neglect the ~ $100 keyboard price. Windows tablets are generally well-suited for creative STEAM work.

Typically look at last year’s models (new, and especially refurbished) to get them at about half the price of this year’s models. For refurbished electronics, the “bathtub curve” of early failure has already been worked out.