Scientific Computing

I received an email attachment, with no filename extension because of the spam filters in corporate email. I “knew” it was a legitimate file because I had just requested it from a notable researcher. Rather than bother the sender to tell me the original file extension, I determined the file type by:

file emailedfilename

which gave output

gzip compressed data, from Unix

Thus I changed the filename to be emailedfilename.tar.gz since a gzip’d file almost always contains a tar archive, and I could extract the files.

Note that tar is smart enough even with the wrong file extension to work, so I could have instead just done

tar -xf emailedfilename

This discussion pertains to United States of America Federal Communications Commission Part 97 regulations on Amateur Radio.

FCC 47 CFR § 97.5(b)2 lays out the structure for a club station license. In practical terms, this gives amateur radio clubs a memorable callsign of note they can rally around, to build branding, etc. The club station license does not confer operating privileges, but the trustee must hold an amateur radio license.

The trustee will receive FCC official mail, and ARRL LoTW (Logbook of the World) will only sign up clubs through the trustee. Thus clubs should ensure the trustee is someone who actually checks their physical mail, that is on campus at least several times a month on average. The trustee does not have to be be a school employee. However, some club constitution or bylaws require the trustee to be a school employee.

Control Operator vs. Club Trustee: §97.103(b) notes that by default, the station licensee (here, the school club trustee) is the control operator. Even if another person is the control operator, §97.103(a) holds the trustee and control operator equally responsible. Intuitively, §97.105 states the control operator is responsible for “immediate proper operation of the station, regardless of the type of control.”

Student operating privileges: provided the control operator is in control, anyone operating (including non-licensed persons) may use the control operator’s license privileges. Ideally, the club control operator will have an Extra class ham radio license so that operators get to use maximum privileges. §97.115(b) notes that for third-party communications (someone besides control operator working the radio), the “control operator is present at the control point and is continuously monitoring and supervising the third party’s participation”. Many clubs take §97.115(b) to mean the control operator is physically on site, indeed in the radio room itself.

Reference: FCC § 97.5

At the meeting, it was mentioned that

• Juno Waves instrument uses python from day one
• other researchers also using Fortran from Python
• PyAstro is a useful conference
• Major packages like Numpy typically lack funding. Numpy got its first funding in 2017!
• see paper: The AstroPy Problem on funding geoscience software development
• should make open source software part of CEDAR Decadal Survey
• Autoplot: one line command to plot many science formats including: CDF, HDF5, NetCDF and many more.

NetCDF4 uses a subset of HDF5 features, and adds some new features. NetCDF4 reads/writes specially structured HDF5 files. Performance of HDF5 and NetCDF4 is highly similar including on supercomputers. The main idea behind NetCDF4 is a simpler API than HDF5, while maintaining the same performance.

Python h5py makes HDF5 read/write very easy. NetCDF4 is a little more complicated to use from Python.

HDF5 from low-level languages such as C, C++ and Fortran is a little elaborate as compared to NetCDF4 ease of use. Using HDF or NetCDF4 from Python is easier as these examples collapse down to a couple lines of code.

AGU Fall Meeting 2017 Talk ED32A-12

Eclipse MegaMovie is a Google provide to register thousands of images taken during the solar eclipse, to provide spatiotemporal diversity not available any other practical way.

It’s pretty evident this is a laudable first pass. It looks like a lot of photos have over-exposure and dynamic range issues. This could perhaps be addressed by “gap data” techniques. Currently the nice filters are over $1000, but perhaps a compromise filter can be made much more cheaply.

• AGU FM17 poster “Eclipse Megamovie 2017: How did we do?”
• AGU FM17 talk “Eclipse Megamovie 2017: A Citizen Science Project”

We heard from seasoned developers as well as end-users who had the simplest use case: using DavitPy to load/read SuperDARN data. Topics aired included:

• DavitPy hosting: issue where someone leaves, data inaccessible for weeks. Don’t want that to happen with code. Data should be in archive like Zenodo.
  • consider countries with filtered internet. Code mirrored across multiple sites should help in general.
• Breaking up DavitPy into a lean core that strictly handles SuperDARN data and makes basic plots. All else should be in a DavitPy-extras module.
• Some users just want to load the SuperDARN data without using IDL. I noted that a lean DavitPy is amenable for loading transparently into Matlab. That is, Matlab can use Python user code, so that it’s transparent to Matlab users how to load the SuperDARN data into Matlab.

Previously, I had made an overly large pull request that incorporated fixes for the issues below. I will make separate, small pull requests for these issues.

A general issue for any Python program is that pyproject.toml should be used to configure the package as much as possible. A working example pyproject.toml using these features is illustrative.

Avoid Bash scripts as these don’t work on Windows unless using Windows Subsystem for Linux.

Example of rich Python 3 exception handling with backward compatibility:

import six

if six.PY2:
    ConnectionError = OSError

# ...

try:
    # download/upload from server or device
except ConnectionError:
    print('could not connect to device')
    return

Best practice for forward compatibility: do not use if six.PY3 or if not six.PY3, rather use if six.PY2.

Require “new enough” Python version

To avoid lots of needless GitHub Issues and emails from users with obsolete Python versions, set pyproject.toml to limit to Python required by the program.

Antenna Hungária has broadcast Kossuth Rádió (now also available via internet streaming) on 540 kHz for over 40 years with a two megawatt, 50% efficient transmitter on the 303 meter tower at Transmitter Solt. With the old transmitter, four megawatts of 11 kV utility power resulted in two megawatts of RF on 540 kHz. The new solid-state Nautel NX2000 transmitter commissioned in November 2017 consists of five 400 kW transmitters combined, and “only” 2.2 megawatts input power at 11 kV is needed. Six-inch and twelve-inch RF coaxial hardline pump the RF to the combiner and then the antenna tower. This is the most powerful medium wave transmitter in Europe, and is equal to other most powerful two megawatt medium wave transmitters in the world.

Several one+ megawatt transmitters are shown in this graph.

Models: a groundwave propagation model covering 10 kHz - 30 MHz is described in ITU-R P.368. A very simple example (too inaccurate for planning work) is from ITU grwave. During the 1960s when high altitude nuclear weapons testing was permissible, D-layer absorption increases were thought to be responsible for loss of MW skywave reception for several days over thousands of kilometers. Because ground conductivity is so important, a meaningful MW groundwave model should take into account dynamics of earth conductivity in the coverage area.

Reference:

• MWLIST
• 1958 BBC book on LF and MW groundwave coverage modeling

FGPMMOPA6H GPS manual

• PA6H provides 1 PPS (pin 13) after 3-D Fix at 2.8 V (p. 7, 13)
• 3.3V external antennas may draw up to 28 mA (p.15)
• Power supply: 3.3 V, 25 mA
• NMEA: GGA, GSA, GSV, RMC, VTG over 2.8 V TTL (p. 17)

The PA6H GPS module and Raspberry Pi can make an NTP server. Note that the Raspberry Pi has 3.3 V logic so level shifters may be necessary to/from the 2.8 V GPS IO.

NMEA over UART: the Raspberry Pi UART via PySerial can read NMEA with a program such as NMEAutils. The hardware UART connection is to GPIO pins 14 and 15 on the various Raspberry Pi versions.

Red Pitaya NMEA over UART: starting with the Red Pitaya UART Example in C, one could scoop up the NMEA and extract the current time using a 1PPS driven interrupt. Perhaps use PySerial to access /dev/ttyPS1 to use the Red Pitaya UART from Python. The Red Pitaya UART pinout is to pins 7 and 8 of connector E2.

This would not be for setting the clock accurately like NTP, but rather for determining which second the 1PPS tick occurred at. This technique (with different hardware) was used for the HiST auroral tomography system.

The obsolete Fortran 66 statement pause had various behaviors depending on the operating system and compiler. Fortran pause was used for three different purposes in the Fortran 66 / 77 era:

• (most common use) program waits forever, until the user pressed the Enter key, ignoring typed input.
• wait for user to type text on tty stdin, assigning the typed text to a variable.
• (very old programs) drop to a system shell, allowing any shell command to be used

These can be replaced with unambiguous modern Fortran code.

Wait till Enter: the program waits for the Enter key, ignoring any console input.

use, intrinsic:: iso_fortran_env, only: input_unit

print *, 'Waiting for Enter.'
read(input_unit, *)