Scientific Computing

Idiomatic Python pathlib

pathlib is an object-oriented, Python standard library way to handle paths and filenames vs. “dumb” strings. pathlib.Path() replaces most cumbersome code from os.path and glob.glob.

pathlib is standard library and supported throughout Python natively, as well as in SciPy, Numpy, et al.

Eliminate messy os.path.join():

filename = pathlib.Path('mydir/a/b') / 'myfile'

Benchmark: pathlib is very efficient, for example with Python 3.6.2 and Ipython 6.1.0:

%timeit Path('~')
2.46 µs ± 24.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

%timeit Path('~').expanduser()
7.35 µs ± 7.13 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

pathlib.Path.glob() recursively searches for files.

Example: search for all .py files under home directory:

allpy = pathlib.Path('~').expanduser().glob('*.py')

allpy is a generator that you can either

  • iterate over with a for loop
  • enclose in sorted() to get a list. flist = sorted(allpy)

Matlab array size limit

Matlab array size limit attempts to stop the user from creating individual arrays that dig into swap memory. Using swap memory is normally undesirable for data computations because the swap memory can be several orders of magnitude slower than RAM.

However, Matlab doesn’t check the total size of all arrays in the workspace, so the user must be mindful of memory management as in any programming language or data processing task.

The matlab-stdlib ram_free function tells the physical free RAM in bytes across operating systems for Matlab or GNU Octave.


Related: free disk space check Matlab

Canobie Lake Park tech notes and wireless

Canobie’s famous carousel with mechanical music maker has a remarkable amount of LED-replaced lights:

36 sections * (5 + 15 + 12 + 18) + (49 * 8) lights/section

That’s about 2192 lights on the carousel.

If we assume the legacy incandescent lighting was 15 Watts each, that would be 32.9 kW of lighting on one carousel! I think the incandescent lighting consumed more than 7 watts/light because that’s what old night lights consumed. In any case, about 20 - 30 kW seems a reasonable estimate.

LED replacement for the carousel lighting would use about 10% of incandescent power, so nowadays that’s still 2-3 kW – more than an electric oven! If we assume a 10 horsepower motor, that would be roughly 60 amps at 230 Vac single phase, for about 14 kW motor power.

So, an LED-retrofitted carousel might draw about 2 kW sitting still, and 16 kW while spinning. If we assume 3 hours/day of lighted operation, and 6 hours/day spinning and $0.10/kW electric rate, then the electricity cost to run the carousel is estimated at.

(3 * 2 + 6 * 14) * 0.1 = $9.00 / day with LED lighting

(3 * 20 + 6 * 14) * 0.1 = $14.40 / day with incandescent lighting

However, one must consider the considerable labor cost of relamping the carousel. Relamping labor is one of the considerable cost savings for commercial LED light upgrades. Consider the cost of relamping on an incandescent Ferris wheel!

Two-way radio: Canobie Lake Park uses VHF (150 MHz) Motorola CP series portables.

GMRS repeaters on 462.550 and 462.575 (not park-related) were audible. Because of the rural location and spread out nature of the park, simplex (direct) FRS/GMRS radios work well.

Canobie Lake Park’s maintenance of equipment made before the World Wars is impressive. Several pieces were made by W.F. Mangels Co., of Coney Island, NY.

Bumper Cars: Barbieri Azzera (spelling?)

Making/using wineprefixes

Running complicated Windows applications on Linux using WINE requires separate WINEPREFIX. Each WINEPREFIX has unique Windows configuration, without disturbing the setup of other Windows programs on Linux.

This example is for Irfanview, a highly-useful image Windows viewer/editor that runs well on Linux.

  1. Make the new wineprefix. You would make any special configuration in the GUI that pops up.

    WINEPREFIX=~/.wine_irfanview winecfg
  2. Install the winetricks prerequisites for your program. Here, Irfanview requires mfc42.

    WINEPREFIX=~/.wine_irfanview winetricks mfc42
  3. Install the new Windows program

    WINEPREFIX=~/.wine_irfanview wine iview_setup.exe

RTL2832 ADS-B decoder

Decode, log, and display 1090 MHz ADS-B aircraft transponder data with your $20 RTL2832 USB receiver with dump1090. Install:

Add full-access USB privileges:

adduser $(whoami) plugdev

logout then login. Install RTL-SDR drivers and prereqs:

apt install librtlsdr-dev rtl-sdr gcc make

Setup dump1090 ADS-B decoding and logging software

git clone https://github.com/antirez/dump1090

cd dump1090

make

Run:

./dump1090 --interactive

Advanced ADS-B decoding with RTL-SDR

  • This gist explains using dump978 and dump1090 with RTL-SDR
  • COTS $220 978 MHz & 1090 MHz portable receiver is based on Raspberry Pi 3.

PiAware

FlightAware has a fork of dump1090 that uploads to the FlightAware network so you can contribute to the global flight database. This can also run on Raspberry Pi or other Linux computer. They have an optional inexpensive receiver with improved filtering and performance for ADS-B performance.


Related: Quickstart RTL-SDR on Linux

9V battery can not charge modern cell phone

Before the emergency hits, get a $20 10,000 mAh or larger rechargeable USB power bank. It’s possible to get 2-3 phone recharges from this bank for each recharge of the bank.

9V battery cell phone charge from 2014 has been disproved independently by the Daily Mail.

A 9V battery has less than 100 mAh available at the 500-1000 mA charge rate demanded by modern smartphones. This means the 9V battery current output quickly drops to near zero, while the phone still sees the input voltage, kicking the phone CPU into full power mode.

When a smartphone sees a 5V input on its charge connector, it thinks it’s on a source of adequate power, and starts checking email, syncing photos, putting the CPU into full power mode, etc. Phones generally do NOT check if the charging source is actually adequate to have a net positive charge. This means the phone battery will discharge faster by having a 9V battery connected via a cigarette lighter adapter.

SSHFS remote softlinks requires follow_symlinks

Softlinks are useful on any operating system to replace long, complicated directory paths with a simpler one. Options for accessing remote softlinked paths over SSHFS is best done with the sshfs -follow_symlinks option.

Example:

  • Server: joe@1.2.3.4 with external HDD /media/BigHDD
  • Laptop: remote SSHFS ~/X

Setup server:

ln -s /media/BigHDD ~/BigHDD

On laptop create joeBigHDD.sh

#!/bin/sh
sshfs -o follow_symlinks joe@1.2.3.4: ~/X

then

mkdir ~/X
chmod +x joeBigHDD.sh

Connect to remote HDD from laptop via SSHFS

./joeBigHDD.sh

cd ~/X/BigHDD

To disconnect the SSHFS drive, on laptop:

fusermount -uz ~/X

Fix Windows error "missing class msstdfmt.dll"

Microsoft removed 2gb152.exe (1MB) that was formerly at microsoft.com/en-us/download/details.aspx?id=12972. CNet has msstdfmt.dll inside file 2gb152.exe at:

https://download.cnet.com/Exchange-2000-Tool-PST2GB/3000-2094_4-10731662.html.

Extract msstdfmt.dll from 2gb152.exe and copy msstdfmt.dll to:

  • Windows XP: %WINDIR%/system32/
  • Windows 7: %WINDIR%/system/syswow64/

Register this file in Windows from an Administrator command prompt, typing:

  • Windows XP: regsvr32 %WINDIR%/system32/msstdfmt.dll
  • Windows 7: regsvr32 %WINDIR%/syswow64/msstdfmt.dll

Find device open network ports without NMAP

Using the standard Python install (no extra packages), find SSH or other types of servers by Python script.

It uses Python sockets to:

  1. check if a port is open for each address in a subnet
  2. connects to that port, printing server messages
  3. optionally looks for a particular string to verify the server is enabled.

Alternative: find servers with unknown IP address but known ports open is by using Nmap.