To change this,  go to Terminal -> Preferences and click on the Profiles tab. In this tab, on the profile that you wish to use, click on the "Keyboard" tab. Click on the "+" button below the list of keymaps to add mappings for Home and End with the following escape sequences. Note that in the key mapping editor, use the Esc key to enter the starting escape sequence \033. Typing \ will result in an escaped backslash (i.e. \\ ) instead.

• Home: \033OH
• End: \033OF
• F1-4: \033[11~, \033[12~, \033[13~, \033[14~

The function keys should already have a keymap, edit those so that they work as intended in terminal programs. Finally, it might also be useful to check the option below the keymaps "Use Option as Meta key", I do use it for several functions in Vim ( <M-... ).

Don't want to do it yourself? Here's my Terminal profile with the keymaps configured, together with my color scheme.

# Packages we use for plotting

import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd

In subsequent code snippets, data is an array, or slice of a pandas DataFrame, df[feature_cols], where each row is an data point and columns are feature dimensions.

A colab notebook for this post is available here.

PCA: Principal Component Analysis

PCA finds the direction where the most variance is observed, set at first direction. Find the next largest variance after removing the first, set as next direction, and repeat this process until desired number of components are obtained. We are often able to stop well below the original number of dimensions, while capturing the majority of the variances in the data.

As a visualization method, PCA is good when the data is already linearly separable. However, it might not be as useful if the data lies on a lower dimension manifold embedded in a high dimensional space. It is also relatively cheap to compute, thus making it a good first thing.

from sklearn.decomposition import PCA

pca = PCA(n_components=2)
pca_result = pca.fit_transform(data)
df['pca_0'] = pca_result[:, 0]
df['pca_1'] = pca_result[:, 1]
print(f'Explained var: {pca.explained_variance_ratio_}')

plt.figure(figsize=(16,10))
sns.scatterplot(
    x=f'pca_0', y=f'pca_1',
    hue="y",
    palette=sns.color_palette("colorblind", 10),
    data=df,
    legend="full",
    alpha=0.3
)

t-SNE: t-distributed Stochastic Network Embedding

Suppose that our data is inherently low-dimension but lives in a high dimensional space (a rolled up 2D sheet (swiss rolls), a tangled strand of string, are common examples of such cases), PCA and other linear methods would not be an effective visualization.

from sklearn.manifold import TSNE

tsne = TSNE(n_components=2, verbose=1, perplexity=50, n_iter=300)
tsne_result = tsne.fit_transform(data)
df['tsne_0'] = tsne_result[:, 0]
df['tsne_1'] = tsne_result[:, 1]

plt.figure(figsize=(16,10))
sns.scatterplot(
    x='tsne_0', y='tsne_1',
    hue="y",
    palette=sns.color_palette("colorblind", 10),
    data=df,
    legend="full",
    alpha=0.3
)

t-SNE, however, contains some hyperparameters and not setting them correctly could lead to misreading of the manifold. Here's good interactive post to see how each of these parameters matter and how to avoid certain pitfalls when using t-SNE as a visualization technique. How to Use t-SNE Effectively (distill.pub)

UMAP: Uniform Manifold Approximation and Projection

UMAP is a method that isn't included in scikit-learn. Using it is almost exactly the same as scikit-learn methods.

umap_reducer = umap.UMAP()
umap_result = umap_reducer.fit_transform(data)

df[f'umap_0'] = umap_result[:, 0]
df[f'umap_1'] = umap_result[:, 1]

plt.figure(figsize=(16,10))
sns.scatterplot(
    x=f'umap_0', y=f'umap_1',
    hue="y",
    palette=sns.color_palette("colorblind", 10),
    data=df,
    legend="full",
    alpha=0.3
)

Other Methods

scikit-learn is an amazing package. It includes several other dimension reduction methods with a largely similar API.

Changelog

• 2021-06-29 Initial version
• 2021-07-01 Clarity on code, intro and some additional points on PCA

TODO?

• Add additional reading as references
• Add some useful insights and use cases

HTML snippet

<div>
    <div style="position:relative;padding-top:56.25%">
        <iframe src="<youtube-embed-url-here>" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen style="position:absolute;top:0;left:0;width:100%;height:100%;"></iframe>
    </div>
</div>

How this works

Notice that we are not using the width and height properties, instead we are using CSS style. Two important things makes it work, the first padding-top:56.25 creates a 16:9 aspect ratio div box for the iframe to fill up. Next, we set the iframe's style to position:absolute;top:0;left:0;width:100%;height:100%;. This positions the iframe at the top-left corner of the div block and sets it occupy the entire block which is of the correct aspect ratio. Now we have an auto-resizing embedded block that's filled with the desired YouTube video!

Google provides a permanently free  VM (alongside several other services, see them here). This is the smallest instance type (f1-micro) that's available on Google Cloud Platform. We get:

• 614 MB of RAM
• 30 GB persistent storage
• 1 GB egress

For a small personal website, this is more than sufficient.

Create VM Instance

This should be pretty straight forward. From GCP console, navigate to Compute Engine and click on "Create and instance". Make sure to select f1-micro and use a disk size of 30GB for things to be free. You should see that this costs approximately $5 a month, but also an additional note saying that the first 744 hours of this instance is free. Follow the on screen instructions and after a few minutes, your instance should be ready!

A static public IP should be assigned as well, and you should set up DNS setting accordingly with your registrar.

Installing Ghost

I chose Ubuntu as my starting image and configuration for most things is a breeze. Detailed instructions on installing Ghost can be found on Ghost's Documentation page, but as always, there's a paste-able block to save time.

# log into your instance with gcloud compute ssh <name>

#Add user (ghost requires a special username ghost, create another to not have conflict
sudo adduser ghost-user
sudo usermod -aG sudo ghost-user

#Install dependencies
sudo apt-get update
sudo apt-get upgrade
# Nginx
sudo apt-get install -y nginx
sudo ufw allow "Nginx Full"
# MySQL
sudo apt-get install -y mysql-server

# Node
# Add the NodeSource APT repository for Node 12
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash

# Install Node.js
sudo apt-get install -y nodejs

# Ghost-CLI
sudo npm install ghost-cli@latest -g

With all the software components required, its now time to actually install Ghost. MySQL, in some cases, might require you to have a password, this can be set with:

# To set a password, run
sudo mysql

# Now update your user with this password
# Replace 'password' with your password, but keep the quote marks!
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password';

# Then exit MySQL
quit

Finally, we will install ghost in /var/www as that's the convention.

sudo mkdir -p /var/www/ghost
sudo chown ghost-user:ghost-user /var/www/ghost
sudo chmod 775 /var/www/ghost
cd /var/www/ghost

sudo -u ghost-user -i
ghost install

The install script will ask you several questions. It's pretty straightforward and allowing the script to set up Nginx, MySQL user and database, systemd for automatic startup, etc. Site name and other details isn't important if you are importing from another site.

Updating Ghost

New version is release? Updating is way easier now as we don't need to dance around Heroku's peculiarities.

cd /var/www/ghost
sudo -u ghost-user -i
ghost check-update
ghost upgrade
exit

Migrating Content

Fortunately, migrating content with Ghost is incredibly painless. Every single configuration and post can be exported as a giant, glorious json file. Just head over to the admin page /ghost/ and click on "Labs". There you should see an option to export your content.

This json file can then be easily imported. Just head to the same "Labs" page on the newly running Ghost website hosted on the free-tier VM.

Performance of the VM

I'm serving quite a few things on the VM, mostly random stuff that I'm experimenting with. For the most part, things run reasonably well. However, occasionally the SQL server seems to quit as memory is very limited. I found it helpful to enable swap. This can be done so easily with dphys-swapfile

sudo apt-get install -y dphys-swapfile

#Configure swap size
sudo vim /etc/dphys-swapfile

Finally, Cloudflare's free website plan could also be used to improve overall site performance and some bandwidth. Simply create a free account, add a free website and everything else should be pretty straightforward.

I work mostly on my Dell XPS 9365 these days. Since I'm working with deep learning, it's often helpful to have a GPU locally for experimenting. Since I've been able to get my hands on an Titan RTX, I've decided to go ahead and give my main workhorse a boost when I'm at my desk.

Getting an external GPU working is no longer as difficult compared to several years ago when USB-C and Thunderbolt was initially introduced.

Almost plug and play, simply plug everything in and install CUDA drivers as per instructions from Nvidia.

If you were to reboot now, you will find that the graphical login manager will fail to start. Studying the logs reveals that X isn't able to find a usable display. This is due to X not allowing external GPUs by default. If you have an internal GPU, you might not face this problem. The exteral GPU is now available for CUDA, but not for running X.

To get X working, we need to add Option "AllowExternalGpus" "True" to the X configuration template /usr/share/X11/xorg.conf.d/10-nvidia.conf.

This is how the file should look like after the edit:

Section "OutputClass"
    Identifier "nvidia"
    MatchDriver "nvidia-drm"
    Driver "nvidia"
    Option "AllowEmptyInitialConfiguration"
    Option "AllowExternalGpus" "True"
    ModulePath "/usr/lib/x86_64-linux-gnu/xorg"
EndSection

Important note about hot-plugging

I've not tried hot-plugging and I have no idea what will happen if I do so, but this functionality isn't required by me for now. It should work in theory and much more information can be found in the incredibly useful site at eGPU.io, where I did my research before purchasing my eGPU enclosure.

My setup

• Dell XPS 9365
• Razer Core X Chroma
• NVidia Titan RTX
• Ubuntu 18.04.3 LTS

So Github now has security warnings, this means updates are due? Update instructions on Ghost.org doesn't really help a Heroku hosted option. Took me a while to figure out how to update everything.

Download and extract the new version over the old ones

#Download the newer version (1.26.0 as of writing)
wget https://github.com/TryGhost/Ghost/releases/download/1.26.0/Ghost-1.26.0.zip
cd $APP_DIR

#Overwrite all the old files. Additional node modules installed needs to be added later
unzip ../Ghost-1.26.0.zip

#Reinstall the storage adapter
yarn add ghost-github
#Also fix the submodule with the configs
git submodule foreach git pull origin master

At this point, there will be a bunch of updated files in node_modules. Make sure to ignore this in the git commit.

Small edits required for free databases

The free JawsDB database that we configured previously has a limit of 10 connections. The defaults used by the database migrator seems to exceed this limit. This limit can be set by setting a config variable:

heroku config:set database__pool__max=2

However, it seems like we this value will get interpreted as a string. To fix this, edit core/server/config/index.js (around line 30):

nconf.env({
    seperator: '__',
    parseValues: True,
});

Now we should be ready to commit and push everything

git add .
git commit -m "Update to 1.26.0"
git push heroku master

#Migrate database 
heroku run knex-migrator migrate db

Some security issues still show up on Github.. EOL for 1.x is Jan 2020. Hopefully upgrading to 2.x or 3.x won't be too difficult. But that's for a later time.

Github Storage Adapter Fixes

Updated repo can be found here

index.js was copied from node_modules after yarn add ghost-github. Processing of heroku environment variables were added to the constructor.

Radar stands for "RAdio Detection And Ranging", initially a top-secret military technology for detecting invading aircraft long before they are visible, is now making its way into our daily lives. Many modern vehicles are equipped with short-range radars as a safety feature, in adaptive cruise control and collision avoidance systems. Google's Project Soli takes this to the next level by using it as a close-range sensor for use in mobile devices.

Basic Principles of Radars

Radars work on a simple idea: send out a radio signal, wait for an echo. The time it takes for the echo to arrive is directly proportional to the distance of the reflecting object.

A basic version of this idea would be a Pulse Radar^[1]. Transmit is on for an instant, followed by a period of waiting for echoes. Mathematically, the transmitted signal is:
$$S_T = A(t)sin( 2 \pi f_c t + \phi_0 )$$

Where $A(t)$ is a constant transmit amplitude when the radar is transmitting and zero otherwise. $f_c$ is transmission frequency and $\phi_0$ is the starting phase. Without loss of generality, can assume that the starting phase is $0$ and will drop the term for clarity of notation, and only reintroduce it where the difference is significant.

In addition to estimating range from the time delay, non-zero relative velocity results in frequency shifts in a phenomenon known as the Doppler effect^[2]. As the transmitted signal is a single frequency, we can estimate the relative velocity of the reflecting object by measuring the Doppler effect that causes a change in frequency of the reflected pulse.

Although simple in terms of operating principles, due to the speed of light, pulse radars are blind at very short ranges (below 1Km). While not an issue for long-range applications (e.g. aircraft, ships), this makes them of limited use where the range is small.

FMCW Radars

In contrast with traditional pulse radars, an FMCW (Frequency Modulated Continuous Wave) radar transmits a signal who's frequency changes with time, often referred to as a chirp:

$$S_{T}(t) = A_{T} \cos\left(2 \pi (f_c + f_\tau(t) ) t \right)$$

Where $f_c$ is the starting frequency and $f_\tau(t)$ is a function describing how the frequency changes over time. One possible waveform is a sawtooth (in frequency-time) signal. i.e. (for single chirp):

$$S_{T}(t) = A_{T} \cos\left(2 \pi (f_c + B t ) t \right)$$
Where $B$ is the slope or the rate of frequency. For the rest of the discussion, we assume that we are working with a sawtooth wave.

Similar to the classical radar, we expect to receive a time-delayed and Doppler-shifted version of the transmitted signal. In contrast with the classical radar, both the transmitter and receiver are on simultaneously. Thus, there are no problems with very short ranges.

Estimating Range

The reflected waveform is a delayed version of the transmitted wave. Again, by measuring this delay, we can compute the distance the object is away from the radar. At the receiver, a mixer (multiplier) mixes the reflected signal with the transmitted signal. Next, this signal passes through a low-pass filter and is sampled by an ADC. At any instant, we can describe the signal as:

$$S_{rx} = A \cos(\alpha)\cos(\beta)$$

Where, $\alpha$ is the frequency that is being transmitted and $\beta$ is the frequency that had been reflected. Using the product to sum identity, we can see that:

$$S_{rx} = (A/2) \left( \cos(\alpha-\beta) + \cos(\alpha + \beta) \right)$$

In this form, we see that there are two frequency components in the received signal — one of much lower frequency than the transmitted waveform and one of very high frequency. After a low-pass filter, this leaves us with a signal that does not have very stringent ADC requirements, as compared to the original GHz-band signal.

Since the slope is known, we can determine the time delay (and distance) easily as follows:
$$d = \frac{ f }{2B} \cdot c_0$$
Where $c_0$ is the speed of light in free space.

Since the mixed signal gives us a frequency difference, all we have to do is to perform an FFT over the entire chirp, and the (frequency) location of the (amplitude) peaks is directly proportional to the range of the target. In FMCW radar literature, this is often referred to as the "intermediate frequency", "beat frequency" or the IF signal.

On Doppler Effect

With a sawtooth wave, there is no way to disentangle frequency shifts that is due to a non-zero relative velocity. It is treated as measurement noise for low-velocity targets. If this is not the case, a different waveform might be a more suitable choice.

Estimating Relative Velocity

While we are unable to resolve the velocity of a target from a single chirp, if we look across multiple chirps, the relative velocity can be recovered. Recall that we are assuming that the velocity of the target is small, and its range does not change significantly over several chirps. Numerically this results in FFTs with peaks at the same frequency bin. While unable to be resolved as different distances, this small displacement manifests as a phase shift.

Suppose two chirps are sent $T_c$ seconds (usually in the order of microseconds) apart. Recall that the IF signal is a sinusoid:
$$A\cos(2\pi f t + \phi_0)$$

If the object is stationary, the phase term of the first chirp will be identical to that of the second chirp. However, if there is a slight change in distance between the first and second chirp, the IF signal of the second chirp will be a phase delayed version of the first chirp. With phase delay:
$$\Delta \phi = \frac{4\pi \Delta d}{\lambda}$$

Using a $77\text{GHz}$ radar, a $1\text{mm}$ ($\lambda/4$) displacement will result in a $\pi/2$ phase shift, with only an insignificant change in frequency. (The reader is encouraged to plug in some values here to see this. A typical slope, $B$, for a 77GHz FMCW radar is $50 \text{MHz}/\mu s$.)

Rearranging and dividing by the time between chirps, $T_c$, we obtain the relationship between the phase difference and the velocity of the target:
$$v = \frac{\lambda \Delta \phi}{4 \pi T_c}$$

Numerically, the phase difference can be obtained by performing an FFT across chirps. The number of chirps and the period between the chirps determines the velocity resolution.

In a practical FMCW radar system, $N$ chirps are sent and processed as a group in order to determine the velocity of the target. We call this sequence of of chirps a frame and this is the basic unit of FMCW radar signal.

Conclusion

We have now established the basic principles behind the FMCW radar. We saw that by performing 2 FFTs, one within a chirp and another across chirps, we can estimate the range and relative velocity of a reflecting target. To design a system that operates with some desired performance parameters, we leave the following points as things to ponder about:

1. What are the limitations of an FMCW radar?
2. What determines the minimum resolvable distance (i.e. range resolution)?
3. What is the velocity resolution?
4. Is there an ambiguity in velocity estimation?
5. To measure the speed of vehicles, how long should each chirp be? What's the periodicity of the chirps?
6. What about angle estimation?

A reader with some knowledge in digital signal processing should be able to derive these limits with the information in this post. We will leave these topics as an exercise for now, and provide a detailed treatment in the next post.

Google Colab is a massive contribution to the democratization of machine learning. Not only are GPUs available for free (1x K80 at the time of writing), you can also use Google's TPUs (Tensor Processing Units) for free. While there are some limitations, pretty big and non-trivial models can be trained so long as you have access to the internet and a relatively modern browser. What's more, it should not take more than a few minute of your time to try it out.

Select TPU runtime

In Colab menu, "Runtime -> Change runtime type". In the window that appears, under Hardware Accelerator, select TPU.

4 lines to TPU

from tensorflow.contrib.tpu.python.tpu import keras_support

tpu_grpc_url = "grpc://"+os.environ["COLAB_TPU_ADDR"]
tpu_cluster_resolver = tf.contrib.cluster_resolver.TPUClusterResolver(tpu_grpc_url)
strategy = keras_support.TPUDistributionStrategy(tpu_cluster_resolver)
model = tf.contrib.tpu.keras_to_tpu_model(model, strategy=strategy)

That's it. (That's actually just a single line if you don't care about long lines)

If you already have a working Keras model, this is all you need to get it running in colab. Train it as usual with model.fit_generator(...)

Extra Stuff/Notes

For completeness...

Getting your code and data onto colab

This is probably the hardest thing to do. Colab runtimes are given a 50GB temporary storage (approximately 30GB usable). If your code is on github or somewhere publicly accessible, command line tools are available from within the notebook.

The can be downloaded easily like this

!git clone <your-code.git>
!wget http://your.data.server/dataset.tar.gz

Or you can click on the '>' on the left to open a side panel where you can upload files.

Notes and common problems

• Copying back to CPU takes a while. Reducing the number of checkpoints will speed up the training significantly.
• Use of learning rate scheduler is required, even if it's just a constant.
• The initial compilation of the TPU model might take quite a while, especially for very large models.
• Error messages might be a little cryptic. I would definitely get a model running properly locally before running it on a TPU.

Runnable Notebook

https://colab.research.google.com/drive/12aezd43epJ-lmQdpvowIoqXfoXvFiQMX

At the heart of all audio processing algorithms, is some notion of the quality of the results that signal. In compression, the algorithm attempts to reduce the resources(e.g. bitrate, bandwidth, etc.) required while having as little as possible impact on the input signal. In audio enhancement, the algorithm takes an input signal and attempts to produce a signal that scores better on some quality metric.

But measuring quality is hard. The perception of audio is as much a psychological process as it is also physical. Given a reference signal, we can always use the L2 distance, SNR, or some mathematically defined metric as a measure of quality. Such objective distances do not always correlate closely with how an (averaged) human listener perceives them.

Subjective Measure

Mean Opinion Score (MOS)

Since people opinions might differ, it seems reasonable to collect the opinions of multiple listeners, so as to obtain an average opinion on the quality. Expert listeners (people trained to pick up problems with audio) were asked to rate an audio sample against an original with the following scale.^[1]

The arithmetic mean is then computed to obtain the MOS.

MUSHRA

In a similar spirit of MOS, the MUltiple Stimuli with Hidden Reference and Anchor (MUSHRA) is a method of obtaining an averaged opinion of human listeners. This test is aimed at audio files of intermediate quality. ^[2]

Objective Measures

While having expert human listeners in a well-controlled environment is definitely the gold standard of determining the quality of an audio clip, it's not always practical and scalable, and especially when tuning an audio processing algorithm. Here, we look at some objective measures of speech (audio) quality and their definitions. This list is by no means exhaustive.

We define $x$ as the reference signal and $y$ as the signal under test. Capital letters denote a frequency domain representation.

SegSNR (Segmental Signal-to-Noise Ratio)^[3]

Defined as:
$$\frac{10}{N}\sum^N_{i=1}\left(\frac{\sum^M_jx^2_{i,j}}{\sum^M_j(y_{i,j}-x_{i,j})^2}\right)$$

Where $x$ is the reference signal and $y$ is the signal under test. Subscripts $i, j$ refers to start and end time indexes. This computes the SNR of segments and then obtains an average SNR of all segments.

LSD (Log Spectral Distance)

Defined as:
$$ \frac{1}{N}\sum^N_{i=1}\sqrt{\frac{1}{M/2+1}\sum_{j=0}^{M/2}\left(10\log_{10}\frac{|Y_{i,j}|}{|X_{i,j}|}\right)^2} $$
A frequency domain assessment of speech audio quality.^[4] $X, Y$ are the STFT spectrum of the original and signal under test, subscripted by their time index $i$ and frequency bin $j$.

WSS (Weighted Spectral Slope)

$$\frac{1}{N}\sum_{j=0}^{N-1}\left(\frac{\sum_{i=1}^M W_{i,j}(S_{i,j}-X_{i,j})}{\sum_{i=1}^M W_{i,j}}\right)$$

An auditory model based frequency domain assessment of speech audio quality.^[5][6] The main idea behind this algorithm is to compare the slope of frequencies grouped into weighted sub-bands.

PESQ (Perceptual Evaulation of Speech Quality)

This is a very involved objective metric with the goal of reproducing the MOS of human listeners. Several preprocessing steps were performed to align and equalize the input audio, and finally, a simple neural net is used to predict the MOS scores.^[7]

Suitability of Objective Measures

A study^[8] was conducted to investigate how well the objective measures listed above compared with the MOS scores. The authors found that not all objective measures correlate well with the scores given by human listeners. Some measures may correlate well on one type of noise but not the others.

3 types of noise(from TIMIT) were added to original signals, namely the white noise, factory noise, and babble noise. It was found that SegSNR performed poorly under all noise types and the following were well correlated under types:

• White Noise: LSD, WSS, PESQ
• Factory Noise: LSD, WSS, PESQ
• Babble Noise: LSD, PESQ

The authors also claimed that LSD correlates the best with human listeners.

Differences Between Speech and Music^[9]

In order to better understand the perception of audio quality, it's important to understand some properties of audio signals. Equipped with an understanding of the statistics of audio signals, we can then apply these models to enhance or regenerate missing components, thereby improving the perceived audio quality. We can broadly classify audio into 2 categories, namely speech and music. We will look at how these signals can be modelled and also some characteristics of sounds and audio.

Speech Signals

Speech is an important form of human communications. Due to its importance, there is a wealth of studies on its properties, specialized algorithms are often created just for speech signals alone. PESQ, above, is just one example. Applying PESQ to music probably will not give you a reliable score.

A good quality speech signal should be natural sounding and intelligible. Speech signals can be split into 2 components, the voiced component, modelled as an impulse train of the speaker's pitch, and noise-like unvoiced components, modelled by a white noise generator (Figure 1).

While the fundamental frequency of human speech tends to be from around 85Hz to 255Hz (inclusive of both adult males and females), harmonics can be observed up to 8kHz. Energy can also be observed in even higher frequencies due to the presence of unvoiced portions of speech that isn't produced by the vocal cord. Besides segments with spectral content, the ratio of silence to non-silence time segments is also an important property.

Spectrogram of It's all Greek to me.

Music

Music, on the other hand, tends to have clear bandpass characteristics and regular temporal patterns as seen in the spectrogram. The shape of the spectrum is largely dependent on the instrument.

Spectrum of the 20s-22s segment from Mid-Air Machine - Those Who Discard the World^[10]

Given the differences in the statistics of music and human speech, we should expect that objective measures of quality will differ for voice and music. In fact, ITU had also published a PEAQ measure, in a similar spirit to the PESQ for speech. I have yet to find a study on how well the objective measures compare to a (averaged) human listener's evaluation.

Final Words

With all that we are playing around with deep neural nets, selecting or designing a good loss function is paramount to the success of the network. If we were to select a loss function that doesn't reflect how human listeners perceive audio, all might be just a fool's errand.

Revisions

• 6/5/2019: Fix MathJax/Markdown problems resulting in equations not rendering

Appendix:

Implementations for computing some of the above metrics.
Github link

Necroed my old Raspberry Pi 1 with the Wolfson DAC that was collecting dust.

Card specs^[1]:

• Multiple analogue I/O
• Digital IO (SPDIF)
• Class-D amp for direct connection to speakers (some headers need to be soldered)
• Stereo MEMS microphones
• 24-bit, 192kHz output

Much better than on-board audio^[2]:

• Analoge sound generated using PLL
• 11-bit, 48kHz analogue out
• no input

Used to require some image from element14/farnell. Now supported in official images. (For quite a while now^[3]...)

Files to edit

/boot/config

...
# Wolfson audio
dtoverlay=rpi-cirrus-wm5102

/etc/modprobe.d/cirrus.conf

softdep arizona-spi pre: arizona-ldo1
#Fix card numbering, wolfson(cirrus) 0, onboard 1
options snd slots=snd-soc-rpi-cirrus,snd-bcm2835

Configuring the card's inputs and outputs

Download helper scripts here. Extract these and put them somewhere in your PATH

These are basically amixer scripts to help you configure for various tasks:

• Record_from_*.sh to choose recording input
• Playback_to_*.sh to choose output
• Reset_paths.sh to set inputs and outputs to defaults.
• Cirrus_listen.sh to configure IO mixing. (eg. SPDIF to lineout)

Appendixes

Original docs here

Finally, some high-res audio to play with here.

UNetBootin allows creation of a such a stick, but it only takes care of creating stuff that's needed for legacy BIOS boot^[1]. If you are using UEFI (Macs don't do legacy boot), things won't work as expected. To get it working, add persistent to the "Try Ubuntu without installing" entry of grub.cfg and loopback.cfg in the /boot/grub folder of the USB stick that was created. (Or create a new menu entry)

...
menuentry "Try Ubuntu without installing" {
	set gfxpayload=keep
	linux	/casper/vmlinuz  file=/cdrom/preseed/ubuntu.seed boot=casper  quiet splash persistent ---  
	initrd	/casper/initrd.lz
}
...

The casper-rw file is where persistency is stored. This is simply a large file, and formatted with a filesystem, say ext4.

Creating this file in Linux is easy:

dd if=/dev/zero of=casper-rw count=4192 bs=1M
mkfs.ext4 -F casper-rw

With this file on the usb stick, passing the persistent paramerter to the kernel boot options will mount this partition when booting. Any changes made to the root file system will be stored in this file.

To verify that this is configured correctly, run df -h. You should see that theres a line that says:

Filesystem             Size  Used Avail Use% Mounted on
...
/cow                   3.9G 1019M  2.7G  28% /
...

Bonus Round 0 - Hostname!

Perhaps you want to have a fixed host name for your usb stick. This can be done easily by editing the same menu entry by adding hostname=yournamehere.

Bonus Round 1 - Larger Persistent Storage

The limitation of using a loopback file is that the file can't be larger that the maximum file size supported by the underlying filesystem. Since the Ubuntu stick uses FAT32, we are stuck with a maximum of 4GB. This can be overcame by creating an actual partition for the files instead. So long as the partition has the label casper-rw.

Special names

Besides casper-rw, you can also create a home-rw to be automatically mounted as /home.

Furthermore, you can also create casper-sn* and home-sn* to be used as snapshots. These snapshots are copied to the filesystem after the persistent volumes are mounted. (More details here.)

Bonus Round 2 - Encrypted home

First create a fully encrypted partition. Current Ubuntu Live images (last checked 18.04) have dm-crypt included.

Create encrypted partiton with dm-crypt and LUKS

1. Install cryptsetup if not available: apt install cryptsetup-bin
2. Create encrypted partition: cryptsetup -v -y luksFormat /dev/sdXX
3. Open encrypted partition: cryptsetup luksOpen /dev/sdXX home-rw
4. Check status if desires: `cryptsetup -v status home-rw'
5. Fill with zeros for security: dd if=/dev/zero of=/dev/mapper/home-rw bs=1M status=progress (This can take a very long time)
6. Format with desired filesystem: mkfs.ext4 /dev/mapper/home-rw

Automounting

Encrypted partitions can't be picked up by casper boot's automounting, and editing /etc/fstab doesn't work, this file is regenerated each time on boot. Instead edit /usr/share/initramfs-tools/scripts/casper-bottom/12fstab.(No longer works with 18.04, you'll have to regenerate the squashfs file, too much work.)

Run blkid and take note of the UUID of the encrypted partition. Edit /etc/crypttab such that the volume with be setup automatically during boot. Not specifying a passkey will present you a prompt to enter teh passkey during boot.

#name device passkey type
home-rw UUID="..." none luks 

Finally, add/create a line in rc.local to mount:

#!/bin/sh -e
mount -t ext4 /dev/mapper/home-rw /home

Note: since this is done after the live system creates the user, the default ubuntu user will have no home directory. Graphical login for the ubuntu user will fail. Can be fixed with copying over the added home directory.

Final notes

Live systems can be fragile. apt upgrade can break stuff. I recommend keeping the home parition seperate and upgrading the live image every once in a while rather than upgrading individual packages. Also, surprisingly,
Nvidia drivers .run installation works.

Maybe a completely customized LiveUSB is more worth the time? Maybe next time

Edit:

• 2/6/2019 - fixed typos and missin luksFormat in encryption setup

Have more than 1 GPU on your setup? Data parallelism on multiple identical GPUs is easy when training with Tensorflow Estimators, and just marginally less convenient with Keras' model.fit.

Estimators

strat = tf.distribute.MirroredStrategy(local_gpu_list)
runconfig = tf.estimator.RunConfig(train_distribute=strat,
                                   eval_distribute=strat,
                                  )

If the evaluation dataset's input_fn is something that Tensorflow can't figure out how to split/shard, you might run into errors during evaluation. The exact same input function works properly if in train, but will throw some error when doing evaluation.

Keras models

For Keras models to take advantage of multiple GPUs, it's just slightly more annoying. The model has to be created and compiled in the strategy scope.

Example with sequential model:

from tf.keras import models, layers

strat = tf.distribute.MirroredStrategy(local_gpu_list)
with strat.scope():
    model = models.Sequential([layers.InputLayer(input_shape=[64, 64, 3]),
                               layers.Conv2D(3, 64, padding='same'),
                               ...,
                              ])
    model.compile(loss='binary_crossentropy', optimizer='adam')

TODO:

• Splitting datasets/sharding

Edits:

-5/6/2019 added snippet for Keras models

from tensorflow.python.client import device_lib

devices = device_lib.list_local_devices()

Each item is a DeviceAttribute, where we can use to find out the device types and names. Somehow this isn't well documented in the Tensorflow's documentation. Of which, we should find the attributes name, device_type and memory_limit most useful.

Attributes

name

A string that can be used to specify the compute device in Tensorflow, eg, tf.device(devices[0].name), to explicitly state device placement.

device_type

A string, it can take the following values to specify the type of device it is.
-CPU for the CPUs
-GPU for GPUs that are visible to Tensorflow
-TPU for Google's own TPUs, probably will never see this (or maybe we will? Google AIY Edge TPUs)

Also, more recently XLA (Accelerated Linear Algebra) Devices XLA_CPU and XLA_GPU

memory_limit

Total memory in bytes. Perhaps this can be used to compute the batch size to use?

Other Attributes

There's also the incarnation and locality attribute. locality isn't meaningful here as we are refering to local devices. Thus it's always an empty dict.

I have no idea what incarnation is.

Practical usage example for using multiple GPUs with Estimators

local_gpus = [d.name for d in devices if d.device_type == 'GPU']

strat = tf.distribute.MirroredStrategy(local_gpus)

runconfig = tf.estimator.RunConfig(train_distribute=strat,
                                   eval_distribute=strat)

est = tf.estimator.Estimator(..., config=runconfig)

Obtaining device lists with tf.Session()

Another way of obtaining compute devices is with:

import tensorflow as tf

with tf.Session() as sess:
  devices = sess.list_devices()

This however is slightly different and the objects returned are of session._DeviceAttributes. It's name string now also includes where the device, and this can also reference remote devices if an address is passed into tf.Session, say a TPU worker maybe.

TensorFlow's new Dataset API (available from 1.8) makes creating input pipelines much easier. Using it should be painless if you have something is an iterable, one of the common formats (files in a folder, csv, numpy array) or TFRecord, life is gonna be much easier, and from_generator is perhaps the easiest to get any dataset into TensorFlow.

Usage pattern:

1. Create the 'raw' dataset= one of:
   • tf.data.Dataset.from_generator() for some function with a yield
   • tf.data.TFRecordDataset() for reading from TFRecords
   • tf.data.Dataset.from_tensor_slices() for numpy arrays. (Sparse version available too)
   • tf.data.TextLineDataset() for text files like .csvs
2. Apply transforms, if desired, with dataset.map(...)
3. Randomize order with .shuffle(buffer_size=n)
4. Set with .repeat(n). (Pass nothing for it to repeat forever)
5. Set batch size with .batch(n)
6. Obtain iterator with iter = dataset.make_one_shot_iterator()
7. Elements can now be obtained with: x, y = iter.get_next()
8. Build graph using x, y directly. No placeholders needed!

Migrating from TFRecords and QueueRunners

If you have been using TFRecords and QueueRunners, switching over to the new Dataset API will be very painless.

Your original input pipeline should have something like this

reader = tf.TFRecordReader()
_, example = reader.read(filenamequeue)
fmt = ...
features = tf.parse_single_example(example, features=fmt)
x = features['data']
y = features['label']

In the new API, we create a function will all the parsing and preprocessing we need for each example, into a function. This is then applied to the dataset using .map().

def parse_func(example):
    fmt = { <key1> : tf.FixedLenFeature( <shape>, <dtype>, <default_value(optional),
            <key2> : tf.VarLenFeature( <dtype> ), ...
          }
    parsed = tf.parse_single_example(example, fmt)
    return parsed[<key1>], ...

Full basic example with TFRecord

data_raw = tf.data.TFRecordDataset(filename) #or list of filenames

def _parse_func(example):
   #example is a Tensor of bytes. Needs to be parsed with parse_single_example
   example_fmt = { 'x': tf.FixedLenFeature((), tf.string, ''),
                   'y': tf.FixedLenFeature((), tf.string, ''),
                   }
    parsed = tf.parse_single_example(example, example_fmt)
    #parsed is a dictionary of tensors
    #Can do further processing of the tensors now, or simply return them
    return (parsed['x'], parsed['y'])

data = data_raw.map(_parse_func)

#Make random, repeatable and batched
data = data.repeat().shuffle(buffer_size=BATCH_SIZE*10).batch(BATCH_SIZE)

iter = data.make_one_shot_iterator()
x, y = iter.get_next()

#Build graph, note how x and y are used
net = tf.layers.dense(x, 512, activation=tf.relu)
net = tf.layers.dense(net, 512, activation=tf.relu)
pred = tf.layers.dense(net, 10)

loss = tf.losses.softmax_cross_entropy(pred, y)

train_op = tf.train.GradientDescentOptimizer().optimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for ii in range(MAX_ITER):
        _, curr_loss = sess.run([train_op, loss])
        print('Iter: {}, Loss: {}'.format(ii, curr_loss))

On parsing (compressed) images

Images are parsed as FixedLenFeature, even if they might be compressed and of different sizes (byte lenght). This is because FixedLen here refers to the tensor length, not the number of bytes in the Tensor. A variable sized image is still a single element Tensor of type BytesList.

From (almost) anything else with generators

I find that the best part of the Dataset API is the from_generator(). So long as you know how to iterate thru the examples, you should be able to wrap it into the Dataset API without much difficulty.

Example with HDF5 + generators

Heres a toy example of reading a HDF5 file, with keys x and y. (Unfortunately, in H5PY's documentations, these are called datasets)

import h5py
import tensorflow as tf
import numpy as np

in_file = h5py.File('data.h5', 'r')
x_in = in_file.get('x')
y_in = in_file.get('y')

def gen():
    for x, y in zip(x_in, y_in):
        yield x, y

#Lets assume that x is 3 vector of floats and y is an int
d = tf.data.Dataset.from_generator(gen, 
                                   output_shape = ([3], None),
                                   output_types = (tf.float32, tf.int32)
                                   )

That's it! The HDF5 (or which ever esoteric reader that you might have) is now wrapped in a nice Dataset API, with all the batching, pipelined reading, shuffling, available to you!

TODO:

Planned updates:

• [x] Notes on parse function for TFRecord
• [ ] Fancy initializers
• [ ] Using with graphs built with placeholders
• [ ] Boilerplate file gist?

or how this blog was made.

I was in search of a Markdown based blogging platform that I could use for free. Jekyll and Github Pages is really nice, but the lack of a web/app interface to write stuff is rather limiting at times.

Fortunately, there's Heroku and Ghost. Heroku lets you run small scale web apps for free and with some configuration, you can get Ghost to run on it.

Prelimiaries

For the free databases to work, billing needs to be setup on your heroku account. Create your heroku account, and install the command line tool and enable billing. On top of being able to use addons, you get more free dyno hours too! Don't worry, everything used here is free.

Getting a bare minimum blog working

Once you have done all the preliminary stuff, paste the stuff below and you'll have a (mostly) working blog!

Copy and paste without knowing what's going on? Who cares.. I've comments if you are confused

APP_NAME=ghostblog

#Download the archive, newest available at https://ghost.org/developers
wget https://github.com/TryGhost/Ghost/releases/download/1.24.8/Ghost-1.24.8.zip
mkdir $APP_NAME
cd $APP_NAME
unzip ../Ghost-1.24.8.zip

#Create a git repo and commit
git init
git add -A
git commit -m "Initial everything"

#The heroku command will create the app and set up the git remote
heroku create $APP_NAME
#pushing to heroku will automatically deploy the website
git push heroku master

#Free DB plan (kitefin) from JAWS DB
heroku addons:create jawsdb:kitefin

#JAWSDB_URL is set by the addon after the previous command
DBURL=`heroku config:get JAWSDB_URL`
#This returns mysql://<user>:<password>@<server>:<port>/<database>

#Some regex magic to get the params from the address
DBUSER=`sed 's/.*\/\(.*\):.*@.*/\1/' <<< $DBURL`
DBPASS=`sed 's/.*\/.*:\(.*\)@.*/\1/' <<< $DBURL`
DBSERVER=`sed 's/.*@\(.*\):.*/\1/' <<< $DBURL`

#These config vars corresponds to database.connection.* if set in config.*.js
heroku config:set \
    database__connection__user=$DBUSER\
    database__connection__password=$DBPASS\
    database__connection__host=$DBSERVER\
    database__connection__database=${DBURL##*/}  

#Initialize the database with Ghost's initializer
heroku run knex-migrator init

#Set some server parameters
#Heroku seems to start on random ports
echo export server__port=\$PORT npm start > .profile
git add .profile
git commit -m "Server port config"
git push heroku

#Listen on all addresses, url is read by engine to provide a "Home" link
heroku config:set \
    url=https://$APP_NAME.herokuapp.com \
    server__host=0.0.0.0
    

After this you should be able to see your pretty Ghost site at https://$APP_NAME.herokuapp.com!

Configuring the site

Now that all is done, you should be able to access your admin page at https://<app-name>.herokuapp.com/ghost. When you first access this, you will get to create your account and stuff. Next, you probably will want to delete the "Ghost" user to get rid of the example posts.

If you aren't planning to upload any images through Ghost's interface, you are done! If not read on.

SSL, because the Internet is dangerous

If your are using Heroku's subdomain (i.e appname.herokuapp.com) SSL is automatic. Heroku will only allow you to configure SSL certificates if you upgrade to a paid plan. However all is not lost, use Cloudflare as your DNS and you'll get free SSL from Cloudflare. This is as simple as setting the CNAME entry of your site to appname.herokuapp.com.

After doing this, you might want to change the url config variable to point to your custom domain.

Getting file uploads to work correctly

Web apps in Heroku runs in an ephermeral VM. Once shutdown, file are lost. This means that images that are uploaded will stop existing once your app shuts down due to inactivity. Fortunately, Ghost allows custom storage adapters, meaning we can make use of some free services on out there.

I chose to base mine on ghost-github.

However, the author's documentation stated that you are required to have your access tokens/passwords in the config file, stored in clear, and perhaps inadvertantly on some publicly accessible repository like Github.

To add files to github programatically, you will need to get a personal access token from here.
Aside: you probably want to create a separate machine user and share the repo with this user for this purpose. Obtain the token for this machine user^[1].

Note: Assets repo seems to need to be a public repo. The adapter generates some access token if the repo is private, but it doesnt seem to work.

#Get dependencies in yarn.lock and packages.json
yarn install ghost-github

cd content/adapters/storage
git submodule add https://github.com/moodoki/ghost-github.git

git commit -a -m "Add storage adapter"
git push heroku master

#Config vars read by the adapter
#if you are using a shared repo and a machine user, 
#the REPO_OWNER should be set to the actual owner
heroku config:set \
    GHOST_GH_DESTINATION=<folder> \
    GHOST_GH_REPO=<repo_name> \
    GHOST_GH_REPO_OWNER=<github_repo_owner_username> \
    GHOST_GH_BRANCH=<repo_branch> \
    GHOST_GH_TYPE=token \
    GHOST_GH_TOKEN=<access_token>

#Tell ghost to use the adapter
heroku config:set \
    storage__active=ghost-github

File uploads should be working now!

Disclaimer: I'm no expert with Javascript or NodeJS. I have no idea how Ghost is able to get the config vars from the either environment variables or config files transparently

Links

1. Ghost publishing platform. [Source][Official Website][Archive download]
2. My fork of ghost-github storage adapter. [Github]

On hindsight, perhaps running everything in a free VM in GCP might be a lot easier. Although connections are metered, the always free tier is rather generous, most likely more than sufficent for a moderate sized website. More on this perhaps next time