Eyes-free Linux Ninja!

Arch Linux on the Raspberry Pi and the bsdtar Fix

Mike — Fri, 22 Sep 2017 20:29:56 GMT

A couple of years ago, or more, I wrote a script to get the latest Raspberry Pi filesystem archive from the Arch LinuxARM site, create, partition and mount a new .img file and extract the .tar.gz file into the mounted partitions.

But a while ago, about December 2016 or so I started to get bsdtar errors when extracting the archive.

Note here that it is documented on the Arch Linux ARM site that GNU tar will not work with this operation and it must be bsdtar.

In the last couple of days my need to find a solution has grown more pressing and I went looking for the answer. I turned up this post in the Arch Linux ARM forums.

On page two of the thread it says that version 3.3+ is required.

As my main machine is Debian Stretch my version was not that high.

Below is a script which will get, make and install a sufficiently current version:

#!/bin/bash
#
# Get, build and install the version of bsdtar needed by the Raspberry Pi Arch Linux root file-system get and 
# extract
#

if [ `whoami` != 'root' ]; then
    echo "Script must be run as root, try: $0"
    exit 1
fi

set -e

wget https://www.libarchive.org/downloads/libarchive-3.3.1.tar.gz
tar xzf libarchive-3.3.1.tar.gz
cd libarchive-3.3.1
./configure
make
make install
exit 0

Now:

bsdtar --version

Should show a version > 3.3.

Here is how to clone my repo and use the script:

git clone https://github.com/cromarty/rpi-stuff

Now:

cd rpi-stuff/make-rpi-arch

Execute the script with one argument, which is the version architecture:

sudo ./make-rpi-arch.sh <architecture>

Where <architecture> is one of:

armv6 for the older Pi versions
armv7 for the Pi2
armv8 for the Pi3

Note I have never run an armv8 image.

The script also needs bsdtar (as discussed above), kpartx, and parted.

Internationalising Shell Scripts with Gettext

Mike — Fri, 21 Apr 2017 15:03:48 GMT

As part of my work for the F123 Project I've had to learn how to prepare bash shell scripts for translation into multiple languages using gettext.

This is a little workflow document I have written on the subject.

Shell Internationalisation Workflow

To provide translations for shell scripts we use the gettext package.

Preparing a Shell Script for Translation

Shell scripts contain echo commands to provide user feedback.

We need for the strings displayed by an echo command to be displayed in the language the user of the machine is using.

In the top of the script we put these lines:

export TEXTDOMAIN=myscript
export TEXTDOMAINDIR=/usr/share/locale

. gettext.sh

In the above example myscript is an example script name.

The TEXTDOMAINDIR above contains the usual path to localization for a typical Linux installation. It may be raplaced with:

export TEXTDOMAINDIR=$PWD/locale

For testing a script's translation.

The script author changes typical cho of this syntax:

echo 'Press any key to continue'

With:

echo $(gettext "Press any key to continue") ; echo

We place an extra echo with no arguments at the end because the gettext function does not add a line-feed.

The gettext function is contained in the gettext.sh script included at the top of the script.

If an echoed string contains any shell variables they must be replaced.

For example:

echo "Program name: $0"

Becomes:

PROGNAME=$0
echo $(eval_gettext "Prog name: \$PROGNAME") ; echo

The eval_gettext function is used instead of gettext and the dolla-sign is escaped with a back-slash.

Again an extra echo is put in after the command to echo a translated string.

Extracting Strings into a Portable Object Template (.pot)

Now we run xgettext against our script to create a file of strings to translate.

We do this in this fashion:

xgettext -L Shell -o myscript.pot myscript

This reads all the echo commands which contain references to the gettext and eval_gettext functions and writes them to a .pot (portable object template) file.

Here is a simple example of the contents of a .pot file for a script which contains only one echo command:

# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2017-04-19 22:52+0100\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#
#: myscript:10
#, sh-format
msgid "Hello world: $PROGNAME"
msgstr ""

In the above example there are values in upper-case at the top in the comments which should be filled out with the details of the package-name, and the author details etc.

And this line:

"Content-Type: text/plain; charset=CHARSET\n"

Should be changed to read:

"Content-Type: text/plain; charset=UTF-8\n"

Or whichever character-set we are using.

The Translation Process

The first-cut .pot file should be passed to a translator with the name changed to drop the ending t:

cp myscript.pot myscript.po

Now give the .po (portable object) file to a translator.

After the line which reads:

  "Content-Transfer-Encoding: 8bit\n"

There will be entries of the form:

  msgid "Some string in the original language"
  msgstr ""

A translator places the translation of the string in the msgid entry in the msgstr entry between the quotes.

The translator MUST preserve any shell variable reference that is in the original string. For example:

msgid "Attention! $ERRORCODE"
      msgstr "Achtung! $ERRORCODE"

In this way a translator goes right through the file and translates the strings, leaving the msgid entries as-is, and placing their translations in the corresponding msgstr entries.

Creating `Machine Object` Files

The files which will actually be used when a script runs, to provide translated strings is a .mo file.

We create this from a .po file like this:

msgfmt -o myscript.mo myscript.po

Which will create myscript.mo from myscript.po.

Merging New Strings

Of course the development of a complex script is never static and it may be necessary to add new strings, or to remove old ones. We can use another utility to merge strings from a new .pot file into our .po file and re-create our .mo file.

For this we use msgmerge, like this:

msgmerge -U myscript.po myscript.pot

This will search the .pot file for new or deleted strings and merge them into the .po file. In this way previous translation work is not lost when things change.

And again we then create a new .mo file from the update .po file.

Installing a Translation

Using our myscript example from above, the .mo file is installed in:

  /usr/share/locale/<LANG>/LC_MESSAGES/myscript.mo

Where is the language into which the script has been translated.

For example, for Brazilian Portuguese:

/usr/share/locale/pt_BR/LC_MESSAGES/myscript.mo

The Worklow in Short

Script is written and tested, including gettext support.
Run xgettext to extract all echo strings into a .pot file.
Rename the .pot file to .po and pass to translator.
When step 3 is done, run msgfmt to create the .mo file.
Install the .mo file (after testing).
If something new is written in the script, go to step 2 and repeat to step 5.

Notes/Suggestions

The .po files to be translated should probably either be contained in a directory structure which clearly defines the language, or should contain the language in the file-name, although the completed .mo file cannot.

So, for myscript, perhaps:

myscript/
    en/myscript.en.po (English)
            fr/myscript.fr.po (French)
                        pt_BR/myscript.pt_BR.po (Brazilian Portuguese)

How to Generate a Table of Contents in a Kramdown Document

Mike — Sat, 01 Apr 2017 20:15:59 GMT

Here's how to generate a table of contents in kramdown.



# Heading
{:.no_toc} 

## Table of Contents
{:.no_toc}

* TOC 
{:toc}

Introduction

The kramdown gem converts a kramdown file into html. It generates the id tags for each heading and uses them to generate a table of contents for a document. This becomes obvious if you take a file written in kramdown and convert it to html.

kramdown input-file.md > output-file.html

The ids are put on the generated html for all headings.

How To Automatically Generate A Table of Contents

Typically, you will want to have a numbered list for your table of contents or a bulleted list of entries. This is done by either writing 1. TOC or * TOC at the place in your file where you want the table of contents to be located. Underneath this line, specify the table of contents itself:



{:toc}

You can have either a numbered list of items or a bulleted list, in the normal way for lists:



1. TOC 
{:toc} 

* TOC 
{:toc}

If you do not want a heading to be included in the table of contents, put:



`{:.no_toc}

Underneath the entry you want to exclude from the table of contents, often your top-level heading.

Notes

I have noticed that there is either a bug or an accidental omission in the generation of the toc. For example,, given headings like this:



# First level one heading
## First level two heading 
## Second level two heading 
## Third level two heading
# Second level one heading

I would expect the second level one heading to be a member of the list of level one headings, but it is left hanging in between two lists of second level headings, and is a member of no list. This bug makes sub-lists and then a return to a lower level further down impossible.

So you will need to have all headings which go to make up a table of contents the same level.

A bit annoying but it beats constantly scrolling up and down a document as you write it to try to remember the order of the headings and their names.

Tool for Installing Talking Arch

Mike — Sat, 01 Apr 2017 10:13:51 GMT

I've written a bash script that will install the Talking Arch distro non-interactively.

To use it follow these steps:

Boot Talking Arch
Set up your partitions
`wget http://www.raspberryvi.org/Downloads/ta-tools-1.0.0.tar.gz
tar zxf ta-tools-1.0.0.tar.gz
Make sure the contained script is executable and run it.

It will do the first pre-chroot of the install and write a here script into the chroot and mark it executable.

After the chroot, run the newly created script to create the installation.

Here is the full script:



#!/bin/bash
#
# Talking Arch setup
#

# Change the commenting of the variables below
REGION=Europe
#REGION=America

CITY=London
#CITY=Chicago

LOCALE=en_UK
#LOCALE=en_US

KEYMAP=uk
#KEYMAP=us

DEVICE=${1}

HOSTNAME=arch

ROOTPASSWORD=root

USER=user
PASSWORD=password

# This MUST be a full path to where the script must be in the chroot
HERE_SCRIPT=/mnt/root/001-ta-tools.sh

# Crash out on any error
set -e

echo 'Mounting the disk partition(s)...'
mount ${DEVICE} /mnt

echo 'Pacstrapping the new system in what will be the chroot...'
pacstrap /mnt git base espeakup alsa-utils openssh acpi wget base-devel nmap lynx

echo 'Setting up fstab...'
genfstab -U -p /mnt >> /mnt/etc/fstab

#
# Write a 'here' script into the chroot before we chroot
#
cat  ${HERE_SCRIPT}
#!/bin/bash

set -e

echo 'Setting up localisation stuff...'
sed -i 's:#${LOCALE}.UTF-8 UTF-8:${LOCALE}.UTF-8 UTF-8:' /etc/locale.gen
echo "LANG=${LOCALE}.UTF-8" > /etc/locale.conf
export LANG=${LOCALE}.UTF-8
locale-gen

echo 'Removing /etc/localtime if it exists before replacing it...'
[ -f /etc/localtime ] && rm /etc/localtime
ln -s /usr/share/zoneinfo/${REGION}/${CITY} /etc/localtime

echo 'Setting up keymap...'
echo "KEYMAP=${KEYMAP}" > /etc/vconsole.conf

echo 'Setting the hardware clock...'
hwclock --systohc --utc

echo 'Setting the hostname...'
echo ${HOSTNAME} > /etc/hostname

echo 'Setting the password for root...'
echo -e "${ROOTPASSWORD}\n${ROOTPASSWORD}\n" | passwd root

echo 'Adding an ordinary user...'
useradd -m -g users -G wheel,storage,power,audio -s /bin/bash ${USER}
echo -e "${PASSWORD}\n${PASSWORD}\n" | passwd ${USER}

echo 'Editing /etc/sudoers to grant the usual privileges...'
sed -i 's:# %wheel ALL=(ALL) ALL: %wheel ALL=(ALL) ALL:' /etc/sudoers

echo 'Installing grub...'
pacman -S --noconfirm --noprogress grub-bios

echo 'Setting up grub...'
grub-install --target=i386-pc --recheck /dev/sda
grub-mkconfig -o /boot/grub/grub.cfg

echo 'Setting some stuff in /etc/hosts.allow...'
echo 'SSHD: ALL' >> /etc/hosts.allow

echo 'Enabling the sshd service...'
systemctl enable sshd.service

echo 'Enabling the dhcpcd service...'
systemctl enable dhcpcd.service

echo 'Enabling the espeakup service...'
systemctl enable espeakup.service

echo 'About to exit chroot...'
exit

echo 'un-mounting hard-disk partitions from virtual mount point...'
 umount /mnt/home
 umount /mnt

EOF

# Mark the created here script executable
chmod +x ${HERE_SCRIPT}


echo 'All done'
exit 0

Installing `yaourt` on Arch Linux

Mike — Sat, 04 Mar 2017 20:05:10 GMT

You can install yaourt from the AUR:

git clone https://aur.archlinux.org/package-query.git
cd package-query
makepkg -si
cd ..
git clone https://aur.archlinux.org/yaourt.git
cd yaourt
makepkg -si
cd ..

Examples

Search and install:

yaourt Sync database, upgrade packages, search aur and devel (all packages based on dev

version) upgrades:

yaourt -Syu --devel --aur Build package from source:
yaourt -Sb Check, edit, merge or remove *.pac* files:
yaourt -C Get a PKGBUILD (support splitted package):
yaourt -G Build and export package, its sources to a directory:
yaourt -Sb --export Backup database:
yaourt -B Query backup file:
yaourt -Q --backupfile

Notes

Yaourt gives an opportunity to edit the PKGBUILD and if the answer is 'y' it asks which editor to use if none is selected by your environment. Type in 'nano'. When getting tclx for use with emacspeak you will need to add armv7 if it is for a Pi2.

Libre Office Writer Shortcut Keys

Mike — Fri, 22 Jul 2016 13:53:53 GMT

Here is a link to the shortcut keys for Libre Office Writer.

There are loads of shortcut keys and the folks at Libre Office have thought with great attention to accessibility.

There is the odd dialog where you may need to use screen review mode and move the mouse cursor and click with the keyboard to get focus into a control, but these instances are few and far between.

Nikola plugin for kramdown

Mike — Thu, 04 Feb 2016 17:44:49 GMT

I’m not surprised that markdown has become so successful and ubiquitous. But one of the things I think it lacks is syntax for creating tables.

For this reason, I have been using a txt2tags plugin for a while. My sites all have a lot of tabular data, both as a function of the nature of the sites and also because I just like tables.

But I recently discovered kramdown.

kramdown has a super-set of markdown syntax, including simple, but powerful table-creation syntax.

So I’ve written a plugin for Nikola along the lines of the txt2tags plugin to compile HTML posts and pages from kramdown files. For which I have started using the .kd extension.

This post is written in kramdown.

So this post is somewhat by way of a test of my kramdown plugin for Nikola.

Third-level Heading

The above heading is marked with two hash characters and in the same way that markdown posts have their heading demoted by one level I have done the same with kramdown.

Here is a table of silly data:

Date	Weather	Activity
01-Feb-2016	Rain	Stayed in
02-Feb-2016	Dry and cloudy	Stayed in again
03-Feb-2016	Cold, -4C	Had to go out for most of the day
04-Feb-2016	Even colder than yesterday	Stayed in and wrote a kramdown plugin for Nikola

Lists

Lists are the same as in markdown

unordered list item one
Unordered list item two

Links

The links in the above text are all done like references are done in markdown, with the actual URI of each link at the bottom of the text file.

Atom.io Editor Totally Inaccessible

Mike — Mon, 23 Nov 2015 13:37:44 GMT

I've just tried the atom.io editor.

I did this because there was to be a talk about it at the Portsmouth LUG, which I went to last Saturday.

It's totally inaccessible on Linux, Debian with the Mate desktop.

Apparently it is an editor written in Javascript and, if I have this correct, it renders the screen with the Chrome browser, or sub-system. There's also something called node.js involved.

These are technologies I know little about, except that it doesn't make a sound with Orca.

So there's another editor we can't use.

Back to good old Emacs.

Transmission Command-line Bit-torrent Programs

Mike — Thu, 20 Aug 2015 11:19:00 GMT

I was after a method of downloading and seeding bit-torrents exclusively from the command-line.

I tried aria2 but found it unreliable. I don't know if it was because I was doing something wrong but none of the torrent files I downloaded were not corrupted in some way.

And anyway I wanted something to run in the background.

I tried transmission-cli. Better results but again this was not something that just ran in the background.

So I installed a pair of related programs: transmission-daemon and transmission-remote.

The Daemon part of this program is exactly that, a daemon that runs, as all daemons do, in the background.

And the Remote part is a command-line program to control it.

The remote part is included in the transmission-cli package.

So:

$ sudo apt-get install transmission-daemon transmission-cli

Will install them both on Debian or Debian derivatives. Although why you're running a derivative when you could run Debian I don't know.

The default installation of transmission-daemon is configured for the user it runs to be set to transmission and the password is also set to transmission.

And by default it is only possible to access the daemon and the management web page from the local machine (localhost).

This led to a tedious situation, any command entered to make transmission-remote do anything with the daemon meant this was the command-line:

$ transmission-remote localhost --auth transmission:transmission <command>

I don't want to have to type that in every time.

So I set up an alias in .bash_aliases:

alias trem='transmission-remote localhost --auth transmission:transmission'

Now I can just type:

$ trem <command>

For example, to add a torrent:

$ trem --add <torrent>

To list all the current torrents and their status:

$ trem --list

To remove and delete a torrent:

$ trem -tN --remove-and-delete

(where N is the number in the list returned by the --list command).

To start a torrent:

$ trem -tN --start

And to stop one:

trem -tN --stop

There are loads of other commands but these are the ones I use the most, and mostly --add and --list.

Note also that transmission will handle the magnet links found on the Pirate Bay download site.

Capturing Regions of the Screen

Mike — Wed, 05 Aug 2015 18:40:12 GMT

Here are some nice keystrokes that work in Debian, and probably other distros as well as these are essentially Gnome actions:

PrintScreen -- Takes a screenshot of your entire desktop and saves it to the Desktop folder
Alt + PrintScreen -- Saves a screenshot of the focused window to the Desktop Folder
Shift + PrintScreen -- Lets you select an area of the screen, and saves to the Desktop Folder
Ctrl + PrintScreen -- Takes a screenshot of your entire desktop and copies it to the clipboard
Ctrl + Alt + PrintScreen -- copies a screenshot of the focused window to the clipboard
Ctrl + Shift + PrintScreen -- Lets you select an area of the screen, and copies it to the clipboard
Ctrl + Shift + Alt + R -- Records a Screencast) of your entire desktop and saves it to your Videos folder

The 'Desktop' folder may be different on other flavours, like 'Pictures' on Fedora.

The commands which allow selection of an area are not much use when you can't see but the others are nice. I particularly like the idea of a screen-cast.

Thanks to the guys on the Surrey Linux User Group email list for these.

Eyes-free Linux Ninja!

Arch Linux on the Raspberry Pi and the bsdtar Fix

Internationalising Shell Scripts with Gettext

Shell Internationalisation Workflow

Preparing a Shell Script for Translation

Extracting Strings into a Portable Object Template (.pot)

The Translation Process

Creating Machine Object Files

Merging New Strings

Installing a Translation

The Worklow in Short

Notes/Suggestions

How to Generate a Table of Contents in a Kramdown Document

Introduction

How To Automatically Generate A Table of Contents

Notes

Tool for Installing Talking Arch

Installing `yaourt` on Arch Linux

Examples

Notes

Libre Office Writer Shortcut Keys

Nikola plugin for kramdown

Third-level Heading

Lists

Links

Atom.io Editor Totally Inaccessible

Transmission Command-line Bit-torrent Programs

Capturing Regions of the Screen

Creating `Machine Object` Files