planet.uknot.org

January 18, 2017

Roger Bell-West

The Last Frontier, Alistair MacLean

1959 thriller. British agent Michael Reynolds travels to Budapest to retrieve a British scientist kidnapped by the Russians and due to be shown off at a conference. US vt The Secret Ways.

January 18, 2017 10:26 AM

January 17, 2017

Roger Bell-West

Colour Scheme, Ngaio Marsh

1943 classic English detective fiction; twelfth of Marsh's novels of Inspector Roderick Alleyn. At a run-down hot-springs resort in the back country of New Zealand, one guest seems to be going out of his way to offend everyone; then he vanishes.

January 17, 2017 09:03 AM

January 16, 2017

Roger Bell-West

Castle season 8

2015-2016, 22 episodes. Famous writer Richard Castle continues to work with NYPD homicide detective (now Captain) Kate Beckett, in the final season of this police procedural.

January 16, 2017 09:03 AM

January 15, 2017

Liam Proven

6music

So... when the lack of apps for my beloved Blackberry Passport, and the issues with running sideloaded Android apps, became problematic, I decided to check out a cheap Chinese Android Phablet.

(P.S. The Passport is for sale! Let me know if you're interested.)

The Passport superseded a Samsung Galaxy Note 2, which subsequently got stolen, unfortunately. It was decent, occasionally sluggish, ran an elderly version of Android with no updates in ages, and had a totally useless stylus I never used. It replaced an iPhone 4 which replaced an HTC Desire HD, which replaced a Nokia Communicator E90 -- the best form-factor for a smartphone I've ever had, but nothing like it exists any more.

I wanted a dual-core or quad-core phablet, bigger than 5.5", with dual SIM and a memory card. That was my starting point.  I don't have or use a tablet and never have -- I'm a keyboard junkie. I spend a lot of time surfing the web, on social networks, reading books and things on my phone. I wanted one as big as I could get, but still pocketable. My nicked Samsing was 5.5" and I wanted a little larger. I tried a 6" phablet in a shop and wanted still bigger if possible. I also tried a 6.8" Lenovo Phab Pro in a shop and that was a bit too big (but I might be persuaded -- with a tiny bezel, such a device might be usable).

I looked at several sites -- the main 2 are:
* Coolicool -- http://coolicool.com/
* GearBest -- http://www.gearbest.com/
I've also been told DX.com is good ( http://www.dx.com/ ) but not tried it myself.

... and Googled in general. I found tons of them. I even started to built a spreadsheet to compare them, there were so many.

So I started looking for things to eliminate certain offerings.

Old models are easy to eliminate. 512MB RAM or something? Too little, strike that line out. I wanted at least 2GB. Tiny amount of Flash? 2, 4, 8GB Flash -- strike that line out.

Then it got a bit harder and needed to pay closer attention to the specs.

  • Dual SIM but one of them is the card slot -- strike that line out.

  • Not an ARM chip (e.g. Intel) -- strike that line out.

  • No removable battery -- strike that line out. (I didn't think to check that spares are available for mine. They aren't. :-( )

  • Low-res screen. At least 720P is normal now. 1280*600 is grainy and low-res these days. Anything less than 720P across (i.e. the narrow axis) got struck out.

I still had more than half a dozen and much confusion.

So then I started looking at Android versions.

Android 4.2 was the oldest I'd consider. It's the last version to play Flash, which is handy, but it's really old now. That had mostly already gone with the models with very low memory or storage.

Android 4.4 is OK, but still old.

So I looked for devices with Android 5.1 (as 5.0 and 5.0.x were buggy and leaked memory).

Bing, everything went away. The iRulu Victory v3 was the only one left, so I bought one.

Basically, now, there are 3 layers in the market.

There's high-end, low-end, and a poorly-defined layer in between.

There are premium big-name devices: Samsung, LG, Sony, HTC, Google Nexus. These are typically poor at what I consider useful extras like dual-SIM, memory card slots, removable batteries. But on the other hand, they abound in very hi-res screens (even with pointless gimmickry like curved screens or bevelled edges), extras like fingerprint readers and NFC and so on that I don't want. Even external expansion connectors like headphone sockets are now disappearing. They are 3x -- 4x the price of low-end devices.

There's the low-end: mostly Chinese, you've never heard of them. Basic but workable. Beware of ancient versions of Android, tiny amounts of memory and Flash, very slow processors or very low-res screens. As of last year, if it's less than £100, it's junk.

But they extend upwards slightly into decent, usable, cheap phones (and tablets, I guess -- I've not looked.)

The high end of the Chinese no-brand generics blurs into the mid-range of smaller brands.

Lenovo sits here, perhaps unexpectedly. Internationally it's a PC company but in its home market it makes hundreds of phones and tablets that non-Chinese markets rarely see. On international sale are about 1% of its models. It has some tempting devices and next time I might go for one, but as ever, they're always compromised -- the memory card is in SIM slot 2, the batteries aren't removable, etc.

Some of the semi-premium brands sit here too -- Huawei, Asus, Xiaomi.

A bit more expensive, but better build quality and support. Software-wise, you might get updates, which is good, but then again, you'll get some horrible customised skin on Android, and preloaded crapware, which is bad.

There are also smaller European companies, making deals with unheard-of Chinese vendors, rebranding them and offering European-customised Android and a bit of after-sales support. If you want to buy in the UK and are scared of dealing directly with China or whatever, this is a good choice.

Main examples:

* WileyFox -- https://www.wileyfox.com/
* OnePlus -- https://oneplus.net/uk/

So, the iRulu Victory v3 I'm currently using...

https://www.irulu.com/phone/p-irulu-victory-v3-smartphone-6.5-inch-4g-lte-unlock-cell-phone-android-5.1-gsm-wcdma-ips-touch-screen-qualcomm-quad-core-gms-certified_1123492.html

I bought it off eBay, new, from Singapore.

I found the best site for checking specs is GSMArena but there are others, e.g.

http://www.devicespecifications.com/en/model/502a3723

It's cheap and plasticky, but it works and it cost £125 new. I spent an extra fiver on a flip case, from a different vendor on eBay. It's one of those where you replace the back cover and the case attaches to that. Another fiver bought a pack of screen protectors. I'll only ever need one, but it was cheaper to buy 5 than 3. I made a balls up of fitting the first one, so I got a shop to do the next, and they did a lovely job.

It's a huge, fairly high-spec phablet for the money -- 2GB RAM, 16GB Flash, both Dual SIM and a µSD slot. (Often makers/vendors claim this but a memory card goes in the 2nd socket. I needed dual SIM plus a memory card.)

The only downside is that there's no compass. Not no compass app -- that's trivial -- no compass chip. Makes navigation on foot trickier. Not a problem in a car but I very rarely need that .

It's huge but it's light as the case and frame aren't metal. As it's not Gorilla Glass or other super-tough screen covering, I regard a screen protector and a hard case as essential. I've dropped it once or twice and it's not cracked or anything yet.

It fits in most trouser pockets fine, even jeans, and it's a talking point and conversation starter because it's so big. But then I have huge hands and can just, barely, use this one-handed.

The battery life is poor, despite an alleged 3000 mAh battery. I carry a power bank. I need to reboot daily, and carefully move all apps that can be moved from internal storage to my memory card.

The iRulu isn't lasting well. I have occasionally errors with the SIM not registering, the backlight sometimes flickers, and the volume-down key no longer works. I have to press volume up and then use a fingertip to move the onscreen slider. However, it remains usable. If it lasts a year, I'll be happy. If it only lasts 9 months, for the money, I won't be gutted. In my experience, premium-brand Android phones are on their last legs after 2 years. If my ChiPhone lasts half as long but costs a quarter of the money, I'm quids in. If it lasts a third as long and costs a quarter, I'm still good. If it lasts a third and costs a third, it's still not bad -- I'm better off if it gets lost or stolen.

A tip of the hat to my mate Louis Dobson ( http://thegentlemansnapper.blogspot.cz/ ) who told me that this was an option right after I bought my Samsung Galaxy Note 2, about 3 years ago. Go adopt one of his dogs.

January 15, 2017 02:36 PM

Roger Bell-West

Psycho, Robert Bloch

1959 horror. Mary Crane has stolen $40,000 from her employer and is running away to get married. Unfortunately she's chosen to stay at the wrong motel.

January 15, 2017 09:03 AM

January 14, 2017

Jonathan McDowell

Cloning a USB LED device

A month or so ago I got involved in a discussion on IRC about notification methods for a headless NAS. One of the options considered was some sort of USB attached LED. DealExtreme had a cheap “Webmail notifier”, which was already supported by mainline kernels as a “Riso Kagaku” device but it had been sold out for some time.

This seemed like a fun problem to solve with a tinyAVR and V-USB. I had my USB relay board so I figured I could use that to at least get some code to the point that the kernel detected it as the right device, and the relay output could be configured as one of the colours to ensure it was being driven in roughly the right manner. The lack of a full lsusb dump (at least when I started out) made things a bit harder, plus the fact that the Riso uses an output report unlike the relay code, which uses a control message. However I had the kernel source for the driver and with a little bit of experimentation had something which would cause the driver to be loaded and the appropriate files in /sys/class/leds/ to be created. The relay was then successfully activated when the red LED was supposed to be on.

hid-led 0003:1294:1320.0001: hidraw0: USB HID v1.01 Device [MAIL  MAIL ] on usb-0000:00:14.0-6.2/input0
hid-led 0003:1294:1320.0001: Riso Kagaku Webmail Notifier initialized

I subsequently ordered some Digispark clones and modified the code to reflect the pins there (my relay board used pins 1+2 for USB, the Digispark uses pins 3+4). I then soldered a tricolour LED to the board, plugged it in and had a clone of the Riso Kaguku device for about £1.50 in parts (no doubt much cheaper in bulk). Very chuffed.

In case it’s useful to someone, the code is released under GPLv3+ and is available at https://the.earth.li/gitweb/?p=riso-kagaku-clone.git;a=summary or on GitHub at https://github.com/u1f35c/riso-kagaku-clone. I’m seeing occasional issues on an older Dell machine that only does USB2 with enumeration, but it generally is fine once it gets over that.

(FWIW, Jon, who started the original discussion, ended up with a BlinkStick Nano which is a neater device with 2 LEDs but still based on an Tiny85.)

January 14, 2017 11:53 AM

Roger Bell-West

Marlow Tabletop and Board Games 5 December 2016

Third time at the Marlow Donkey and the sixth monthly meeting of this Meetup-based boardgames group.

January 14, 2017 09:04 AM

January 13, 2017

Roger Bell-West

Twilight Robbery, Frances Hardinge

2011 non-historical fiction, sequel to Fly By Night. Mosca Mye and Eponymous Clent are still in trouble, and attempting to flee through Toll, the town that controls the only bridge across the big river. But both entering and leaving have their price. US vt Fly Trap.

January 13, 2017 09:03 AM

January 12, 2017

Roger Bell-West

Pyramid 97: Strange Powers

Pyramid, edited by Steven Marsh, is the monthly GURPS supplement containing short articles with a loose linking theme. This time it's the loose idea of bizarre phenomena and, well, strange powers.

January 12, 2017 09:00 AM

January 11, 2017

Roger Bell-West

One Fine Day in the Middle of the Night, Christopher Brookmyre

1999 tartan noir. Gavin Hutchison has a brilliant idea: convert an oil rig to an offshore resort hotel for xenophobic tourists, with all the comforts of home but warmer weather since it'll be moored off the African coast. What better way to show it off than by hosting a school reunion there?

January 11, 2017 09:01 AM

January 10, 2017

Andy Smith (strugglers.net)

XFS, Reflinks and Deduplication

btrfs Past

This post is about XFS but it’s about features that first hit Linux in btrfs, so we need to talk about btrfs for a bit first.

For a long time now, btrfs has had a useful feature called reflinks. Basically this is exposed as cp --reflink=always and takes advantage of extents and copy-on-write in order to do a quick copy of data by merely adding another reference to the extents that the data is currently using, rather than having to read all the data and write it out again, as would be the case in other filesystems.

Here’s an excerpt from the man page for cp:

When –reflink[=always] is specified, perform a lightweight copy, where the data blocks are copied only when modified. If this is not possible the copy fails, or if –reflink=auto is specified, fall back to a standard copy.

Without reflinks a common technique for making a quick copy of a file is the hardlink. Hardlinks have a number of disadvantages though, mainly due to the fact that since there is only one inode all hardlinked copies must have the same metadata (owner, group, permissions, etc.). Software that might modify the files also needs to be aware of hardlinks: naive modification of a hardlinked file modifies all copies of the file.

With reflinks, life becomes much easier:

  • Each copy has its own inode so can have different metadata. Only the data extents are shared.
  • The filesystem ensures that any write causes a copy-on-write, so applications don’t need to do anything special.
  • Space is saved on a per-extent basis so changing one extent still allows all the other extents to remain shared. A change to a hardlinked file requires a new copy of the whole file.

Another feature that extents and copy-on-write allow is block-level out-of-band deduplication.

  • Deduplication – the technique of finding and removing duplicate copies of data.
  • Block-level – operating on the blocks of data on storage, not just whole files.
  • Out-of-band – something that happens only when triggered or scheduled, not automatically as part of the normal operation of the filesystem.

btrfs has an ioctl that a userspace program can use—presumably after finding a sequence of blocks that are identical—to tell the kernel to turn one into a reference to the other, thus saving some space.

It’s necessary that the kernel does it so that any IO that may be going on at the same time that may modify the data can be dealt with. Modifications after the data is reflinked will just case a copy-on-write. If you tried to do it all in a userspace app then you’d risk something else modifying the files at the same time, but by having the kernel do it then in theory it becomes completely safe to do it at any time. The kernel also checks that the sequence of extents really are identical.

In-band deduplication is a feature that’s being worked on in btrfs. It already exists in ZFS though, and there is it rarely recommended for use as it requires a huge amount of memory for keeping hashes of data that has been written. It’s going to be the same story with btrfs, so out-of-band deduplication is still something that will remain useful. And it exists as a feature right now, which is always a bonus.

XFS Future

So what has all this got to do with XFS?

Well, in recognition that there might be more than one Linux filesystem with extents and so that reflinks might be more generally useful, the extent-same ioctl got lifted up to be in the VFS layer of the kernel instead of just in btrfs. And the good news is that XFS recently became able to make use of it.

When I say “recently” I do mean really recently. I mean like kernel release 4.9.1 which came out on 2017-01-04. At the moment it comes with massive EXPERIMENTAL warnings, requires a new filesystem to be created with a special format option, and will need an xfsprogs compiled from recent git in order to have a mkfs.xfs that can create such a filesystem.

So before going further, I’m going to assume you’ve compiled a new enough kernel and booted into it, then compiled up a new enough xfsprogs. Both of these are quite simple things to do, for example the Debian documentation for building kernel packages from upstream code works fine.

XFS Reflink Demo

Make yourself a new filesystem, with the reflink=1 format option.

# mkfs.xfs -L reflinkdemo -m reflink=1 /dev/xvdc
meta-data=/dev/xvdc              isize=512    agcount=4, agsize=3276800 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=0, rmapbt=0, reflink=1
data     =                       bsize=4096   blocks=13107200, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=6400, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Put it in /etc/fstab for convenience, and mount it somewhere.

# echo "LABEL=reflinkdemo /mnt/xfs xfs relatime 0 2" >> /etc/fstab
# mkdir -vp /mnt/xfs
mkdir: created directory ‘/mnt/xfs’
# mount /mnt/xfs
# df -h /mnt/xfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G  339M   50G   1% /mnt/xfs

Create a few files with random data.

# mkdir -vp /mnt/xfs/reflink
mkdir: created directory ‘/mnt/xfs/reflink’
# chown -c andy: /mnt/xfs/reflink
changed ownership of ‘/mnt/xfs/reflink’ from root:root to andy:andy
# exit
$ for i in {1..5}; do
> echo "Writing $i…"; dd if=/dev/urandom of=/mnt/xfs/reflink/$i bs=1M count=1024;
> done
Writing 1…
1024+0 records in 
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.34193 s, 247 MB/s
Writing 2…
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.33207 s, 248 MB/s
Writing 3…
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.33527 s, 248 MB/s
Writing 4…
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.33362 s, 248 MB/s
Writing 5…
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.32859 s, 248 MB/s
$ df -h /mnt/xfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G  5.4G   45G  11% /mnt/xfs
$ du -csh /mnt/xfs
5.0G    /mnt/xfs
5.0G    total

Copy a file and as expected usage will go up by 1GiB. And it will take a little while, even on my nice fast SSDs.

$ time cp -v /mnt/xfs/reflink/{,copy_}1
‘/mnt/xfs/reflink/1’ -> ‘/mnt/xfs/reflink/copy_1’
 
real    0m3.420s
user    0m0.008s
sys     0m0.676s
$ df -h /mnt/xfs; du -csh /mnt/xfs/reflink
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G  6.4G   44G  13% /mnt/xfs
6.0G    /mnt/xfs/reflink
6.0G    total

So what about a reflink copy?

$ time cp -v --reflink=always /mnt/xfs/reflink/{,reflink_}1
‘/mnt/xfs/reflink/1’ -> ‘/mnt/xfs/reflink/reflink_1’
 
real    0m0.003s
user    0m0.000s
sys     0m0.004s
$ df -h /mnt/xfs; du -csh /mnt/xfs/reflink
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G  6.4G   44G  13% /mnt/xfs
7.0G    /mnt/xfs/reflink
7.0G    total

The apparent usage went up by 1GiB but the amount of free space as shown by df stayed the same. No more actual storage was used because the new copy is a reflink. And the copy got done in 4ms as opposed to 3,420ms.

Can we tell more about how these files are laid out? Yes, we can use the filefrag -v command to tell us more.

$ filefrag -v /mnt/xfs/reflink/{,copy_,reflink_}1
Filesystem type is: 58465342
File size of /mnt/xfs/reflink/1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/1: 1 extent found
File size of /mnt/xfs/reflink/copy_1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:     917508..   1179651: 262144:             last,eof
/mnt/xfs/reflink/copy_1: 1 extent found
File size of /mnt/xfs/reflink/reflink_1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/reflink_1: 1 extent found

What we can see here is that all three files are composed of a single extent which is 262,144 4KiB blocks in size, but it also tells us that /mnt/xfs/reflink/1 and /mnt/xfs/reflink/reflink_1 are using the same range of physical blocks: 1572884..1835027.

XFS Deduplication Demo

We’ve demonstrated that you can use cp --reflink=always to take a cheap copy of your data, but what about data that may already be duplicates without your knowledge? Is there any way to take advantage of the extent-same ioctl for deduplication?

There’s a couple of software solutions for out-of-band deduplication in btrfs, but one I know that works also in XFS is duperemove. You will need to use a git checkout of duperemove for this to work.

A quick reminder of the storage use before we start.

$ df -h /mnt/xfs; du -csh /mnt/xfs/reflink
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G  6.4G   44G  13% /mnt/xfs
7.0G    /mnt/xfs/reflink
7.0G    total
$ filefrag -v /mnt/xfs/reflink/{,copy_,reflink_}1
Filesystem type is: 58465342
File size of /mnt/xfs/reflink/1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/1: 1 extent found
File size of /mnt/xfs/reflink/copy_1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:     917508..   1179651: 262144:             last,eof
/mnt/xfs/reflink/copy_1: 1 extent found
File size of /mnt/xfs/reflink/reflink_1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/reflink_1: 1 extent found

Run duperemove.

# duperemove -hdr --hashfile=/var/tmp/dr.hash /mnt/xfs/reflink
Using 128K blocks
Using hash: murmur3
Gathering file list...
Adding files from database for hashing.
Loading only duplicated hashes from hashfile.
Using 2 threads for dedupe phase
Kernel processed data (excludes target files): 4.0G
Comparison of extent info shows a net change in shared extents of: 1.0G
$ df -h /mnt/xfs; du -csh /mnt/xfs/reflink
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G  5.4G   45G  11% /mnt/xfs
7.0G    /mnt/xfs/reflink
7.0G    total
$ filefrag -v /mnt/xfs/reflink/{,copy_,reflink_}1
Filesystem type is: 58465342
File size of /mnt/xfs/reflink/1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/1: 1 extent found
File size of /mnt/xfs/reflink/copy_1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/copy_1: 1 extent found
File size of /mnt/xfs/reflink/reflink_1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/reflink_1: 1 extent found

The output of du remained the same, but df says that there’s now 1GiB more free space, and filefrag confirms that what’s changed is that copy_1 now uses the same extents as 1 and reflink_1. The duplicate data in copy_1 that in theory we did not know was there, has been discovered and safely reference-linked to the extent from 1, saving us 1GiB of storage.

By the way, I told duperemove to use a hash file because otherwise it will keep that in RAM. For the sake of 7 files that won’t matter but it will if I have millions of files so it’s a habit I get into. It uses that hash file to avoid having to repeatedly re-hash files that haven’t changed.

All that has been demonstrated so far though is whole-file deduplication, as copy_1 was just a regular copy of 1. What about when a file is only partially composed of duplicate data? Well okay.

$ cat /mnt/xfs/reflink/{1,2} > /mnt/xfs/reflink/1_2
$ ls -lah /mnt/xfs/reflink/{1,2,1_2}
-rw-r--r-- 1 andy andy 1.0G Jan 10 15:41 /mnt/xfs/reflink/1
-rw-r--r-- 1 andy andy 2.0G Jan 10 16:55 /mnt/xfs/reflink/1_2
-rw-r--r-- 1 andy andy 1.0G Jan 10 15:41 /mnt/xfs/reflink/2
$ df -h /mnt/xfs; du -csh /mnt/xfs/reflink
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G  7.4G   43G  15% /mnt/xfs
9.0G    /mnt/xfs/reflink
9.0G    total
$ filefrag -v /mnt/xfs/reflink/{1,2,1_2}
Filesystem type is: 58465342
File size of /mnt/xfs/reflink/1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/1: 1 extent found
File size of /mnt/xfs/reflink/2 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262127:         20..    262147: 262128:            
   1:   262128..  262143:    2129908..   2129923:     16:     262148: last,eof
/mnt/xfs/reflink/2: 2 extents found
File size of /mnt/xfs/reflink/1_2 is 2147483648 (524288 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262127:     262164..    524291: 262128:            
   1:   262128..  524287:     655380..    917539: 262160:     524292: last,eof
/mnt/xfs/reflink/1_2: 2 extents found

I’ve concatenated 1 and 2 together into a file called 1_2 and as expected, usage goes up by 2GiB. filefrag confirms that the physical extents in 1_2 are new. We should be able to do better because this 1_2 file does not contain any new unique data.

$ duperemove -hdr --hashfile=/var/tmp/dr.hash /mnt/xfs/reflink
Using 128K blocks
Using hash: murmur3
Gathering file list...
Adding files from database for hashing.
Using 2 threads for file hashing phase
Kernel processed data (excludes target files): 4.0G
Comparison of extent info shows a net change in shared extents of: 3.0G
$ df -h /mnt/xfs; du -csh /mnt/xfs/reflink
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G  5.4G   45G  11% /mnt/xfs
9.0G    /mnt/xfs/reflink
9.0G    total

We can. Apparent usage stays at 9GiB but real usage went back to 5.4GiB which is where we were before we created 1_2.

And the physical layout of the files?

$ filefrag -v /mnt/xfs/reflink/{1,2,1_2}
Filesystem type is: 58465342
File size of /mnt/xfs/reflink/1 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             last,shared,eof
/mnt/xfs/reflink/1: 1 extent found
File size of /mnt/xfs/reflink/2 is 1073741824 (262144 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262127:         20..    262147: 262128:             shared
   1:   262128..  262143:    2129908..   2129923:     16:     262148: last,shared,eof
/mnt/xfs/reflink/2: 2 extents found
File size of /mnt/xfs/reflink/1_2 is 2147483648 (524288 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..  262143:    1572884..   1835027: 262144:             shared
   1:   262144..  524271:         20..    262147: 262128:    1835028: shared
   2:   524272..  524287:    2129908..   2129923:     16:     262148: last,shared,eof
/mnt/xfs/reflink/1_2: 3 extents found

It shows that 1_2 is now made up from the same extents as 1 and 2 combined, as expected.

Less of the urandom

These synthetic demonstrations using a handful of 1GiB blobs of data from /dev/urandom are all very well, but what about something a little more like the real world?

Okay well let’s see what happens when I take ~30GiB of backup data created by rsnapshot on another host.

rsnapshot is a backup program which makes heavy use of hardlinks. It runs periodically and compares the previous backup data with the new. If they are identical then instead of storing an identical copy it makes a hardlink. This saves a lot of space but does have a lot of limitations as discussed previously.

This won’t be the best example because in some ways there is expected to be more duplication; this data is composed of multiple backups of the same file trees. But on the other hand there shouldn’t be as much because any truly identical files have already been hardlinked together by rsnapshot. But it is a convenient source of real-world data.

So, starting state:

(I deleted all the reflink files)

$ df -h /mnt/xfs; sudo du -csh /mnt/xfs/rsnapshot
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G   30G   21G  59% /mnt/xfs
29G     /mnt/xfs/rsnapshot
29G     total

A small diversion about how rsnapshot lays out its backups may be useful here. They are stored like this:

  • rsnapshot_root / [iteration a] / [client foo] / [directory structure from client foo]
  • rsnapshot_root / [iteration a] / [client bar] / [directory structure from client bar]
  • rsnapshot_root / [iteration b] / [client foo] / [directory structure from client foo]
  • rsnapshot_root / [iteration b] / [client bar] / [directory structure from client bar]

The iterations are commonly things like daily.0, daily.1daily.6. As a consequence, the paths:

rsnapshot/daily.*/client_foo

would be backups only from host foo, and:

rsnapshot/daily.0/*

would be backups from all hosts but only the most recent daily sync.

Let’s first see what the savings would be like in looking for duplicates in just one client’s backups.

Here’s the backups I have in this blob of data. The names of the clients are completely made up, though they are real backups.

Client Size (MiB)
darbee 14,504
achorn 11,297
spader 2,612
reilly 2,276
chino 2,203
audun 2,184

So let’s try deduplicating all of the biggest one’s—darbee‘s—backups:

$ df -h /mnt/xfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G   30G   21G  59% /mnt/xfs
# time duperemove -hdr --hashfile=/var/tmp/dr.hash /mnt/xfs/rsnapshot/*/darbee
Using 128K blocks
Using hash: murmur3
Gathering file list...
Kernel processed data (excludes target files): 8.8G
Comparison of extent info shows a net change in shared extents of: 6.8G
9.85user 78.70system 3:27.23elapsed 42%CPU (0avgtext+0avgdata 23384maxresident)k
50703656inputs+790184outputs (15major+20912minor)pagefaults 0swaps
$ df -h /mnt/xfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G   25G   26G  50% /mnt/xfs

3m27s of run time, somewhere between 5 and 6.8GiB saved. That’s 35%!

Now to deduplicate the lot.

# time duperemove -hdr --hashfile=/var/tmp/dr.hash /mnt/xfs/rsnapshot
Using 128K blocks
Using hash: murmur3
Gathering file list...
Kernel processed data (excludes target files): 5.4G
Comparison of extent info shows a net change in shared extents of: 3.4G
29.12user 188.08system 5:02.31elapsed 71%CPU (0avgtext+0avgdata 34040maxresident)k
34978360inputs+572128outputs (18major+45094minor)pagefaults 0swaps
$ df -h /mnt/xfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        50G   23G   28G  45% /mnt/xfs

5m02 used this time, and another 2–3.4G saved.

Since the actual deduplication does take some time (the kernel having to read the extents, mainly), and most of it was already done in the first pass, a full pass would more likely take the sum of the times, i.e. more like 8m29s.

Still, a total of about 7GiB was saved which is 23%.

It would be very interesting to try this on one of my much larger backup stores.

Why Not Just Use btrfs?

Using a filesystem that already has all of these features would certainly seem easier, but I personally don’t think btrfs is stable enough yet. I use it at home in a relatively unexciting setup (8 devices, raid1 for data and metadata, no compression or deduplication) and I wish I didn’t. I wouldn’t dream of using it in a production environment yet.

I’m on the btrfs mailing list and there are way too many posts regarding filesystems that give ENOSPC and become unavailable for writes, or systems that were unexpectedly powered off and when powered back on the btrfs filesystem is completely lost.

I expect the reflink feature in XFS to become non-experimental before btrfs is stable enough for production use.

ZFS?

ZFS is great. It doesn’t have out-of-band deduplication or reflinks though, and they don’t plan to any time soon.

by Andy at January 10, 2017 08:45 PM

Roger Bell-West

Horniman Museum, Forest Hill

I'd driven past it many times when living in Lee Green, but had never been inside; over Christmas I remedied this. Images follow: cc-by-sa on everything.

January 10, 2017 09:04 AM

January 09, 2017

Roger Bell-West

January 08, 2017

Roger Bell-West

Death and the Dancing Footman, Ngaio Marsh

1941 (some sources say 1942) classic English detective fiction; eleventh of Marsh's novels of Inspector Roderick Alleyn. For his own amusement, Jonathan Royal invites six mortal enemies to a party at his house in Dorset. Really, the first surprise is that he isn't murdered.

January 08, 2017 09:01 AM

January 07, 2017

Roger Bell-West

Musaigen no Phantom World

2016 contemporary fantasy novel adaptation in 13 episodes: Anidb, vt "Myriad Colors Phantom World". Phantoms, weird magical creatures created by the human subconscious, are everywhere; Haruhiko and his school friends seal them away when they cause trouble.

January 07, 2017 09:00 AM

January 06, 2017

Roger Bell-West

Lost Things, Melissa Scott and Jo Graham

2012 historical fantasy, first in the Order of the Air series. In 1929, as Lake Nemi is drained in search of its archaeological treasures, an ancient evil is loosed. Only four heroic aviator-magicians will be able to fight it.

January 06, 2017 09:02 AM

Jonathan McDowell

2016 in 50 Words

Idea via Roger. Roughly chronological order. Some things were obvious inclusions but it was interesting to go back and look at the year to get to the full 50 words.

Speaking at BelFOSS. Earthlings birthday. ATtiny hacking. Speaking at ISCTSJ. Dublin Anomaly. Co-habiting. DebConf. Peak Lion. Laura’s wedding. Christmas + picnic. Engagement. Car accident. Car write off. Tennent’s Vital. Dissertation. OMGWTFBBQ. BSides. New job. Rachel’s wedding. Digital Privacy talk. Graduation. All The Christmas Dinners. IMDB Top 250. Shay leaving drinks.

(This also serves as a test to see if I’ve correctly updated Planet Debian to use https and my new Hackergotchi that at least looks a bit more like I currently do.)

January 06, 2017 08:03 AM

January 05, 2017

Roger Bell-West

Royal Navy School

2016 reality television, 7 episodes. The series follows recruits through the ten weeks of selection for the Royal Navy at HMS Raleigh.

January 05, 2017 09:04 AM

January 04, 2017

Steve Kennedy

Wicked Cool Shell Scripts

The exact title is "Wicked Cool Shell Scripts, 101 scripts of Linux, OS X and UNIX systems, 2nd Edition" by Dave Taylor and Brandon Perry from No Starch Press.

It's been a while since the first version of the book came out (2004) and a lot has changed in the world of UNIX since that time so though many of the ideas from the first version are still valid, it has been updated to take into account systems like macOS and even Windows.

The book is broken into the following sections

  • Acknowledgments
  • Introduction
  • Chapter 0: A Shell Scripts Crash Course
  • Chapter 1: The Missing Code Library
  • Chapter 2: Improving on User Commands
  • Chapter 3: Creating Utilities
  • Chapter 4: Tweaking Unix
  • Chapter 5: System Administration: Managing Users
  • Chapter 6: System Administration: System Maintenance
  • Chapter 7: Web and Internet Users
  • Chapter 8: Webmaster Hacks
  • Chapter 9: Web and Internet Administration
  • Chapter 10: Internet Server Administration
  • Chapter 11: OS X Scripts
  • Chapter 12: Shell Script Fun and Games
  • Chapter 13: Working with the Cloud
  • Chapter 14: ImageMagick and Working with Graphics Files
  • Chapter 15: Days and Dates
  • Appendix A: Installing Bash on Windows 10
  • Appendix B: Bonus Scripts
  • Index

It's a long book at 305 pages, and it's probably more of a reference book for ideas on shell scripting and what you can do rather than reading it from start to finish - though if you're new to the subject, it's probably worth reading Chapter 0 before moving to other chapters.

If you want to get more out of using the command line (on pretty much any UNIX like OS) this could be the book for you.

It's available for purchase direct from No Starch Press for $34.95 Print Book and FREE Ebook or $27.95 Ebook (PDF, Mobi, and ePub), slightly more through O'Reilly (but you can get it through their Safari service) and considerably cheaper through Amazon.

by Steve Karmeinsky (noreply@blogger.com) at January 04, 2017 05:26 PM

Roger Bell-West

Surfeit of Lampreys, Ngaio Marsh

1940 classic English detective fiction; tenth of Marsh's novels of Inspector Roderick Alleyn. The Lampreys are an impoverished aristocratic family; after Lord Charles's rich brother refused to give them yet another handout, he was found stabbed to death. Mad wife? Annoyed brother? Random stranger? Well, it'll never be that last in a classic mystery. US vt Death of a Peer.

January 04, 2017 09:03 AM

January 03, 2017

Roger Bell-West

Boardgaming in Cambridge, December 2016

With role-playing cancelled due to illness, I played some boardgames in Cambridge instead.

January 03, 2017 09:04 AM

January 02, 2017

Roger Bell-West

Planetfall, Emma Newman

2015 science fiction. Ren works as the 3D printer engineer in a colony on an extrasolar planet, right next to the alien biotech artefact known as God's City. Then a stranger shows up, a descendant of the people thought to have been lost in an accident during planetfall.

January 02, 2017 09:03 AM

January 01, 2017

Roger Bell-West

December 2016 Trailers

Some trailers I've seen recently, and my thoughts on them. (Links are to youtube. Opinions are thoroughly personal.)

January 01, 2017 09:00 AM

December 31, 2016

Mark Goodge

2016 and all that

Bowie_1983_serious_moonlight_blu

It’s been an interesting year. For some, it’s been interesting in the sense of the apocryphal Chinese curse, “may you live in interesting times”. For others, it’s been interesting in a less ironic sense.

Politics and celebrities are two of the staples of the media. So it’s not surprising that big events in both have dominated popular thinking. The unexpected results of both the EU referendum here and the presidential election in the US, combined with a higher than average death toll of genuinely A list musicians, has led many people to think that 2016 has been some kind of apocalyptic nightmare.

In reality, it’s nothing of the sort. Yes, we have lost more leading players from the world of music this year than normal, including one of my own personal favourite artists (David Bowie). But, for all that, it’s within the bounds of statistical variation. And, as has been pointed out elsewhere, much of it is simple demographics. The post-war generation of leading musicians are ageing; we can hardly be shocked when someone such as Leonard Cohen dies at the ripe old age of 86. And those who have had their lives cut unseemingly short have almost all lived self-destructive lives, particularly with drugs and alcohol. A lesson here, maybe, is that youthful excesses have a lasting effect; even if you clean up your life in your middle age the damage is already done.

Outside the shallow world of celebrity and the naval-gazing of politics, though, 2016 has actually been a pretty good year. Globally, the number of people living in absolute poverty continues to fall. And one of the biggest contributors to that is technology: the expansion of low-cost Internet access is one of the biggest contributors to GDP growth in poorer countries.

It’s been good news on the conservation front, too. The Giant Panda is now officially off the endangered list. The number of rhinos poached in South Africa has dropped, and China’s decision to ban trade in ivory has been hailed as a potential game changer for elephants.

Despite continued fighting in Syria and elsewhere in the Middle East, and terror attacks in Brussels and Berlin, the overall number of people killed by terrorism and warfare have declined slightly from 2015 and the long term trend is still very much downwards. It may not seem apparent from our media’s focus on countries nearer to us, but Africa, the Americas and Asia have all seen a marked drop in armed conflict.

If you want more good news stories, then these articles from Quartz and the Guardian are good starting points. For those who like a more data-oriented approach then the aptly-title Our World in Data has some charts you really should be looking at.

But what about Brexit and Trump? Well, to begin with, not everyone thinks those are bad. I’ll go into both of these in a separate article, but wherever you stand on the issue you can’t ignore the fact that a majority of those voting preferred the UK to leave the EU, and even if Trump didn’t get a majority of the popular vote he clearly won the election under the constitutional system which is currently in force. And even if you are on the other side to the winners in both of those cases, there are still reasons to be optimistic. But that’s for another time. Right now, let’s just be pleased at how 2016 has turned out.

Header image credit: Image by Jeffchat1 via Wikimedia Commons. Licensed under the Creative Commons Attribution-Share Alike 3.0 Unported licence. David Bowie performing on the Serious Moonlight Tour, November 193. Which is the tour where I saw him play live, at Milton Keynes Bowl in July 1983. Happy days.

by Mark at December 31, 2016 06:11 PM

Jonathan McDowell

IMDB Top 250: Complete. Sort of.

Back in 2010, inspired by Juliet, I set about doing 101 things in 1001 days. I had various levels of success, but one of the things I did complete was the aim of watching half of the IMDB Top 250. I didn’t stop at that point, but continued to work through it at a much slower pace until I realised that through the Queen’s library I had access to quite a few DVDs of things I was missing, and that it was perfectly possible to complete the list by the end of 2016. So I did.

I should point out that I didn’t set out to watch the list because I’m some massive film buff. It was more a mixture of watching things that I wouldn’t otherwise choose to, and also watching things I knew were providing cultural underpinnings to films I had already watched and enjoyed. That said, people have asked for some sort of write up when I was done. So here are some random observations, which are almost certainly not what they were looking for.

My favourite film is not in the Top 250

First question anyone asks is “What’s your favourite film?”. That depends a lot on what I’m in the mood for really, but fairly consistently my answer is The Hunt for Red October. This has never been in the Top 250 that I’ve noticed. Which either says a lot about my taste in films, or the Top 250, or both. Das Boot was in the list and I would highly recommend it (but then I like all submarine movies it seems).

The Shawshank Redemption is overrated

I can’t recall a time when The Shawshank Redemption was not top of the list. It’s a good film, and I’ve watched it many times, but I don’t think it’s good enough to justify its seemingly unbroken run. I don’t have a suggestion for a replacement, however.

The list is constantly changing

I say I’ve completed the Top 250, but that’s working from a snapshot I took back in 2010. Today the site is telling me I’ve watched 215 of the current list. Last night it was 214 and I haven’t watched anything in between. Some of those are films released since 2010 (in particular new releases often enter high and then fall out of the list over a month or two), but the current list has films as old as 1928 (The Passion of Joan of Arc) that weren’t there back in 2010. So keeping up to date is not simply a matter of watching new releases.

The best way to watch the list is terrestrial TV

There were various methods I used to watch the list. Some I’d seen in the cinema when they came out (or was able to catch that way anyway - the QFT showed Duck Soup, for example). Netflix and Amazon Video had some films, but overall a very disappointing percentage. The QUB Library, as previously mentioned, had a good number of DVDs on the list (especially the older things). I ended up buying a few (Dial M for Murder on 3D Bluray was well worth it; it’s beautifully shot and unobtrusively 3D), borrowed a few from friends and ended up finishing off the list by a Lovefilm one month free trial. The single best source, however, was UK terrestrial TV. Over the past 6 years Freeview (the free-to-air service here) had the highest percentage of the list available. Of course this requires some degree of organisation to make sure you don’t miss things.

Films I enjoyed

Not necessarily my favourite, but things I wouldn’t have necessarily watched and was pleasantly surprised by. No particular order, and I’m leaving out a lot of films I really enjoyed but would have got around to watching anyway.

  • Clint Eastwood films - Gran Torino and Million Dollar Baby were both excellent but neither would have appealed to me at first glance. I hated Unforgiven though.
  • Jimmy Stewart. I’m not a fan of It’s a Wonderful Life (which I’d already watched because it’s Lister’s favourite film), but Harvey is obviously the basis of lots of imaginary friend movies and Rear Window explained a Simpsons episode (there were a lot of Simpsons episodes explained by watching the list).
  • Spaghetti Westerns. I wouldn’t have thought they were my thing, but I really enjoyed the Sergio Leone films (A Fistful of Dollars etc.). You can see where Tarantino gets a lot of his inspiration.
  • Foreign language films. I wouldn’t normally seek these out. And in general it seems I cannot get on with Italian films (except Life is Beautiful), but Amores Perros, Amelie and Ikiru were all better than expected.
  • Kind Hearts and Coronets. For some reason I didn’t watch this until almost the end; I think the title always put me off. Turned out to be very enjoyable.

Films I didn’t enjoy

I’m sure these mark me out as not being a film buff, but there are various things I would have turned off if I’d caught them by accident rather than setting out to watch them.

I’ve kept the full list available, if you’re curious.

December 31, 2016 04:01 PM

Roger Bell-West

Roger's 2016 in 50 Words

A friend likes to sum up his year in a set number of words, and I copy this fine idea. "Think of it as a short and un-boastful summary of the year, which nobody is expected to understand all of."

December 31, 2016 09:04 AM

December 30, 2016

Jess Rowbottom

An Interesting 2016

The year didn’t quite go to plan and I’m not where I thought I’d be – a mix of reasons, but mostly history dragging its heels. For many of us, triggered by the political and social landscape changes, the 24-hour rolling news, and all this “post-truth” stuff, it’s easy to get mired down in negativity. However there is a need for positive thinking, so I wrote down a big pile of awesome things which happened to me in 2016.

While in a café in Leeds on Christmas Eve having a cuppa with my kids, I thought of the following:

  • Met my soulmate
  • Swam in the sea, in a bikini
  • Spoke at a European Commission meeting in Brussels
  • Learned to play guitar properly
  • First international travel on my new passport
  • Dated for the first time in 20 years
  • Took the whole family to a festival
  • Recorded and released an album
  • Went camping as a family
  • Visited Sweden for the Eurovision Song Contest
  • Taught myself to shoot and edit video
  • Learned what “squad goals” means
  • Performed on-stage with my daughter
  • Went up 2 cup sizes
  • Danced with one of my two favourite bands
  • Got blocked by the President-elect of the US
  • Wrote, arranged, scored and conducted a piece for orchestra
  • Was sent a holiday postcard by my other favourite band
  • Guest-blogged for a sex toy company
  • Got told off by translators because of my Yorkshireisms
  • Walked in my underwear through a field of lesbians
  • Met a load of fun new friends
  • Snogged my girl in front of protesting Christian fundamentalists
  • Rid myself of a lot of toxic people
  • Weight down below 80kg for the first time in 3 years

 

…which I don’t think is a bad haul! They’re listed in no particular order, by the way.

What was I most proud of? Well, obviously the album as it spanned 11 months of the year. There have been so, so many kind words about it from friends, relatives, acquaintances I know and people I don’t. I was randomly approached and asked if I’m Jess of The Bleeding Obvious, and at the launch I performed with many talented friends on-stage. I’m hoping for more of that.

Singers at the Bleeding Obvious album launch, from L-R: Jacqui Wicks, Colleen Taylor, Irene Purcell, Ralph Dartford, Anthony Jackson-Stubbs, Jessica Rowbottom, Betsie Baker, Scott Wainwright, Ellie Rowbottom and Ruby Macintosh. Photo: Cathie Heart

I’m also hoping for more family time with us all together – some of my fondest memories of the year are of camping at L-Fest Lesbian Festival last July (the header photo on this post is from the sign outside our tent), and of course having all the kids in one place on Christmas Day was wonderful too.

So where do I go from here?

Well I’m heading into 2017 with a firm resolution to be optimistic, which is why the second release as The Bleeding Obvious is going to be titled “Positive”. There’ll be more about that later in the year (hah, just you try and avoid it) but in the meantime I’ve been working with Ralph Dartford of A Firm Of Poets, illustrating some of his words with music (you can already listen to Olé and Bob over on Soundcloud). There’ll be some festival appearances I’m sure.

Finally, my partner Helen proposed to me on Christmas Day… I’ll leave it as a cliffhanger as to what my reply was…

by Jess at December 30, 2016 09:54 AM

Roger Bell-West

2016 in Books

In 2016 I've read 133 books, down a little from recent years.

December 30, 2016 09:04 AM

December 29, 2016

Roger Bell-West

2016 in GURPS

The GURPSDay bloggers suggested a series of questions about the year in GURPS, so…

December 29, 2016 09:01 AM

Denesh Bhabuta

Cuida bem de ti

Intercontinental, New York, USA          28 December, 2016          11.20pm ET

“Trust in the intuition that your being manifests.
Believe in the potential, in the seed that lives in your heart and gives birth to flowers of wisdom.”

Cuida bem de ti

Há que ter confiança naquilo que vives.

Confia na intuição que o teu ser manifesta.
Acredita no potencial, na semente que vive em teu coração e dela faz nascer flores de sabedoria.

Por vezes deixas de viver tantas coisas pelo simples fato de acreditares que não és capaz de transformar a tua conduta, os teus sentimentos, a tua abertura para com a vida e assim, as infinitas possibilidades que te são dadas não podem alcançar o teu coração.

Tua capacidade provém da tua fé, por isso deixa brotar no teu solo aquilo que desejas viver
e rega com a tua dedicação, com a tua alegria,
com a tua determinação e não tardará o momento onde poderás ver o que a vida reserva para ti.

Sê atento para contigo e verás que, dia a dia,
as oportunidades virão e com elas
o aprendizado que necessitas viver.

Há que ter gentileza para com as coisas
do teu coração, há que ter tamanha compaixão
e dar-te o perdão momento a momento.

É de vital importância lembrar-te quão precioso és.

Portanto, cuida bem de ti para que o teu ser possa se manifestar com toda a sua luminosidade.

Lembra que ao acender uma vela iluminas não só a tua casa mas o caminho daquele que está por vir.

Cris Couto

by Admin at December 29, 2016 04:29 AM

December 28, 2016

Roger Bell-West

2016 in boardgames

2016 was a very boardgame-ful year. I didn't plan it that way, but I'm not objecting.

December 28, 2016 09:04 AM

December 27, 2016

Roger Bell-West

Akagami no Shirayukihime season 2

2016 fantasy romance, shoujo manga continuation in 12 episodes as well as an OVA that bridges the two series: AniDB, vt "Snow White with the Red Hair". Shirayuki the herbalist continues to develop her low-key romance with Prince Zen.

December 27, 2016 09:00 AM

December 26, 2016

Roger Bell-West

Dying to Write, Judith Cutler

1996 mystery; second of Cutler's novels of Sophie Rivers, teacher in a sixth-form college in Birmingham and part-time chorister. Sophie goes on a residential writing course; one of her fellow students dies, and a tutor goes missing.

December 26, 2016 09:00 AM

December 25, 2016

Roger Bell-West

The Box of Splendor and the Cosmic Pot

I continue to design boardgame accessories for 3D printing; these two are for Splendor and Cosmic Encounter. Images are cc-by-sa by RogerBW unless otherwise specified.

December 25, 2016 09:00 AM

Jonathan McDowell

The terrible PIC ecosystem

I recently had call to play with some 1-Wire devices at work (more of which in a future post). It was taking a while for the appropriate programmer to turn up, so of course I pulled out my trusty BusPirate. It turned out the devices in question would only talk in overdrive mode, while the Bus Pirate could only offer standard mode. So I set about trying to figure out how to add the appropriate support.

This is is a huge endorsement for test equipment with Free Software firmware. Rather than giving up I was able to go and grab the current firmware, which has been adopted by the community since Dangerous Prototypes have discontinued development. What let me down was the ecosystem around the PIC24FJ64GA002.

My previous recent experience with microcontrollers has been with the ATTiny range and the STM32. Getting up and running with both of these was fairly easy - the tool chains necessary were already present in Debian, so all it took was a simple apt invocation to install everything I needed to compile code and program it to the devices.

Not so with the PIC series, which surprised me. There seems to be some basic support for the earlier PIC16 range, but for later chips there’s nothing that works out of the box with Debian. Investigation revealed that this was because there’s nothing maintained that enabled Free development for the PIC range. The accepted solution is the closed MPLAB X. Now, in one sense fair play to Microchip for making this available. But in another, shame on you. I can’t imagine ever choosing to build something based on a chip that only had a closed source tool chain available. I want things I can use in Makefiles and properly script, that are available in my distro of choice and that generally work in the same fashion as the tool chains I’m used to. I understand there might be some benefit in a closed compiler in terms of performance (and have HPC friends who would never trust a benchmark provided using GCC), but in general that’s not the space I move in. Nor does it seem to be the sort of attitude you should be taking if you are trying to attract the hobbyist and small production run market.

Any yet this seems common amongst hardware manufacturers. People whose core business is selling physical items, where the software is only relevant in terms of being able to use those items, seem to consider the software to be precious. Instead of opening up programming specifications and allowing a more widespread use of the hardware, increasing sales. I understand there are some cases where this isn’t practical, but the default attitude is definitely one of being closed rather than open, which is a terrible shame.

Anyway. I do have some Bus Pirate 1-Wire overdrive support now working (pending some testing to ensure standard mode still works), but I am glad I never spent a lot of time getting involved with PICs now.

December 25, 2016 12:51 AM

December 24, 2016

Roger Bell-West

Must Love Board Games

2016 short romantic comedy, dir. Travis Hedges Williams, Kacie Barnes, Tyler Grezaffi: IMDb

When Quinn meets David, she has to decide whether he is a worthy suitor: she knows the rules, just as she knows the rules of her favourite games. But he doesn't seem to be following them.

December 24, 2016 09:02 AM

December 23, 2016

Roger Bell-West

Flash Point solo: Well-Oiled Machine

Flash Point is another game I don't get to the table often enough for my taste, so I gave it a solo play one evening. Lots of images (cc-by-sa).

December 23, 2016 09:03 AM

December 22, 2016

Roger Bell-West

Heads You Lose, Christianna Brand

1941 detective fiction; second of Brand's novels, and first to feature Inspector Cockrill. A visitor to a country house says "I wouldn't be seen dead in a ditch in a [hat] like that"… and is soon proved wrong. The next night another woman is murdered.

December 22, 2016 09:04 AM

December 21, 2016

Roger Bell-West

Dimension W

2016 science fiction, 12 episodes: AniDB. In a world where all power is provided by "coils" that draw from a parallel dimension, Mabuchi Kyouma is a "collector" who gets hold of the illegal ones.

December 21, 2016 09:03 AM

December 20, 2016

Roger Bell-West

Rizzoli & Isles season 6

2015-2016, 18 episodes. Detective Jane Rizzoli and medical examiner Maura Isles continue in their odd-couple crimefighting.

December 20, 2016 09:03 AM

December 19, 2016

Roger Bell-West

The Mill on the Shore, Ann Cleeves

1994 detective fiction; seventh of Cleeves's novels of amateur private detectives George and Molly Palmer-Jones. Jimmy Morrissey was the public voice of environmentalism and conservation, but killed himself with an overdose of antidepressants. Or did he?

December 19, 2016 09:01 AM

December 18, 2016

Jonathan McDowell

Timezones + static blog generation

So, it turns out when you move to static blog generation and do the generation on your laptop, which is usually in the timezone you’re currently physically located, it can cause URLs to change. Especially if you’re prone to blogging late at night, which can result in even just a shift to DST changing things. I’ve forced jekyll to UTC by adding timezone: 'UTC' to the config, and ensuring all the posts now have timezones for when they were written (a lot of the imported ones didn’t), so hopefully things should be stable from here on.

December 18, 2016 11:28 PM

Roger Bell-West

Pyramid 96: Tech and Toys IV

Pyramid, edited by Steven Marsh, is the monthly GURPS supplement containing short articles with a loose linking theme. This time it's high technology and the things one can do with it.

December 18, 2016 09:00 AM

December 17, 2016

Roger Bell-West

God Emperor of Didcot, Toby Frost

2008 humorous science fiction, second in the series. Captain Isambard Smith of the British Empire in space is sent to the Didcot system, where a revolutionary movement threatens the Empire's supply of tea.

December 17, 2016 09:04 AM

December 16, 2016

Roger Bell-West

Leaving Earth Outer Planets solo: Venus is Red, Comrade

I know, I know. I got my copy of Leaving Earth: Outer Planets back in July, and I haven't had it to the table until now. I was hoping to get some suckers to play it with me, but instead I ran a solo game on a grey Sunday afternoon. (I haven't played any version of Leaving Earth solo before.)

Images were taken on the Powershot A1300 in "custom white balance" mode, under halogen incandescents; I thought it did quite a decent job. As usual, click on images to get the bigger version; cc-by-sa on everything.

December 16, 2016 09:02 AM

December 15, 2016

Roger Bell-West

Death in High Heels, Christianna Brand

1941 detective fiction; first of Brand's novels, and first of two featuring Inspector Charlesworth. One of the senior staff of a London dress-shop dies of poison; her co-workers are the only plausible suspects.

December 15, 2016 09:00 AM

December 14, 2016

Roger Bell-West

Bubuki Buranki 1

2016 mecha science fiction, 12 episodes: AniDB. In a very changed world, Kazuki Azuma leaves an island in the sky to return to Japan, and gets involved with power struggles and giant robots.

December 14, 2016 09:04 AM

December 13, 2016

Roger Bell-West

Spin and Die, Stella Whitelaw

2000 mystery, third in Whitelaw's Jordan Lacey series. Jordan takes on a job following a woman suspected of compensation fraud against the local department store, and then tries to track down vanishing stock in the run-up to Christmas. And of course there's also a murder.

December 13, 2016 09:03 AM

December 12, 2016

Jonathan McDowell

No longer a student. Again.

99 Problems

(image courtesy of XKCD)

Last week I graduated with a Masters in Legal Science (now taught as an MLaw) from Queen’s University Belfast. I’m pleased to have achieved a Distinction, as well an award for Outstanding Achievement in the Dissertation (which was on the infringement of privacy by private organisations due to state mandated surveillance and retention laws - pretty topical given the unfortunate introduction of the Investigatory Powers Act 2016). However, as previously stated, I had made the decision that I was happier building things, and wanted to return to the world of technology. I talked to a bunch of interesting options, got to various stages in the hiring process with each of them, and happily accepted a role with Titan IC Systems which started at the beginning of September.

Titan have produced a hardware accelerated regular expression processor (hence the XKCD reference); the RXP in its FPGA variant (what I get to play with) can handle pattern matching against 40Gb/s of traffic. Which is kinda interesting, as it lends itself to a whole range of applications from network scanning to data mining to, well, anything where you want to sift through a large amount of data checking against a large number of rules. However it’s brand new technology for me to get up to speed with (plus getting back into a regular working pattern rather than academentia), and the combination of that and spending most of the summer post DebConf wrapping up the dissertation has meant I haven’t had as much time to devote other things as I’d have liked. However I’ve a few side projects at various stages of completion and will try to manage more regular updates.

December 12, 2016 10:27 PM

Roger Bell-West

Boardgaming At Home, November 2016

A seven-player games session where we actually stuck with a single group of seven all the way through. Images follow: cc-by-sa on everything.

December 12, 2016 09:02 AM

December 11, 2016

Roger Bell-West

Traitor's Purse, Margery Allingham

1940 classic English detective fiction; eleventh of Allingham's novels of Albert Campion. An amnesiac Campion knows he was in the middle of dealing with a truly serious plot… but what is it all about, and whom can he trust? US vt The Sabotage Murder Mystery.

December 11, 2016 09:02 AM

December 10, 2016

Roger Bell-West

Marlow Tabletop and Board Games 7 November 2016

Second time at the Marlow Donkey and the fifth monthly meeting of this Meetup-based boardgames group.

December 10, 2016 09:03 AM

December 09, 2016

Roger Bell-West

There's Something In a Sunday, Marcia Muller

1990 mystery; ninth in Muller's series about Sharon McCone, private investigator in San Francisco. Sharon's assigned to follow Frank Wilkinson as he spends his Sunday going round the horticultural high spots of San Francisco. But her client's obviously lying about why he wants the job done, and soon he ends up dead.

December 09, 2016 09:04 AM

December 08, 2016

Roger Bell-West

Ojisan to Marshmallow

2016 contemporary comedy, 12 short episodes: AniDB, vt "Middle-Aged Man and Marshmallow". Hige is an overweight middle-aged man working at Web Related Company (sic); he loves marshmallows. Several of the office ladies are in love with him, for no obvious reason.

December 08, 2016 09:04 AM

December 07, 2016

Roger Bell-West

Death at the Bar, Ngaio Marsh

1940 classic English detective fiction; ninth of Marsh's novels of Inspector Roderick Alleyn. In an isolated village in Devon, a game of darts at the pub ends in death. Accidental? Surely not.

December 07, 2016 09:03 AM

December 06, 2016

Roger Bell-West

Leaving Earth

Leaving Earth, designed by Joe Fatula, is a game of the exploration of the solar system for 1-5 players.

December 06, 2016 09:00 AM

December 05, 2016

Steve Kennedy

Canon EOS 5D Mark III - The Guide to Understanding and Using Your Camera

This post should have been published a LONG time ago.

O'Reilly are known for their computing books, but they also do technology on other subjects through Rocky Nook. Here's a review on the Canon EOS 5D Mark III - The Guide to Understanding and Using Your Camera from Paul Clark (a professional photographer).

Having worked through the EOS range from 40D, 50D, 5D2 and now to the 5D3, this book has unlocked many things I thought I already knew about Canon digital cameras, even before we get to the new features of the 5D3. It's usefully laid out - in part, going through each menu feature much like the manual does, but adding a lot of When and Why to the How of the manual's basic descriptions. For the first time I really understood what the stopping down preview was all about, and some of the Live View functions. As well as the feature dissection, the book also goes into great detail on some of the really complicated areas that benefit from a chapter to themselves. Focus, and in particular the 5D3's sophisticated autofocus settings are really well explained. I suspect that the actual range of functions is so vast and complex to set up that in the field one would hardly ever have time to do more than a couple of familiar settings, but it's nice to know what's on offer anyway.

The illustrations are clear and plentiful, and the text very readable. I'm a big fan of the style, but if I could pick out one area for improvement it would be a stronger steer towards "what really works" for different shooting scenarios - e.g. which of the many options on focus point selection might work best for a particular settings, or what three custom settings should be the shooter's priority when preparing. There are a few of these "opinions and tips", such as whether the Rate button really offers any value, but there's always room for more in a book like this.

Overall: excellent.

Next time publishing will be in a more timely manner.

by Steve Karmeinsky (noreply@blogger.com) at December 05, 2016 03:27 PM

Roger Bell-West

3d printer notification system

How can I tell when the 3D printer has finished a job, without going down to the cave where it lives and checking it?

December 05, 2016 09:01 AM

December 04, 2016

Roger Bell-West

The Praxis, Walter Jon Williams

2003 space opera, first book of Dread Empire's Fall. The Shaa subjugated the galaxy, binding all the races they met – including humanity – under their universal philosophy, the Praxis. But ten thousand years later, the last Shaa has chosen to die.

December 04, 2016 09:03 AM

December 03, 2016

Roger Bell-West

Boku dake ga Inai Machi

2016 seinen manga adaptation, 12 episodes: AniDB, vtt "Erased" and "The Town Without Me". Fujinuma Satoru sometimes slips back in time to just before a life-threatening accident, which lets him try to correct it. When his mother is murdered, he finds himself thrown back to eighteen years earlier, when he was in elementary school and someone was killing his classmates.

December 03, 2016 09:03 AM