Planet Uknot

September 24, 2017

Roger Bell_West

Marlow Tabletop and Board Games 18 September 2017

This Meetup-based boardgames group continues to meet at the Marlow Donkey.

September 24, 2017 08:04 AM

September 23, 2017

Andy Smith (strugglers.net)

Giving Cinema Paradiso a try

Farewell, LoveFiLM

I’ve been a customer of LoveFiLM for something like 12 years—since before they were owned by Amazon. In their original incarnation they were great: very cheap, and titles very often arrived in exactly the order you specified, i.e. they often managed to send the thing from the very top of the list.

In 2011 they got bought by Amazon and I was initially a bit concerned, but to be honest Amazon have run it well. The single list disappeared and was replaced by three priority lists; high, normal and low, and then a list of things that haven’t yet been released. New rentals were supposed to almost always come from the high priority list (as long as you had enough titles on there) but in a completely unpredictable order. Though of course they would keep multi-disc box sets together, and send lower-numbered seasons before later seasons.

Amazon have now announced that they’re shutting LoveFiLM by Post down at the end of October which I think is a shame, as it was a service I still enjoy.

It was inevitable I suppose due to the increasing popularity of streaming and downloads, and although I’m perfectly able to do the streaming and download thing, receiving discs by post still works for me.

I am used to receiving mockery for consuming some of my entertainment on little plastic discs that a human being has to physically transport to my residence, but LoveFiLM’s service was still cheap, the selection was very good, things could be rented as soon as they were available on disc, and the passive nature of just making a list and having the things sent to me worked well for me.

Cinema Paradiso

My first thought was that that was it for the disc-by-post rental model in the UK. That progress had left it behind. But very quickly people pointed me to Cinema Paradiso. After a quick look around I’ve decided to give it a try and so here are my initial thoughts.

Pricing

At a casual glance the pricing is slightly worse than LoveFiLM’s. I was paying £6.99 a month for 2 discs at home, unlimited rental per month. £6.98 at Cinema Paradiso gets you 2 discs at home but only 4 rentals per month.

I went back through my LoveFiLM rental history for the last year and found there were only 2 months where I managed to rent more than 4 discs, and those times I rented 5 and 6 discs respectively. Realistically it doesn’t seem like 4 discs per month will be much of a restriction to me.

Annoyingly, Cinema Paradiso have a 2 week trial period but only if you sign up to the £9.98 subscription (6 discs a month). You’d have to remember to downgrade to the cheaper subscriptions after 2 weeks, if that’s all you wanted.

Selection

I was pleasantly surprised at how good the selection is at Cinema Paradiso. Not only did they have every title that is currently on my LoveFiLM rental list (96 titles), but they also had a few things that LoveFiLM thinks haven’t been released yet.

I’m not going to claim that my tastes are particularly niche, but there are a few foreign language films and some anime in there, and release dates range from the 70s to 2017.

Manual approval

It seems that new Cinema Paradiso signups need to be manually approved, and this happens only on week days between 8am and mid day. I’ve signed up on a Saturday evening so nothing will get sent out until Monday I suppose.

It’s probably not a big deal as we’re talking about the postal service here so even with LoveFiLM nothing would get posted out until Monday anyway. It is a little jarring after moving away from the behemoth that is Amazon though, and serves as a reminder that Cinema Paradiso is a much smaller company.

Searching for titles

The search feature is okay. It provides suggestions as you type but if your title is obscure then it may not appear in the list of suggestions at all you and need to submit the search box and look through the longer list that appears.

A slight niggle is that if you have moused over any of the initial suggestions it replaces your text with that, so if your title isn’t amongst the suggestions you now have to re-type it.

I like that it shows a rating from Rotten Tomatoes as well as from their own site’s users. LoveFiLM shows IMDB ratings which I don’t trust very much, and also Amazon ratings, which I don’t trust at all for movies or TV. Seeing some of the shockingly-low Rotten Tomatoes scores for some of my LoveFiLM titles resulted in my Cinema Paradiso list shrinking to 83 titles!

Rental list mechanics

It’s hard to tell for sure at this stage because I haven’t yet got my account approved and had any rentals, but it looks to me like the rental list mechanics are a bit clunky compared to LoveFiLM’s.

At LoveFiLM at the point of adding a new title you would choose which of the three “buckets” to put a rental in; high priority, normal priority, or low priority. Every title in those buckets were of equal priority to every other item in the same bucket. So, when adding a new title all you had to consider was whether it was high, medium or low.

Cinema Paradiso has a single big list of rentals. In some ways this might appeal because you can fine-tune what order you would like things in. But I would suggest that very few people want to put that much effort into ordering their list. Personally, when I add a new title I can cope with:

  • “I want to see this soon”
  • “I want to see this some time”
  • “I want to see this, but I’m not bothered when”

Cinema Paradiso appears to want me to say:

  • “Put this at the top, I want it immediately!”
  • “This belongs at #11, just after the 6th season of American Horror Story, but before Capitalism: A love Story
  • “Just stick it at the end”

I can’t find any explanation anywhere on their site as to how the selection actually works, so the logical assumption is that they go down your list from top to bottom until they find a title that you want that they have available right now. Without the three buckets to put titles in, it seems to me then that every addition will have to involve some list management unless I either want to see that title really soon, or probably never.

I’ll have to give it a go but this mechanism seems a bit more awkward than LoveFiLM’s approach and needlessly so, because LoveFiLM’s way doesn’t make any promises about which of the titles in each bucket will come next either, nor even that it will be anything from the high priority bucket at all. Although I cannot remember a time when something has come that wasn’t from the high priority bucket.

Cinema Paradiso does let you have more than one list, and you can divide your disc allocation between lists, but I don’t think I could emulate the high/normal/low with that. Having a 2 disc allocation I’d always be getting one disc from the “high” list and one disc from the “normal” priority, which isn’t how I’d want that to work.

Let’s see how it goes.

Referral

I did not know when I signed up that there was a referral scheme which is a shame because I do know some people already using Cinema Paradiso. If you’re going to sign up then please use my referral link. I will get a ⅙ reduction in rental fees for each person that does.

by Andy at September 23, 2017 11:38 PM

Roger Bell_West

The Devil You Know, K J Parker

2016 fantasy novella. The greatest philosopher of all time is offering to sell his soul to the Devil. All he wants is twenty more years to complete his life’s work. But the demon assigned to the task is deeply suspicious. For one thing, it seems like an obviously bad deal for the human. For another, the philosopher asked for him by name.

September 23, 2017 08:01 AM

Zoe O'Connell

In response to Janice Turner: An unpublished letter to The Times

Last Saturday, The Times published an opinion piece by Janice Turner in which she tells a version of events that took place at Speakers’ Corner last week during a protest by trans activists. By the time of publication, Janice’s narrative of an elderly woman being beaten up had already been proven false by video circulating on YouTube. This is my letter to the editor in response to that piece, sent on Saturday afternoon – The Times have chosen not to publish it.

Dear Editor,

I am writing in response to Janice Turner’s article “The battle over gender has turned bloody”.

Janice seems to be unaware that the incident which occurred during a protest last week was videoed and that it was posted on YouTube. The video tells a very different story to the one she presents, in which she claims a trans activist committed an unprovoked assault on a 60 year old woman. Or perhaps she has taken a leaf out of Donald Trump’s campaign playbook, and wants to try to establish her view as the pure and unadulterated truth regardless of the evidence to the contrary.

What the video shows is Janice’s “60-year-old in specs and sensible shoes called Maria”, who she clearly want to portray as someone defenceless, holding a trans activist in a headlock and trying to kick them repeatedly. I understand the police were called, viewed the video and concluded no action was needed because Maria’s injuries had been sustained as a result of her being pulled off by one of the activist’s friends.

Although stills are available, the video has since been taken offline. Presumably because the person who posted it realised that crying foul when you sustain injuries in the process of assaulting someone else is not a good PR tactic.

I condemn all violence. If Janice wants to condemn violence, she too should condemn all violence. Not just those incidents that help prop up her narrative of hate.

Yours,

Councillor Zoe O’Connell

The post In response to Janice Turner: An unpublished letter to The Times appeared first on Complicity.

by Zoe O'Connell at September 23, 2017 08:00 AM

September 22, 2017

Roger Bell_West

Food pills and energy density

In Marty Jopson's new book The Science of Food, he demolishes the idea of food pills by looking at the energy density of fat and working out the mass of fat-pills one would need to eat. But why would one restrict oneself to the energy that an unmodified human body can get from food?

September 22, 2017 08:03 AM

September 21, 2017

Andy Smith (strugglers.net)

Tricky issues when upgrading to the GoCardless “Pro” API

Background

Since 2012 BitFolk has been using GoCardless as a Direct Debit payment provider. On the whole it has been a pleasant experience:

  • Their API is a pleasure to integrate against, having excellent documentation
  • Their support is responsive and knowledgeable
  • Really good sandbox environment with plenty of testing tools
  • The fees, being 1% capped at £2.00, are pretty good for any kind of payment provider (much less than PayPal, Stripe, etc.)

Of course, if I was submitting Direct Debits myself there would be no charge at all, but BitFolk is too small and my bank (Barclays) are not interested in talking to me about that.

The “Pro” API

In September 2014 GoCardless came out with a new version of their API called the “Pro API”. It made a few things nicer but didn’t come with any real new features applicable to BitFolk, and also added a minimum fee of £0.20.

The original API I’d integrated against has a 1% fee capped at £2.00, and as BitFolk’s smallest plan is £10.79 including VAT the fee would generally be £0.11. Having a £0.20 fee on these payments would represent nearly a doubling of fees for many of my payments.

So, no compelling reason to use the Pro API.

Over the years, GoCardless made more noise about their Pro API and started calling their original API the “legacy API”. I could see the way things were going. Sure enough, eventually they announced that the legacy API would be disabled on 31 October 2017. No choice but to move to the Pro API now.

Payment caps

There aren’t normally any limits on Direct Debit payments. When you let your energy supplier or council or whatever do a Direct Debit, they can empty your bank account if they like.

The Direct Debit Guarantee has very strong provisions in it for protecting the payee and essentially if you dispute anything, any time, you get your money back without question and the supplier has to pursue you for the money by other means if they still think the charge was correct. A company that repeatedly gets Direct Debit chargebacks is going to be kicked off the service by their bank or payment provider.

The original GoCardless API had the ability to set caps on the mandate which would be enforced their side. A simple “X amount per Y time period”. I thought that this would provide some comfort to customers who may not be otherwise familiar with authorising Direct Debits from small companies like BitFolk, so I made use of that feature by default.

This turned out to be a bad decision.

The main problem with this was that there was no way to change the cap. If a customer upgraded their service then I’d have to cancel their Direct Debit mandate and ask them to authorise a new one because it would cease being possible to charge them the correct amount. Authorising a new mandate was not difficult—about the same amount of work as making any sort of online payment—but asking people to do things is always a pain point.

There was a long-standing feature request with GoCardless to implement some sort of “follow this link to authorise the change” feature, but it never happened.

Payment caps and the new API

The Pro API does not support mandates with a capped amount per interval. Given that I’d already established that it was a mistake to do that, I wasn’t too bothered about that.

I’ve since discovered however that the Pro API not only does not support setting the caps, it does not have any way to query them either. This is bad because I need to use the Pro API with mandates that were created in the legacy API. And all of those have caps.

Here’s the flow I had using the legacy API.

Legacy payment process

This way if the charge was coming a little too early, I could give some latitude and let it wait a couple of days until it could be charged. I’d also know if the problem was that the cap was too low. In that case there would be no choice but to cancel the customer’s mandate and ask them to authorise another one, but at least I would know exactly what the problem was.

With the Pro API, there is no way to check timings and charge caps. All I can do is make the charge, and then if it’s too soon or too much I get the same error message:

“Validation failed / exceeds mandate cap”

That’s it. It doesn’t tell me what the cap is, it doesn’t tell me if it’s because I’m charging too soon, nor if I’m charging too much. There is no way to distinguish between those situations.

Backwards compatible – sort of

GoCardless talk about the Pro API being backwards compatible to the legacy API, so that once switched I would still be able to create payments against mandates that were created using the legacy API. I would not need to get customers to re-authorise.

This is true to a point, but my use of caps per interval in the legacy API has severely restricted how compatible things are, and that’s something I wasn’t aware of. Sure, their “Guide to upgrading” does briefly mention that caps would continue to be enforced:

“Pre-authorisation mandates are not restricted, but the maximum amount and interval that you originally specified will still apply.”

That is the only mention of this issue in that entire document, and that statement would be fine by me, if there would have continued to be a way to tell which failure mode would be encountered.

Thinking that I was just misunderstanding, I asked GoCardless support about this. Their reply:

Thanks for emailing.

I’m afraid the limits aren’t exposed within the new API. The only solution as you suggest, is to try a payment and check for failure.

Apologies for the inconvenience caused here and if you have any further queries please don’t hesitate to let us know.

What now?

I am not yet sure of the best way to handle this.

The nuclear option would be to cancel all mandates and ask customers to authorise them again. I would like to avoid this if possible.

I am thinking that most customers continue to be fine on the “amount per interval” legacy mandates as long as they don’t upgrade, so I can leave them as they are until that happens. If they upgrade, or if a DD payment ever fails with “exceeds mandate cap” then I will have to cancel their mandate and ask them to authorise again. I can see if their mandate was created before ~today and advise them on the web site to cancel it and authorise it again.

Conclusion

I’m a little disappointed that GoCardless didn’t think that there would need to be a way to query mandate caps even though creating new mandates with those limits is no longer possible.

I can’t really accept that there is a good level of backwards compatibility here if there is a feature that you can’t even tell is in use until it causes a payment to fail, and even then you can’t tell which details of that feature cause the failure.

I understand why they haven’t just stopped honouring the caps: it wouldn’t be in line with the consumer-focused spirit of the Direct Debit Guarantee to alter things against customer expectations, and even sending out a notification to the customer might not be enough. I think they should have gone the other way and allowed querying of things that they are going to continue to enforce, though.

Could I have tested for this? Well, the difficulty there is that the GoCardless sandbox environment for the Pro API starts off clean with no access to any of your legacy activity neither from live nor from legacy sandbox. So I couldn’t do something like the following:

  1. Create legacy mandate in legacy sandbox, with amount per interval caps
  2. Try to charge against the legacy mandate from the Pro API sandbox, exceeding the cap
  3. Observe that it fails but with no way to tell why

I did note that there didn’t seem to be attributes of the mandate endpoint that would let me know when it could be charged and what the amount left to charge was, but it didn’t set off any alarm bells. Perhaps it should have.

Also I will admit I’ve had years to switch to Pro API and am only doing it now when forced. Perhaps if I had made a start on this years ago, I’d have noted what I consider to be a deficiency, asked them to remedy it and they might have had time to do so. I don’t actually think it’s likely they would bump the API version for that though. In my defence, as I mentioned, there is nothing attractive about the Pro API for my use, and it does cost more, so no surprise I’ve been reluctant to explore it.

So, if you are scrambling to update your GoCardless integration before 31 October, do check that you are prepared for payments against capped mandates to fail.

by Andy at September 21, 2017 09:06 PM

Roger Bell_West

The Librarians, season 3

2016-2017 modern fantasy, 10 episodes; the Librarians, who hunt down magical artefacts, take on a god and a government agency.

September 21, 2017 08:01 AM

September 20, 2017

Steve Kennedy

Alivecor Kardia

Alivecor make products that read your EKG. The Kardia Band was previously reviewed and although convenient as it's always on your wrist (it's an Apple Watch add-on) it's extremely susceptible to movement, external interference and lots of other constraints so it's not always easy to get an accurate reading. It also takes a while to process the result as a lot of noise has to be filtered out.

The Kardia Mobile is much easier to use. It's a unit with two electrode pads on it (underneath there's a battery compartment which holds a CR2016 coin cell) which should give about 12 months of use.

Download the free iOS or Android Kardia app and then set-up an account. Then take a reading and the app will search for the Kardia Mobile device and pair with it (it uses Bluetooth 4 so a recent'ish iOS or Android phone must be used with a recent OS).

The app will wait a while, while you put your first and second fingers from both hands on either pad, then relax and the app will record your EKG. It needs to read about 30 seconds to get a sensible heart rate reading.

After the reading is saved it will tell you if there are any abnormalities or anything. As the pads are large and the signal is being received from across the body, the results are much cleaner and less susceptible to interference (compared to the Watch version).

The EKG Characteristics are single lead ECG, 10 mV peak-to-peak input dynamic range, 30 second to 5 minute recording duration, 300 samples per second sampling rate at 16 bit resolution.

There are extra features such as unlimited storage and history, summary reports for your doctor, blood pressure monitoring and tracking weight and medication (though the last two need manual intervention) that are available through the premium service which is available as an in-app purchase.

There's also a phone attachment strip that allows the Kardia Mobile to be carried with the phone (glues on to the phone).

The Kardia Mobile costs £99 direct from Alivecor.

by Steve Karmeinsky (noreply@blogger.com) at September 20, 2017 05:50 PM

Xiaomi Huami AMAZFIT A1603 Smartband

Xiaomi make lots of stuff, but they also work with partners and specifically Huami who make wearables (under the AMAZFIT brand). In the US it's known as the Amazfit ARC.

Though the functionality is almost identical to the Xiaomi Band 2, it looks much nicer and the silicon strap has a 'hatch' effect on it which is very comfortable to wear. The actual unit is permanently attached to the strap and has a soft 'brushed' metal feel. Underneath there's an optical heart rate sensor and charging pins and the top is a UV coated scratch resistant OLED display.

Like most trackers it measures steps but also sleep (you obviously have to wear it at night), distance and active calories. The battery life is VERY good (Huami claim 20 days). It can also display notifications from your phone (currently iOS and Android are supported).

As a bonus it's also water proof and can be used when swimming, though it doesn't track swimming.

There's a USB charging cable that is magnetically polarised so it 'snaps' on to the charging pins on the bottom of the unit, this also stops you getting it the wrong way around.

There is an Amazfit app available and through it's pretty basic, it does support all the devices features and notifications, alarms etc can be set through it. However, it the ARC also works with the Xiaomi Mi Fit app, which has a lot more features and links with Apple Health and Android Fit on the relevant platforms. The Mi Fit app also seems to have a few more advanced features like being able to track heart rate when sleeping which give more accurate results (though reduces battery life).

It's definitely a cut above the Mi Band2 in terms of looks and comfort and on a par with traditional fitness trackers from established US companies, though at a very competitive price. However, the apps of more expensive western vendors are generally quite a bit better with more functionality.

The list price is $99 from the US Amazfit site, but various Asian sites have it available for around £30 including shipping.

by Steve Karmeinsky (noreply@blogger.com) at September 20, 2017 05:49 PM

Roger Bell_West

September 2017 Trailers

Some trailers I've seen recently, and my thoughts on them. (Links are to youtube. Opinions are thoroughly personal.)

September 20, 2017 08:01 AM

September 19, 2017

Roger Bell_West

Hand In Glove, Ngaio Marsh

1962 classic English detective fiction; twenty-second of Marsh's novels of Inspector Roderick Alleyn. Friction in a pair of country houses is the precursor to murder, but everything's tangled in the extreme.

September 19, 2017 08:00 AM

September 18, 2017

Roger Bell_West

Building a File Server 1: planning

Many people these days want to store more data than can be conveniently accommodated on one hard disc. You can buy boxes to store files, or build your own. I've built and upgraded several, and in these posts I'm going to talk about how I did it.

September 18, 2017 08:02 AM

September 17, 2017

Roger Bell_West

The Elder Ice, David Hambling

2014 Lovecraftian horror novella. In 1920s London, ex-boxer Harry Stubbs is working for a firm of lawyers that's looking for valuable assets to pay off some of Shackleton's creditors. But what did he really find on those polar expeditions?

September 17, 2017 08:04 AM

September 16, 2017

Roger Bell_West

Warehouse 13 Game

Here's another boardgame kickstarter that may be of interest, based on a television series I quite enjoyed.

September 16, 2017 08:02 AM

September 15, 2017

Roger Bell_West

Murder in the Dark, Kerry Greenwood

2006 historical detection, sixteenth in Greenwood's Phryne Fisher series (1920s flapper detective in Australia). Phryne is invited to a decadent Christmas party at Chirnside Manor; someone's trying to discredit and kill the hosts. She might not have gone, except that someone's sent her a coral snake to discourage her.

September 15, 2017 08:04 AM

September 14, 2017

Roger Bell_West

Lies that git will tell you

I do actually like git. I find it needlessly obfuscatory and deliberately confusing in its syntax and terminology, but it basically does its job reasonably well. However, there are some popular blatant untruths that I think people would do well to know about.

September 14, 2017 08:01 AM

September 13, 2017

Roger Bell_West

Extension tubes

Thanks to a blog reader, I borrowed some more macro equipment.

September 13, 2017 08:00 AM

September 12, 2017

Roger Bell_West

The Women of Nell Gwynne's, Kage Baker

2009 steampunk SF novella, very loosely connected with the Company series. The finest brothel in Whitehall is also a nest of spies, but very discreet ones.

September 12, 2017 08:02 AM

September 11, 2017

Roger Bell_West

Conviction

2016-2017, 13 episodes. Hayes Morrison, daughter of an ex-president and living with a party-girl reputation, runs a unit that reviews old cases.

September 11, 2017 08:00 AM

September 10, 2017

Roger Bell_West

Type 31e Frigate: First Thoughts

Now that construction has begun on the Type 26 Global Combat Ship, some early announcements have been made about the next British military shipbuilding project: the Type 31e frigate.

September 10, 2017 08:02 AM

September 09, 2017

Roger Bell_West

Red and amber filters

There is a practice common to the road systems of many countries which we don't use in the UK. Why not?

September 09, 2017 08:03 AM

September 08, 2017

Roger Bell_West

Dreamsnake, Vonda N. McIntyre

1978 Hugo-, Nebula- and Locus-award-winning science fiction. On a post-apocalyptic earth, various small groups of people scratch out a living; Snake is a healer, using bioengineered venomous serpents to produce drugs that cure ills and relieve pain.

September 08, 2017 08:00 AM

September 07, 2017

Roger Bell_West

Science Fiction Nitpicking

Why do science fiction fans have a reputation for caring about nitpicky details that no normal person would regard as important?

September 07, 2017 08:01 AM

September 06, 2017

Roger Bell_West

Marlow Tabletop and Board Games 4 September 2017

This Meetup-based boardgames group continues to meet at the Marlow Donkey.

September 06, 2017 08:00 AM

September 05, 2017

Roger Bell_West

Earth 2788, Janet Edwards

2014 young adult science fiction, collection of short stories in the Earth Girl setting.

September 05, 2017 08:01 AM

September 04, 2017

Jonathan Dowland

Sortpaper: 16:9 edition

sortpaper 16:9

sortpaper 16:9

Back in 2011 I stumbled across a file "sortpaper.png", which was a hand-crafted wallpaper I'd created some time in the early noughties to help me organise icons on my computer's Desktop. I published it at the time in the blog post sortpaper.

Since then I rediscovered the blog post, and since I was looking for an excuse to try out the Processing software, I wrote a Processing Sketch to re-create it, but with the size and colours parameterized: sortpaper.pde.txt. The thumbnail above links to an example 1920x1080 rendering.

September 04, 2017 09:34 PM

Roger Bell_West

Polaroid macro lens set

I picked up a cheap set of macro lenses for the GF1.

September 04, 2017 08:03 AM

September 03, 2017

Andy Smith (strugglers.net)

When is a 64-bit counter not a 64-bit counter?

…when you run a Xen device backend (commonly dom0) on a kernel version earlier than 4.10, e.g. Debian stable.

TL;DR

Xen netback devices used 32-bit counters until that bug was fixed and released in kernel version 4.10.

On a kernel with that bug you will see counter wraps much sooner than you would expect, and if the interface is doing enough traffic for there to be multiple wraps in 5 minutes, your monitoring will no longer be accurate.

The problem

A high-bandwidth VPS customer reported that the bandwidth figures presented by BitFolk’s monitoring bore no resemblance to their own statistics gathered from inside their VPS. Their figures were a lot higher.

About octet counters

The Linux kernel maintains byte/octet counters for its network interfaces. You can view them in /sys/class/net/<interface>/statistics/*_bytes.

They’re a simple count of bytes transferred, and so the count always goes up. Typically these are 64-bit unsigned integers so their maximum value would be 18,446,744,073,709,551,615 (264-1).

When you’re monitoring bandwidth use the monitoring system records the value and the timestamp. The difference in value over a known period allows the monitoring system to work out the rate.

Wrapping

Monitoring of network devices is often done using SNMP. SNMP has 32-bit and 64-bit counters.

The maximum value that can be held in a 32-bit counter is 4,294,967,295. As that is a byte count, that represents 34,359,738,368 bits or 34,359.74 megabits. Divide that by 300 (seconds in 5 minutes) and you get 114.5. Therefore if the average bandwidth is above 114.5Mbit/s for 5 minutes, you will overflow a 32-bit counter. When the counter overflows it wraps back through zero.

Wrapping a counter once is fine. We have to expect that a counter will wrap eventually, and as counters never decrease, if a new value is smaller than the previous one then we know it has wrapped and can still work out what the rate should be.

The problem comes when the counter wraps more than once. There is no way to tell how many times it has wrapped so the monitoring system will have to assume the answer is once. Once traffic reaches ~229Mbit/s the counters will be wrapping at least twice in 5 minutes and the statistics become meaningless.

64-bit counters to the rescue

For that reason, network traffic is normally monitored using 64-bit counters. You would have to have a traffic rate of almost 492 Petabit/s to wrap a 64-bit byte counter in 5 minutes.

The thing is, I was already using 64-bit SNMP counters.

Examining the sysfs files

I decided to remove SNMP from the equation by going to the source of the data that SNMP uses: the kernel on the device being monitored.

As mentioned, the kernel’s interface byte counters are exposed in sysfs at /sys/class/net/<interface>/statistics/*_bytes. I dumped out those values every 10 seconds and watched them scroll in a terminal session.

What I observed was that these counters, for that particular customer, were wrapping every couple of minutes. I never observed a value greater than 8,469,862,875. That’s larger than a 32-bit counter would hold, but very close to what a 33 bit counter would hold (8,589,934,591).

64-bit counters not to the rescue

Once I realised that the kernel’s own counters were wrapping every couple of minutes inside the kernel it became clear that using 64-bit counters in SNMP was not going to help at all, and multiple wraps would be seen in 5 minutes.

What a difference a minute makes

To test the hypothesis I switched to 1-minute polling. Here’s what 12 hours of real data looks like under both 5- and 1-minute polling.

As you can see that is a pretty dramatic difference.

The bug

By this point, I’d realised that there must be a bug in Xen’s netback driver (the thing that makes virtual network interfaces in dom0).

I went searching through the source of the kernel and found that the counters had changed from an unsigned long in kernel version 4.9 to a u64 in kernel version 4.10.

Of course, once I knew what to search for it was easy to unearth a previous bug report. If I’d found that at the time of the initial report that would have saved 2 days of investigation!

Even so, the fix for this was only committed in February of this year so, unfortunately, is not present in the kernel in use by the current Debian stable. Nor in many other current distributions.

For Xen set-ups on Debian the bug could be avoided by using a backports kernel or packaging an upstream kernel.

Or you could do 1-minute polling as that would only wrap one time at an average bandwidth of ~572Mbit/s and should be safe from multiple wraps up to ~1.1Gbit/s.

Inside the VPS the counters are 64-bit so it isn’t an issue for guest administrators.

by Andy at September 03, 2017 08:17 PM

Sean Cardus

Brew Day – BBX Vanilla Porter

I’ve decided to take part in this year’s “Black Beer of Christmas” Facebook group brew-off(up?). The idea is to brew a beer, bottle it and send a batch of it to your local hub. Shortly afterwards, you’ll get a box back with a selection of other brewers beer! This year’s theme is Porter, so I’ve decided to have a go at brewing a Vanilla Porter. Recipe: 4.5kg Munton’s Maris Otter / Propino Blend (5.9 EBC) 1kg Munich Malt (20 EBC)...

Read More Read More

by Sean at September 03, 2017 11:24 AM

Roger Bell_West

False Scent, Ngaio Marsh

1960 classic English detective fiction; twenty-first of Marsh's novels of Inspector Roderick Alleyn. The famous comedic actress Mary Bellamy has been getting increasingly troublesome, and now feels that all of her best friends have betrayed her. But only one of them is going to kill her.

September 03, 2017 08:03 AM

September 02, 2017

Roger Bell_West

August 2017 Trailers

Some trailers I've seen recently, and my thoughts on them. (Links are to youtube. Opinions are thoroughly personal.)

September 02, 2017 08:02 AM

September 01, 2017

Roger Bell_West

Finnish Submarine Vesikko

On Suomenlinna, a complex of islands in the bay of Helsinki, lies the submarine Vesikko. With photographs (all taken on the Lumix GF1): cc-by-sa on everything.

September 01, 2017 08:04 AM

August 31, 2017

Sean Cardus

Brewery Upgrade: HERMS System (Part 4)

The PID control box, temperature probe, water heating pot and the re-circulation coil are ready (see parts one, two and three).  The remaining jobs I need to complete are: Install fittings for the pipe work Test everything! The parts used here are: PT100 temperature probe with 1/4″ BSP fitting 1/2″ to 1/4″ BSP Reducing bush 1/2″ BSP female nipple 2 x 1/2″ Camlock Type F fittings 2 x 1/2″ Camlock Type C fittings 2 x hose clips 0.5m silicone hose I...

Read More Read More

by Sean at August 31, 2017 08:33 PM

Jonathan McDowell

C, floating point, and help!

Floating point is a pain. I know this. But I recently took over the sigrok packages in Debian and as part of updating to the latest libsigkrok4 library enabled the post compilation tests. Which promptly failed on i386. Some narrowing down of the problem leads to the following test case (which fails on both gcc-6 under Debian/Stretch and gcc-7 on Debian/Testing):

#include <inttypes.h>
#include <stdio.h>
#include <stdint.h>

int main(int argc, char *argv[])
{
        printf("%" PRIu64 "\n", (uint64_t)((1.034567) * (uint64_t)(1000000ULL)));
}

We expect to see 1.034567 printed out. On x86_64 we do:

$ arch
x86_64
$ gcc -Wall t.c -o t ; ./t
1034567

If we compile for 32-bit the result is also as expected:

$ gcc -Wall -m32 t.c -o t ; ./t
1034567

Where things get interesting is when we enable --std=c99:

$ gcc -Wall --std=c99 t.c -o t ; ./t
1034567
$ gcc -Wall -m32 --std=c99 t.c -o t ; ./t
1034566

What? It turns out all the cXX standards result in the last digit incorrectly being 6, while the gnuXX standards (gnu11 is apparently the default) result in the correct trailing 7. Is there some postfix I can add to the value to prevent the floating point truncation taking place? Or do I just have to accept this? It works fine on armel, so it’s not a simple 32/64 bit issue.

August 31, 2017 04:58 PM

Roger Bell_West

Pyramid 106: Dungeon Fantasy Roleplaying Game II

Pyramid, edited by Steven Marsh, is the monthly GURPS supplement containing short articles with a loose linking theme. This time it's the second of the three issues promised during the Dungeon Fantasy RPG kickstarter.

August 31, 2017 08:00 AM

August 30, 2017

Roger Bell_West

Pyracantha berries on the turn

The pyracantha at the front of the house has a fine crop of berries, and they're on the turn.

August 30, 2017 08:03 AM

August 29, 2017

Roger Bell_West

Marlow Tabletop and Board Games 21 August 2017

This Meetup-based boardgames group continues to meet at the Marlow Donkey.

August 29, 2017 08:03 AM

August 28, 2017

Andrew Elwell

PSU tinkering, Part 1

As previously blogged, I've got a couple of 12v 88.7A PSUs that I'm trying to control under arduino. Stage 1 complete - It powers up with a trivial bit of code

/* Arduino control for (ex) server PSU 
* Andrew Elwell <andrew.elwell@gmail.com> August 2016
* Released under BSD licence
*/

/* Controls / Pins based on data sheet available at
* https://belfuse.com/resources/PowerSolutions/SFP1050/bcd20031_ab_sfp1050-12bg.pdf
*
* A6/B4/C4/D4 +3.3 standby (power to arduino)
* A3/B1/B3/C1/C3/D3 Return
* B5(SDA) / C5(SCL) I2C
* B6 Bring low for PS ON
* C6 AC OK (if high)
* D6 PWR OK (if high)
*
*/


#include <wire.h>

int ACOK = 2;
int PSON = 3;
int PWROK = 4;
int LED = 13;

void setup() {
Wire.begin(); // join i2c bus (address optional for master)
pinMode(ACOK, INPUT);
pinMode(PSON, OUTPUT);
pinMode(PWROK, INPUT);
pinMode(LED, INPUT);

digitalWrite(PSON,HIGH) ; // Stay off until ready
}

void loop() {
if (digitalRead(ACOK) == HIGH) {
digitalWrite(PSON,LOW) ;
}
if (digitalRead(PWROK) == HIGH) {
digitalWrite(LED,HIGH) ;
}
}

The one gotcha that I needed to get it working was to also bring PS A0 low (I2C address) and suddenly green led and 12v out!



by Andrew Elwell (noreply@blogger.com) at August 28, 2017 03:34 PM

Roger Bell_West

Mr Standfast, John Buchan

1919 thriller, third of Buchan's books about Richard Hannay; wartime thud-and-blunder. Hannay has to go undercover among the pacifists and conscientious objectors to root out a German agent.

August 28, 2017 08:03 AM

August 27, 2017

Jonathan McDowell

On my way home from OMGWTFBBQ

I started writing this while sitting in Stansted on my way home from the annual UK Debian BBQ. I’m finally home now, after a great weekend catching up with folk. It’s a good social event for a bunch of Debian folk, and I’m very grateful that Steve and Jo continue to make it happen. These days there are also a number of generous companies chipping in towards the cost of food and drink, so thanks also to Codethink and QvarnLabs AB for the food, Collabora and Mythic Beasts for the beer and Chris for the coffee. And Rob for chasing us all for contributions to cover the rest.

I was trying to remember when the first one of these I attended was; trawling through mail logs there was a Cambridge meetup that ended up at Steve’s old place in April 2001, and we’ve consistently had the summer BBQ since 2004, but I’m not clear on what happened in between. Nonetheless it’s become a fixture in the calendar for those of us in the UK (and a number of people from further afield who regularly turn up). We’ve become a bit more sedate, but it’s good to always see a few new faces, drink some good beer (yay Milton), eat a lot and have some good conversations. This year also managed to get me a SheevaPlug so I could investigate #837989 - a bug with OpenOCD not being able to talk to the device. Turned out to be a channel configuration error in the move to new style FTDI support, so I’ve got that fixed locally and pushed the one line fix upstream as well.

August 27, 2017 06:42 PM

Roger Bell_West

Mainland European Internet

Some recent experience using hotel- and guest-house Internet services while travelling through Germany, Sweden and Finland.

August 27, 2017 08:02 AM

August 26, 2017

Roger Bell_West

Favourite pictures from the Finnish trip

Here are a few of my favourite pictures, taken on the recent trip to Finland.

August 26, 2017 08:03 AM

August 25, 2017

Roger Bell_West

Arabella of Mars, David D. Levine

2016 clockpunkish science fiction. In 1812 Arabella Ashby, daughter of a Martian plantation owner, is sent Home to Earth to learn proper ladylike behaviour; but she'll soon need to find her way onto a Marsman to return to the planet of her birth.

August 25, 2017 08:01 AM

August 24, 2017

Roger Bell_West

Military Museum of Finland

On Suomenlinna, a complex of islands in the bay of Helsinki, lies the Military Museum of Finland. With photographs (all taken on the Lumix GF1): cc-by-sa on everything.

August 24, 2017 08:01 AM

August 23, 2017

Roger Bell_West

Mysterium

Mysterium, designed by Oleksandr Nevskiy and Oleg Sidorenko, is a co-operative game of deduction for 2-7 players.

August 23, 2017 08:01 AM

August 22, 2017

Steve Kennedy

Kardia Band - an ECG monitor for your Apple Watch

The Kardia Band is made by AliveCor and is a watch strap for the Apple Watch (it comes in both 38 and 42mm versions). It replaces the standard Apple Watch strap and there's a big chunk of metal in the bottom strap.

To record your ECG place your thumb on the metal blob and your fore-finger onto the other side of your wrist. Run the Kardio app, sit back and RELAX. It's VERY sensitive to any kind of movement. After 30 seconds it will beep and then press save and it will try and filter out your ECG. If you move or there's lots of electric interference it's unlikely you'll get a good reading. Assuming you're not moving around much and the software processes the data, you'll then get a normal (or not) reading and you can scroll through the waveform, like it's been printed on paper and you move along along it.

AliveCor also make a Bluetooth version where you place your fingers on which is actually considerably cheaper and I'd guess gives better readings, but then you have to remember to take it with you or buy their special iPhone cases which wil hold the device.

The Kardia band does work and it's not reported any abnormal heart readings, which is probably a good thing. It's nice that you can take a reading whenever you like (as long as your wearing your Apple Watch), but it's very sensitive to environmental conditions.

The Kardia Band costs £199 in the UK directly from the AliveCor site (it's not available in the US as it's not FDA approved). The watch strap and metal insert have been improved since earliy version when the metal bit tended to fall out.

The Kardia Mobile Bluetooth sensor is available for £99 from AliveCor and is also available in the US.

by Steve Karmeinsky (noreply@blogger.com) at August 22, 2017 08:59 PM

Jonathan McDowell

Notes on upgrading from Jessie to Stretch

I upgraded my last major machine from Jessie to Stretch last week. That machine was the one running the most services, but I’d made notes while updating various others to ensure it went smoothly. Below are the things I noted along the way, both for my own reference and in case they are of use to anyone else.

  • Roundcube with the sqlite3 backend stopped working after the upgrade; fix was to edit /etc/roundcube/debian-db-roundcube.php and change sqlite3:// to sqlite:// in the $config['db_dsnw'] line.
  • Dovecot no longer supports SSLv2 so had to remove !SSLv2 from the ssl_protocols list in /etc/dovecot/conf.d/10-ssl.conf
  • Duplicity now tries to do a mkdir so I had to change from the scp:// backend to the sftp:// backend in my backup scripts.
  • Needed to add needs_root_rights=yes to /etc/X11/Xwrapper.config so Kodi systemd unit could still start it on a new VT. Need to figure out how to get this working without the need for root.
  • Upgrading fail2ban would have been easier if I’d dropped my additions in /etc/fail2ban/jail.d/ rather than the master config. Fixed for next time.
  • ejabberd continues to be a pain; I do wonder if it’s worth running an XMPP server these days. I certainly don’t end up using it to talk to people myself.
  • Upgrading 1200+ packages takes a long time, even when the majority of them don’t have any questions to ask during the process.
  • PostgreSQL upgrades have got so much easier. pg_upgradecluster 9.4 main chugged away but did exactly what I needed.

Other than those points things were pretty smooth. Nice work by all those involved!

August 22, 2017 08:29 PM

Roger Bell_West

The Private Wife of Sherlock Holmes, Carole Nelson Douglas

2012 historical mystery novelette, part of Douglas's series about Irene Adler. Irene visits London and enlists the help of Sherlock Holmes to save an old friend from a complex blackmail plot.

August 22, 2017 08:01 AM

August 21, 2017

Sean Cardus

Brewery Upgrade: HERMS System (Part 3)

The PID control box, temperature probe, and the water heating pot are ready (see parts one and two).  The remaining jobs I need to complete are: Make a copper pipe coil Install fittings for the pipe work Test everything! So, next up is some copper bending! The parts used here are: 3 meters of 12mm copper pipe Kids play sand 2 x 1/2″ / 12mm compression fittings 1/2″ Tee piece A 500ml beer bottle I first tried to bend the...

Read More Read More

by Sean at August 21, 2017 06:10 PM

Brewery Upgrade: HERMS System (Part 2)

The PID control box and temperature probe are ready (see part 1 here).  The next jobs I need to complete are: Fit a heating element to the asparagus pot Make a copper pipe coil Install fittings for the pipe work Test everything! So, next up is setting up the pot that’s going to hold my warm water and act as the heat exchanger.  The parts I used are: 3 Litre Asparagus Steamer 1500W Kettle Element The steamer is basically just...

Read More Read More

by Sean at August 21, 2017 05:00 PM

Roger Bell_West

Mainland European Driving

In the UK, drivers are told to stay in the lane closest to the edge of the road (lane 1), pull out to overtake, and pull back in afterwards. In mainland Europe they actually do this.

August 21, 2017 08:01 AM

August 20, 2017

Roger Bell_West

Singing in the Shrouds, Ngaio Marsh

1958 classic English detective fiction; twentieth of Marsh's novels of Inspector Roderick Alleyn. The passengers aboard the Cape Farewell are travelling to South Africa; but it seems that the Flower Murderer who's been plaguing London is among their number.

August 20, 2017 08:02 AM

August 19, 2017

Roger Bell_West

Finnish Air Force Museum

Near the Jyväskylä airport at Tikkakoski is the Suomen Ilmavoimamuseo, the museum of the Finnish Air Force. With photographs (all taken on the Lumix GF1): cc-by-sa on everything.

August 19, 2017 08:03 AM

August 18, 2017

Roger Bell_West

2017 Hugo Awards: what happened in the end

How well did this year's Hugo awards match my (obviously correct and superior) preferences?

August 18, 2017 08:01 AM

August 17, 2017

Roger Bell_West

GURPS Vehicles: Steampunk Conveyances

Vehicles aren't the defining image of steampunk, but they're certainly one of the important aspects of it, whether it's personal carriages or massive airships. This book lists a number of historical, speculative and fantastic vehicle designs, with stats for use in GURPS.

August 17, 2017 08:02 AM

August 16, 2017

Roger Bell_West

Armada, Ernest Cline

2015 science fiction. Zack Lightman is a dreamer and geek with anger management problems. But his video gaming skills are going to be needed to fight off a real alien invasion. Spoilers.

August 16, 2017 08:03 AM

August 15, 2017

Zoe O'Connell

Companies House and Charities Commission advice on former names

Companies House advice on dead names
A topic that comes up frequently in Trans circles is the problem of “Dead” (I.e. pre-transition) names, something that many trans people are reluctant to make public for a whole variety of reasons. Unfortunately, many forms ask for this information with the assumption that most people who have changed names have only done so because they have married.

I ran into this problem myself just over a year ago, in the context of a charity I became trustee of as a result of my role as a councillor. Neither Companies House nor the Charities Commission actually require this information but that is not made clear, and when holding public office it is often necessary to dot the i’s and cross the t’s on paperwork to avoid unforeseen repercussions later on. I asked both organisations for confirmation, and I reproduce both their responses complete with reference number in the hope that it helps others in future.

Charity Commission advice on Dead Names Click the images for full size versions, and plain text versions of the respones from Charity Commission and Companies House are also available if anyone needs them.

The post Companies House and Charities Commission advice on former names appeared first on Complicity.

by Zoe O'Connell at August 15, 2017 08:30 AM

Roger Bell_West

Colony

Colony, designed by Ted Alspach, Toryo Hojo and Yoshihisa Nakatsu, is a dice-based resource-building game of post-apocalyptic survival for 2-4 players.

August 15, 2017 08:03 AM

August 14, 2017

Zoe O'Connell

More payment changes at InHouse Pharmacy

I have not been keeping a particularly close eye on online pharmacy services for a while, but an interesting email from InHouse Pharmacy sent to their customers has come my way – and it suggests that the ongoing pressure on online pharmacies has not let up recently, with international payment provider OrbitRemit now refusing to serve online pharmacies.

I could not find any further information about this change on either InHouse Pharmacy or OrbitRemit’s web sites, although I did run across an FAQ on IHP’s web site that’s now specifically blaming “BigPhama” – something that’s long been assumed to be the root cause of their problems, but had not previously been confirmed: “Unfortunately we no longer accept Visa Debit and Credit Cards or MasterCard due to lobbyist pressure from BigPharma interests.

The good news for IHP and those using their services is that Bitcoin is becoming increasingly mainstream and now list this as their preferred payment option. Although it offers no buyer protection compared to other services, it is hard to see any way that pressure can be brought to bear on IHP via this route.

The post More payment changes at InHouse Pharmacy appeared first on Complicity.

by Zoe O'Connell at August 14, 2017 08:30 AM

Roger Bell_West

Spider Dance, Carole Nelson Douglas

2004 historical mystery, eighth of Douglas's novels about Irene Adler. Still in New York, Irene Adler looks into the last days of the woman who might have been her mother, and finds that other people are taking a violent interest in the matter.

August 14, 2017 08:02 AM