statistically speaking... guardian writer is confused about statistics

This piece by guardian writer Barbara Ellen probably needs to be rechecked by their sub-editors for statistical correctness.

If you really think you're gay, guys, why not act on it?

The latest National Survey of Sexual Attitudes and Lifestyles revealed many things (from unplanned pregnancies to unreported sexual assaults), but also that there has been a 400% increase in (mainly younger) women, some fully lesbian, declaring same sex sexual experimentation – around 16%. By contrast, the number of men who'd had a same sex experience had only gone up 1% (to 7%) since 1991.

The data that Barbara Ellen is quoting appears to be this:

Percentage of the population who have ever had same-sex experience (people aged 16-44)



So I think there are two problems with the quoted statement above.

1) There has been a 300% increase in women declaring same sex experimentation.

The absolute increase of 4% to 16%, is 12%... and 12% is 300% of the 1991 figure of 4%.

2) Barbara is also comparing the change in the proportion (of women declaring same sex experimentation) as a percentage of the original 1990-91 value (which should be 300%), to the absolute change (of men declaring same sex experimentation), from 6% to 7%, which is 1%.

However these are not equivalent values. The correct comparison figure for men (of percentage change in proportion of population), to the 300% figure would be 16%, i.e.  

The change from 6% to 7% is a 16.66% increase, when looked at as a percentage change in the proportion.

So the correct way of describing that would be:

"also that there has been a 300% proportional increase of (mainly younger) women, some fully lesbian, declaring same sex sexual experimentation – around 16%. By contrast, the proportional change in percentage of men who'd had a same sex experience  was only 16.66% (to 7%) since 1991."

However, it would also be possible to look at both figures as absolute changes, and that would result in the following correct version:

"also that there has been a 12% absolute increase in (mainly younger) women, some fully lesbian, declaring same sex sexual experimentation – to around 16%. By contrast, the number of men who'd had a same sex experience had only gone up 1% (to 7%) since 1991."

I might be wrong about the interpretation, if I am using the wrong source data...

"In Florida one in three death row prisoners are subsequently proven innocent. "

The following quoted proportion of judicial failure seemed unlikely, even for Florida:
In the US one in ten death row prisoners are subsequently exonerated.
That's not let off the punishment and put in prison instead, or found not guilty on a technicality;
that's proven innocent.
In Florida it's one in three.

So I decided to determine a more realistic estimate of that proportion.

How to whitelist an email address in gmail

Go to "settings", it's the button with the cog icon on the top right

Select the filter tab from the row of links along the top

 Scroll down to the bottom of that list of filters. If you do not have many filters setup, like I do... you may not need to scroll down;

but there should be a link "Create a new filter", click that:

Enter the email address of the users that you wish to whitelist, in the "From:" field and then Click, "create filter with this search":

Then from the new options select, "Never send to Spam"

Also: if you use priority inbox, you might also want to select "Always mark it as important"

And now, no more rummaging around in the "all mail", or "Spam" folder

IT is a misnomer. It's no longer just information technology. It's really real technology. (draft)

(I was hoping this post made more sense, but now I've written it, I am not sure it captured the idea...)

I remember one of the early programming examples was to program a "library" database of books, authors, and lendings. Or how when google search returned relevant results (circa ~1998ish), allowing you to actually find the information you are looking for.

And then the dotcom boom of the early naughties, which promised to "create paradigm changing efficiencies".

But all that is just information technology facilitating existing industrial or commercial applications. Or in other words, a load of bollock-speak

I am about to get in a car running an embedded system, drive to a hospital where lives are sustained by computer controlled pumps and pacemakers.

When a person is killed by a missile from a predator controlled drone, they are not being killed by IT, they are being killed by really real technology.

I think the term "Information Technology", as in, "let's see what the IT department think", or "Do we have the IT for this?"

When you can scan and print 3d objects, send CAD files off by email and received working prototypes in the next days post. Then it's no longer just information.

Gut bacteria promoting colorectal cancer (40 minute biostatistics)

A couple of studies have been published recently, and quite a bit written about them, which link the abundance of types of bacteria found in the mouth with incidence of colorectal cancer.

These finding result from the observation that the oral bacterium Fusobacterium nucleatum is often found in high abundance in colorectal carcinoma tissue samples.

Gram-negative stained culture of F. nucleatum. Image Courtesy of J. Michael Miller, Ph.D.,(D)ABMM of National Center for Zoonotic, Vector-borne, and Enteric Diseases. Picture submitted by him to American Society for Microbiology

Image & caption nicked from:

I've not read up on the science and microbiology in any depth, but I got the feeling that it would be interesting to plot and summarize some of the public health data available on two variables relating to Oral health and Colorectal cancer incidence.

Hence I pulled down some data from the web and made some charts.

US state by state data sets are often complete, with up-to-date and freely available for many observable values.

After a bit of google searching, I came up with these 2 sets from CDC:

The US Centers for disease control and Prevention have data for colorectal cancer for 2009 here;

[Age-Adjusted Invasive Cancer Incidence Rates and 95% Confidence Intervals by State (Table 5.4.1M) *†Rates are per 100,000 persons and are age-adjusted to the 2000 U.S. standard population (19 age groups - Census P25-1130). Rates are per 100,000 persons and are age-adjusted to the 2000 U.S. standard population (19 age groups - Census P25-1130).]

and the data available for download

The site also carries summary data from the BRFSS survey, which is the "Behavioral Risk Factor Surveillance System", which collected data for 2008 asking adults 18+, whether they "had visited a dentist or dental clinic in the past year":

The two data sets can be merged, and then scatter plot and correlations, and best fit plotted. (assuming linear association and normally distributed data etc etc)

here is the summary data:


characterize the association... and consider any other ideas..

August 2013


genome sequence for Fusobacterium nucleatum SubspeciesPolymorphum

please join my petition to "Ban internet Perry"

In recent years there has been an alarming increase in what we refer to as "Claire Perry" on the internet.

Many people who subsequently became entrapped by the seductive logic of "Claire Perry" were only casual internet users, who stumble across some "Claire" in a chat room, or at the conservative national convention.

'There are many sad cases, where a politician, with ailing polling numbers and down on his luck, to turn to a bit of "Claire" to perk up his public profile, but then becomes addicted to "Claire" and having to keep going back for a bigger fix each time due to being accused of being soft* on child porn'

This is a two stage proposal, initially we are asking Internet Service providers to popup a warning when users search for dangerous terms like "deranged, untethered, divorced, Christian MILF"

In the second stage we hope to ban other offensively annoying fucktards such as "Michael Gove" and "George Monbiot"

* Whether it is a good thing to be accused of being "hard" on child porn is also dubious.

Sherlock Puzzle (Series 2 Episode 1) "This email will save the world"

I was catching up on the Sherlock series 2 the other day on iplayer.

During the "A Scandal in Belgravia" episode Lara Pulver's character Irene Adler shows Sherlock an email containing a code. Irene Adler then says to Sherlock "go on then, impress a girl".. or words to that effect.

Given that I was developing a soft spot for Irene Adler, that was effectively a challenge I wasn't going to miss, so I copied it down, and had a crack at deciphering it.

Fortunately at that point they did a little slow-motion sequence with overlaid animation of letters appearing in front of Benedict Cumberbatch's head, and I was able to pause it before they gave the answer away.

Any apps which would implement a pairwise comparison of "product features" in head to head playoff elimination format?

I have this massive list of "product features" for an app that I am working on, and I am having trouble just making sense of which to look into first.

I thought it would be cool to have an app that would present all the items successively in "pairwise comparison" format, to enable to determine an "implied priority"

A 1000Minds presentation of two contenders in a pairwise comparison format;

The idea being that initially you would start with a rough list and then successively present pairs of these items (e.g. "add X widget feature" vs "improve Y process") to the stakeholders.

If twitter is Ham-radio for normals, then is ...?

My pet thesis is that there is absolutely nothing new about the internet (I mean as opposed to smoke signals and hieroglyph), except for 2 things:

1) maintaining the persistent weak links in networks and 2) integrated search (or probably more accurately ubiquitous, pandemic indexes... meaning the pulling out, and structuring of data into information from disparate sources)

Hence I take an interest in things that I think are really just something old, dressed up in jQuery or social media, and presented as new.

(I particularly hate the application of the word "cloud" to everything in tech and calling it new. "Hey, come and see our new "cloud" app"... what that thing this is your old desktop app with some marketing guff and a "cloud-" prefix..)

a very funky book

I bought this item from for 0.92GBP plus shipping. for a total of 4.46.

I think the previous owner had it stored in some pretty dank conditions. It's got a quite severe musty funk to it.

Having a baby is...

"...a bit like sitting in the airport waiting for a plane. It's a bit scary, a bit boring, a lot can go wrong, and in reality you have little control over the situation" [anon.]

(And no. I'm not having a baby.)

"fact patterns" - is that new, or did I miss the birth of that phrase? (draft)

From reddit (regarding mandatory sentencing in the US):
As a lawyer I'll say that when it comes to the legal (criminal or civil), honesty is not always the best policy. But silence usually is.

People unfamiliar with the system often mistake truthful statements, arguments of honest mistake or justification, acceptance of responsibility, etc., as doing the "right thing" and pay dearly for doing it.

This issue is especially troubling in instances where the legal system has set up a no-tolerance rule, mandatory reporting, or otherwise deprive responders of "discretion." This happens a lot on the criminal side: domestic abuse (i.e., mandatory arrest), some sex and gun crimes, vehicle-alcohol laws, etc. The domestic abuse one in particular, while well-intentioned, can result in some very bizarre fact patterns for police.

It appears that this guy/gal is a lawyer-speaking, but using the phrase to
mean "sequence of events", or more likely a kind of predictable cascade of institutional response to... (well the fact that I can't define it, suggests it has utility, because I certainly know what they are getting at)

they continue...
For a great many otherwise honest, good people, the first run-in with the legal system, they learn that trying to be helpful, doing the right thing, etc. is some times the wrong thing for them personally.

 The closest definition I am thinking of, is that it functions in a much more precise way as does "story" in something like "it was the same [familiar] story"

ginger steps on venus.

So this joke showed up in my google+ feed at Funny Jokes+ and was reposted at Remo malash+, Haley Falconite+ and others. My first thought was why is Blondie getting all the abuse. I doubt ginger would last long on the surface of venus!

Given that the surface pressure on Venus is ~92x of the Earth, and that spacesuits are designed for low pressure environments. I think something like a deep sea submersible would be required to survive, making it difficult for "ginger to walk on venus" without being squished.

Chef: ERROR: RuntimeError: Please set EDITOR environment variable

I've just setup a workstation on windows, and I'm getting the following error when I try to edit a data bag;

$ knife data bag edit tomhodder tomhodder
ERROR: RuntimeError: Please set EDITOR environment variable

tldr; SOLUTION: use a text editor that understands windows paths ("C:/Users/whatever/file"), rather than unix ("/C/Users/whatever/file"), e.g. export EDITOR=notepad

Is the brain's "embarrassment" architecture single register in working memory?

At various points in my life I am plagued by what could be described as persistent unwanted thoughts. Basically something comes to mind, and is difficult to dislodge maybe for an hour or so, but sometimes much longer. I have strategies I like to use to deal with this, such as to actively get up and do something else to trigger a frame change.

A similar recollection-of-cringe-worthy incident occurred just now, but something slightly unique took place which I thought I would throw out for discussions.

Chef web app developer training - Part 7 continued even more... checks/real life (draft)

The purpose of this series of tutorials is to take web app developers through the basic knowledge necessary to create and maintain the cookbooks used to deploy web applications.

Here I demonstrate that searches on data bags are applicable to other indexes and lists of objects in the chef server architecture.

Chef web app developer training - Part 7 continued even more (draft)

here are some checks that you have setup the data bag correctly;

Chef web app developer training - Part 7 continued (draft)

The purpose of this series of tutorials is to take web app developers through the basic knowledge necessary to create and maintain the cookbooks used to deploy web applications.

Here we are going to create a data bag which we will shortly be consuming in a cookbook recipe.

We are going to create a databag on class01 named for your favourite sports team and add some players to it.

Chef web app developer training - Part 7 (draft)

The purpose of this series of tutorials is to take web app developers through the basic knowledge necessary to create and maintain the cookbooks used to deploy web applications.

The prerequisites for Part 7 are to have a working setcloud configuration for "class01" and your own hosted chef organization e.g. "tomhodder". ("knife cookbook list" etc, should work)

The purpose of Part 7 is to cover "data bags", what they are and how they work.

Chef developer training - Part 6 continued (draft)

Here we run some checks to ensure the settings are correct

Chef developer training - Part 6 (draft)

The prerequisites for Part 6 are to complete part 5

Chef web app developer tutorial - Part 5 continued (draft)

Check that the environment variables are correctly loaded


Chef developer training - Part 5 (draft)

The purpose of this series of tutorials is to take web app developers through the basic knowledge necessary to create and maintain the cookbooks used to deploy web applications.

The prerequisites for Part 5 are to complete part 4, and to have available the configuration values for both "class01" and your own organization e.g. "tomsorg".

The purpose of this part of the tutorial is to make it super easy to switch between different chef-servers, and organizations. The developer workflow will be to typically have a dev or test organization, and some other production target chef-server.

Chef web app developer training - roundup of parts 1-4

These are the requirements to complete for each part of the session; (replace tomhodder with your opscode username where appropriate)

Chef developer training - Part 4 (draft)

The prerequisites for Part 4 are to complete part 3, and to have a username associated with opscode.

Chef developer training (Part 3 steps)

Right now you should have an opscode username, like "tomhodder".

If you are following the tutorials and wish to access the finished versions of the cookbooks, please send your opscode username to me, and I will add you to the "class01" organization.

If you are not following the tutorials, and do not wish to share your opscode username, then please use your alternative organization instead of the "class01" where relevant.

Chef web app developer training - Part 2 (draft)

The purpose of this series of tutorials is to take web app developers through the basic knowledge necessary to create and maintain the cookbooks used to deploy web applications.

Here we create a host chef account, which provides a username and organization that we can use for the later tutorial steps. Once you have created the username at opscode, if you send it to me, I will enable it for access to the "class01" organization that contains the cookbooks for this tutorial.

This part of the tutorial is basically to complete the "Using a Chef Repo" tutorial from the getting started series. However it will serve as a basis for the next parts in this series.

Chef developer training - Part 1 continued (draft)

Once you have completed the the Part 1 steps, you should step through the checks below, ensuring that you are able to replicate the commands and actions.

Chef web app developer training - Part 1 steps (draft)

The purpose of this series of tutorials is to take web app developers through the basic knowledge necessary to create and maintain the cookbooks used to deploy web applications.

Here I am using the stack CentOS, Oracle java and tomcat7, but they are generally applicable to the other app stacks.

This part of the tutorial is based off the Chef Workstation Setup getting started article. I'm skipping the Vagrant and VirtualBox installation part until later.

The ideal situation occurs when the things that we regard as beautiful are also regarded by other people as useful.

--Donald Knuth

Oh yes indeed!

  Stacked graphs–geometry & aesthetics - Byron et al 2008


Byron, Lee and Wattenberg, Martin (2008) ‘Stacked Graphs – Geometry & Aesthetics’, IEEE Transactions on Visualization and Computer Graphics, 14(6), pp. 1245–1252, [online] Available from: (Accessed 30 April 2013).

Van Horn, John Darrell, Irimia, Andrei, Torgerson, Carinna M., Chambers, Micah C., et al. (2012) ‘Mapping Connectivity Damage in the Case of Phineas Gage’, PLoS ONE, 7(5), [online] Available from: (Accessed 30 April 2013).

irrealis mood as an excuse to talk bollocks

I received what initially appeared to be a patent trolling email which made interesting use of the following grammatical construction:
"He might wish to know that I have been granted a patent on the invention he is seeking information on and I am happy to discuss licensing opportunities."
The sentence uses a construction referred to as subjunctive mood, the important part being:
"He might wish to know (that) ..."
This technique allows the author to make statements about the imagination ("In a dream I owned a big blue dog"), or opinion ("I think dogs are nice") in the language that we would usually discuss real things in the world ("I own a dog").

patent troll or just troll? I have definitely been trolled.

I got an email earlier, which I assumed was just an automated template from the bowels of some patent trolling machine that I had triggered by my keywords...

"Dear Sirs,

I understand Tom Hodder works for this firm. I believe he posted the following advert on PeoplePerHour:

He might wish to know that I have been granted a patent on the invention he is seeking information on and I am happy to discuss licencing opportunities.


Jonathan Bishop
Inventor of 'Assisting Human Interaction'
Director, Jonathan Bishop Limited"

INFO: Chef client is running, will wait for it to finish and then run.

tldr; So I wasted a few hours today because I had changed a setting, (or chef had changed) the location of its lock file ( I found that manually removing the file, and restarting chef was a work-around.

because I am gettting annoyed by seeing my own face, staring back at me out of every ****ing webpage I go on...

my initial thought was that if it was going to be something that I was going to be seeing thousands of times a day, it should be an image that I like to look at...

But that got boring.

I decided it might be amusing to have a cartoon, or caricature instead. So I uploaded some profile pix to and they came back with these...

If only I looked like this....

 rather than this crazy looking b*****d...


keyboard abuse

Who thought this layout was a good idea?

It's not like I even use the right shift key frequently.

Surely half a right shift key makes more sense than half an up arrow key?

How am I going to stab away in frustration at the Up Arrow key, to navigate the bash history with such a small target to poke at?

Even google are at it now;

Originally images located by google image search, and purloined from here;*ytimg*com%7Cvi%7CeRO1z3K_m0Y%7C0*jpg/
(but its looks to be of dubious original providence)

cheating on the nine dot problem.

While procrastinating before an essay deadline, I happened across a blog post with the following challenge

In their study, 22 volunteers were faced with the 9 dots problem, a notoriously difficult puzzle. The goal here is to draw exactly four straight lines connecting all nine of these dots, without retracing any line, or lifting your pen from the page.

Can you do it?

Given that I had a relatively important deadline approaching, that was one opportunity to procrastinate I was not going to miss.

so my second favourite loony running the asylum is at it again...

Claire Perry MP asked the following question to the Prime Minister in the HOP ;

"Britain’s universities lead the world in teaching science and engineering, yet we have an annual shortfall of 60,000 graduates, and nine out of 10 postgraduate students in those subjects are from overseas. What more can we do to plug that critical skills gap?"

Is this the same Claire Perry who voted for tuition fees?

"Claire Perry MP, Devizes voted to approve raising the tuition fee cap to £6,000 per year for courses for which there are no plans in place to promote access and student finance information."

Yes, I think it is....

Is this the same Claire Perry who voted to raise the tuition fees to 9000GBP;

"Claire Perry MP, Devizes voted to raise the UK's undergraduate tuition fee cap to £9,000 per year."

yes, again. I think it is.

(I didn't put much research into this post, and so Claire Perry might have a good explanation for why she is publicly lamenting a circumstance which her own actions contributed to. But I don't think you can have it both ways)