Top Down View

So I was talking the other day with friends about the prediction for The Rapture on Saturday. That would be Saturday just gone – the Rapture that didn’t happen. I pointed out that the prediction was based on religious numerology – you look for number patterns in things and assign meanings to them. The great thing about numerology is that you can use it to predict anything you like. I joked that Winnie The Pooh contains a hidden subtext revealing that Pooh The Destroyer is about to come and inflict his wrath on the world.

It was suggested to me that the Pooh quote “When having a smackerel of something with a friend, don’t eat so much that you get stuck in the doorway trying to get out.” is a clear Rapture reference.

But the scary thing is that IT IS – and I’d like to prove it to you. Continue reading Beware Pooh The Destroyer

Through my friend Steve, today I discovered The Pierley-Redford Dissociative Affect Diagnostic personality test.

Go and try it for yourself… it won’t take long & I’ll wait a minute for you.

OK. Interesting? Weird? Yeah, I’d probably agree with both those assessments.

My diagnosis came back as

“Quiet and very self-assured, you tend to keep your own council. Pragmatic and practical to a fault, you are not one to worry about the finer points of philosophical discourse. In fact, because you are very much an individualist, you often finds yourself at odds with the established truth or the wishes of the majority. You will often earn the wrath of an employer by taking upon yourself decisions which are rightly those of your manager. You are not one to take credit unless it is deserved. Similarly however, you will also not happily give credit where it is not due. In a romantic relationship you can be very frustrating. While you do care deeply and sincerely, and are willing to work at a relationship, your confidence in your own abilities can on occasion make it difficult to see the world from a partner’s point of view. Quiet and stoic at times, you can drive a more emotional individual completely up the wall. You can become overstressed and fatigued without knowing it. Taking time to rest between bouts of hard work can help to prevent a breakdown later on”

There’s definitely things in there that I’d agree with but also definitely things that I’d take issue with. So what’s going on here?

Firstly, I didn’t feel strongly drawn to a specific answer for many of the questions. I tried to not project external knowledge or associations onto the shapes, eg “is this a white blood cell consuming invading bacteria?” but that just makes it harder to make a decision & hence easier to go either way with just a shrug of the shoulders. So if I took the test a second time I’d probably answer differently and suddenly I’m not so “quiet and self-assured”. Maybe now I’m “emotionally volatile and prone to sudden changes of opinion” (although of course the test would claim that I’d just conclusively proved that!).

The diagnoses that the test produces are vague and generic. They smack to me of horoscopes… “you will meet a tall, dark stranger”, “money will be an important issue to you in the next seven days” etc. I strongly dislike horoscopes… I believe them to be random mumbo-jumbo designed to ensnare those incapable of critical thought. In fact if a friend is reading the newspaper and asks me for my star sign so they can check my horoscope, I always pick a different sign at random… it’s entertaining to watch them nod sagely as they read about how a twelfth of the population are expected to behave for the next seven days and how they can see those characteristics in me.

Let’s do some research. This personality test would appear to be named after its creators. So try a web search for “Pierley Redford“… all we get are references to this test. Try them separately… obviously Redford is a hard name to search for but again Pierley produces only pages discussing this test. So search for pages mentioning Pierley but not using the word ‘test’ – that clears the results up a lot, but still no references to any actual work, no peer-reviewed journals, no actual person. Doesn’t this seem odd?

Finally, check where this personality test is hosted. http://www.hypnoid.com/ is the website of a web designer. Not a psychologist, a psychiatrist or a university department… a web designer. When you search for “Pierley Redford“, the page title that’s (at the time of writing) cached by Google for this page is “Test – Richard Horton Portfolio”. Portfolio? Ahhh, you mean an exercise to show off the designer’s web skills. Nice.

Still not convinced? OK… try this search: search for “‘dissociative affect diagnostic’ without Pierley or the misspelling Pierly“. We see posts on several other sites linking to the same test but referencing it as the “Brierly/Medford Dissociative Affect Diagnostic”. The posts all seem to date from about the same time and all clearly reference the same test. This leads us to conclude that within the last month this test has been renamed from Brierly/Medford to Pierley-Redford. Same test, different (but almost the same) names. Clearly an invented set of names.

The alleged personality test contains no links to further research and it flickers and pulses annoyingly, like something from Lost. I believe this is purely an exercise in Flash and a piece of rather well-done self-promotion by the designer. It’s cute for sure, but holds about as much meaning as the horoscope in the back pages of Sunday’s paper. Enjoy it but don’t freak yourself out.

I still remember many a lunch spent with work colleagues during the early days of widespread internet adoption where we’d brain-storm a great new internet-related idea that would make us all millionaires. We’d come rushing back to our desks after lunch and immediately search to see if anybody had beaten us to the idea.

Invariably they had. At which point we would abandon the idea.

But a couple of years ago I took an interesting course at BCIT on web application development and business. The tutor talked about various ideas that he’d had and projects that he’d built on-line. His approach was the exact opposite of ours. He said that if he can’t find someone that’s already implemented something similar to his idea then he walks away. His rationale being that its presence validates his idea… someone’s out there making money from the idea. If there’s NOBODY out there offering the service then this shows him that there’s no money to be made from it.

Of course this isn’t always the case… every big idea has someone who does it first and definitely not every website out there is profitable. But if you’re just looking for a web app that you can turn out and make a little money on, seeing an existing market is a good indicator as to the idea’s validity.

Having identified an existing implementation, you shouldn’t simply offer the same thing… your implementation has to differ in some important aspect – something that would make someone choose your site rather than another one. If the competitor’s site is full-featured and complex then make your site simple and easy-to-use. If their site is simple and rigid then make yours flexible and customizable. If they don’t offer a mobile version then make sure you offer one. If they offer subscription levels for 2, 10, 100 (users, accounts, MB – depending on your business model) then you offer subscriptions at 5, 20, 50.

So, in our case, is someone already offering an online vehicle fuel economy tracker? A quick web search shows that yes there are several out there – there’s definitely interest in the service we’re planning. Some sites are definitely better than others but amongst the collection that I investigated, I found some nice simple interfaces, some mobile sites, some good data output and graphs – however not necessarily all on the same site.

One cool feature I had already thought of but was suprised to find implemented was “text message fillups” – register your cellphone and you can send in a text message to record a fillup. Not sure I can implement that without spending money – maybe a later addition!

Amongst the flaws I noticed:

• use of Flash for graphics. Definitely have to make sure that’s avoided
• SEO on all these sites appears to be very spotty. Depending on exactly how you word your search for a fuel economy tracking website, you get wildly different results with sites that might have been top of the list for one search vanishing entirely from the front page for another search

In summary, yes, there appears to be a market for this application. I’m not sure that I’ve identified a unique niche at this point but we’ve definitely found an idea that people want to use.

Our next stage is to think about the application design…

This post is part of a series – read them all

The best ideas for web applications are usually ones where you identify a need of your own – something that you personally want.

Something I’ve done ever since I first owned a car is to record my gas mileage – how many miles I get on each tankful and hence how many MPG (or l/100km) I’m getting. It’s interesting to watch how the numbers vary according to different factors:

• how much of the tank has been used for city driving and how much has been used on the highway
• summer driving vs winter driving
• times when you’ve been consciously driving economically vs times when maybe your right foot’s been a little heavier

When I bought my first car, I typed these figures into a spreadsheet. The spreadsheet grew to be quite sophisticated with different stats, averages and graphs. It gave me everything I wanted. In 2000 when I bought my beautiful Palm Vx I switched to an MPG tracking app on there which meant I always had the numbers with me, but at the cost of using a tool which wasn’t entirely to my liking, didn’t give me all the features I wanted and locked my data away from me.

These days I still record the numbers but I haven’t been tracking them with anything for years. I have a couple of envelopes packed with old gas receipts but I have no statistics based on those figures.

So, what I want is a website where I can enter my gas purchases and track my MPG. Simple.

With the idea in mind, the next stage is research…

This post is part of a series – read them all

I’m going to write a new web application from scratch. Just for fun & with no intention of making my fortune from it (but it wouldn’t hurt if I did).

I’ll be writing it in PHP – possibly plain PHP or possibly using a framework. I’ve done some work in the past with the Zend Framework, CakePHP and Symphony and not felt entirely satisfied with any of them so, if I do use a framework, I’ll be using Code Igniter. I’ve heard good things about CI, talked to a couple of people who love it and worked my way through the tutorials without being put off yet… it seems to fill a nice middle-ground between CakePHP and Symphony.

To make life more fun, I’ll be documenting my progress here, talking about what I’m doing and sharing some of the code.

This post is part of a series – read them all

Hmmm – where HAVE I been?

Well, the answer to that question is “right here!”. I guess the more accurate question is “what have I been doing?”.

One of the things I’ve been doing is restarting my homebrewing. It’s been going very well indeed – incredibly enjoyable work producing 4 batches of beer, 1 of wine and a batch of apfelwein (a dry German cider) so far. Made and enjoyed with (touch-wood) no failures or anything that wasn’t absolutely top-notch – I’ve been really impressed with the outcome (and, I guess, impressed with myself for producing it!). I’ve been photographing the hell out of the process and you can see the photos on flickr over here and maybe we’ll see about another form of documentation in the near future.

I’ve also left my Toastmasters club. I felt I’d gone as far as I could with them. Three years membership, half of it as VP Education taking us to two successive Presidents Distinguished Club awards. I definitely don’t feel I’ve finished with Toastmasters, but it was time for some new challenges for me, so I’m currently looking at new clubs to join.

Professionally, I’ve been doing some part-time consultancy work but I think it’s time to find a full-time development job. To help with that goal I’m going to be brushing off a couple of demo sites and projects over the next month. Always good to have something to show people.

Twice a year, Toastmasters turn their thoughts to contests. In September each club stages a Humorous Speech Contest and a Table Topics Contest with the club winners going on to compete in Area, Division and District competitions.

The organization required for these competitions can be quite daunting to newer members but it’s very important that everything happens in accordance with the official rules. To help ensure that people know how to run a contest successfully there are training sessions held to help get them up to speed.

In the past, I’ve competed, judged, chaired and mentored speech contests so I’m fairly confident that I know my way around one, but I know that you can never know EVERYTHING, so last weekend I attended a speech contest training session run by District 21 Division B in Vancouver.

Despite having experienced many contests in the past and despite being very familiar with the contents of the official rulebook, I learnt LOTS. I took lots of notes and decided to pass them on in the hope that they might help someone else too.

This information isn’t a definitive guide on how to run a contest but it expands on the information in the Toastmasters Speech Contest Rulebook. You should probably be familiar with the rulebook in order to get the most out of these notes…

Why is it important to run a good contest? It’s important because the competitors have put time and effort into taking part, you owe it to them to ensure that your contest is FAIR and abides by the rules.

I was guest judge at a neighbouring club’s contest this week and noticed several things which didn’t go smoothly so I guess I can add a supplemental set of tips:

• Remember that if a contestant in the FIRST contest is also taking part in the SECOND contest then you shouldn’t interview them after the first contest. This is to prevent the judges for the second contest hearing anything that might sway their opinion
• Make sure that no competitors wear any badges or other indication of rank or awards
• If you’re using borrowed lights or anything else that isn’t your club’s usual equipment then make sure that everybody who will use this or be affected by it is given a full demonstration beforehand to ensure there are no slip-ups or misunderstandings
• Prepare your questions before the contestant interview – don’t be reading the bio on stage
• If you’re handing out participation certificates then do it when you interview the competitors. This saves you having to call them all up to the stage again later on

There were also plenty of things that went RIGHT, and I picked up some tips myself:

• If you’ve finished interviewing the contestants and the counters haven’t finished calculating the result yet then call up one of your visiting dignitaries to talk to the audience. If you’ve got an Area/Division/District Governor in attendance then everybody would love to hear from them and this way you don’t have to tag on extra time for that at the end of the contest
• I think it’s reasonably well understood that you only announce third place if you have five or more competitors in a contest. The reason for this is to ensure that there are at least two contestants whose names are not announced. This makes sure that nobody knows who came last. However what do you do if you only have three contestants? You need a second place because you need to have a reserve in case the winner can’t represent your club at the Area contest. But if you announce second place then everybody knows who came last. The answer is that you only announce first place and then you discretely present the second place contestant with their certificate after the meeting has ended. That way you have your second place backup but the audience don’t know who came last. (That tip came direct from our current District Governor, Tom Jones, who was in attendance)

I mentioned earlier “be sure you have plenty of copies of all the forms”. Here’s a quick checklist of the forms I think you need for a contest:

• contestant eligibility & originality form
• contestant bio form
• a sheet to record the speaking order for both contests
• humorous speech judging form
• table topics speech judging form
• tie-breaker judging forms for both contests
• timing record form
• counters record form
• notification of winner form

I’ve got one final tip that I put into our Club Contest agenda when I last ran a contest and has now spread down the road to our neighbours. When you’re giving a prepared speech in a contest and you’re NOT speaking first, you gain an advantage over the first speaker as you have one minute of silence before you take the stage in which to gather your thoughts. This is the one minute that the judges are using to write their scores for the preceding speaker. With nobody speaking before them, the first speaker doesn’t get this time. In order to balance things out and make everything as fair as possible I added one minute of silence to the agenda before the first speaker speaks – I think it’s a nice little touch.

Earlier this week, Facebook announced their location sharing service “Facebook Places“. If you’re familiar with services like Foursquare and Gowalla then you’ll know what to expect here… when you’re out-and-about you ‘check-in’ at your destination – an action which lets your friends (and possibly other people) know where you are. Like a lot of social media broadcast services there’s bound to be a certain amount of ego associated with location sharing – “I’m out at an exclusive/expensive/exciting place and you’re not” but the services talk about the benefits to users such as discovering your friends are in the pub next door so you can meet-up and share a beer.

However there are differences between Foursquare/Gowalla and Facebook. Facebook has, time and time again, played fast and loose with its users’ information and privacy. The general vibe I get from Facebook is that they’re only providing a service in order to get data from you that they can exploit and/or sell. Over the last year I’ve gradually reduced the amount of information I have on there, the amount of information I add and the level to which I share it. I just don’t feel comfortable with the site – yet, with any social media service, you have to be where the people are in order to make the connections so I keep my account active. I tend to use it as a back-up service to Twitter… somewhere for conversations that can’t be constrained into 140 characters (which is kinda weird of me because, as someone pointed out the other day, my Twitter stream is unlocked so EVERYBODY can see that and do what the hell they want with it).

Facebook Places isn’t available in Canada yet but they HAVE enabled the privacy settings so you can go in there and preemptively set your privacy before it comes North of the border. This is important even if you don’t intend to use the service.

You need to click Account->Privacy Settings->Customize Settings. Then there are three different settings to adjust:

• “Things I Share->Places I check in” determines how widely your whereabouts will be broadcast. Do you want everybody to see where you are? Just your friends? Just a subset of your friends? Nobody at all? The most private you can get with this setting is to click on “Custom” and select “Only Me” from the list box

• “Things I Share->Include me in ‘People Here Now’ after I check in” will cause your presence to appear in lists of people at an establishment/event. Depending on Facebook’s implementation, this might override your “Places I check in” setting and let people outside your friends list see where you are. If you’re interested in your privacy you probably want to disable this setting

• “Things Others Share->Friends can check me in to Places” is super-important. It’s the geo equivalent of having other people tag you in photos. Worse in fact. If someone checks you in as being down the pub in the middle of the afternoon there’s nothing to show that you WEREN’T actually there. At least with a photo other people can look at the photo and SEE that you aren’t in it. If this option isn’t disabled then your whereabouts and your privacy are totally out of your control. I suspect all but the most cavalier of users will want to set this option to Disabled. Even if you never intend to use Facebook Places yourself, you should go and turn this setting off

Good luck and be careful out there.

Yes, it IS important to vacuum once in a while… who’d have guessed?

The first computers I ever built (and then disassembled and then rebuilt) never seemed to gather any dust. But over the last 5 years I’ve seen an increasing amount of dust gathering inside my computers’ cases. I have no idea why… it seems to have corresponded roughly to when I moved from the UK to Canada so maybe Canada is a dustier country

Over the last year I’ve seen the internal temperatures in my current desktop PC steadily rising:

Before cleaning

These are the Core0 – Core3 temps and two copies of the hard drive temp.

So last week I opened up the case and vacuumed it. I cleaned all the vents, sucked the dust off the fans, sucked all the dust bunnies out and cleaned the processor heatsink.

The heatsink was problematic. The PC has a Core2Quad processor (big heatsink) in an Apevia X-QPack mATX case (small case). As a result there’s not much space and the narrow nozzle of the vacuum cleaner was never going to get anywhere near the heatsink without me disassembling the case. Rather than do that, I used the poor man’s compressed air duster (ie I blew through a straw) and blew the dust out of the heatsink vanes then vacuumed everything up.

With everything reassembled and allowed to run for 24 hours, the temperatures dropped to:

Temperatures after cleaning

Those temperatures are Core0 – Core3, “CPU temp”, “motherboard temp” and the hard drive. [Actually, at time of writing, they're 5C below those numbers - but today's a much colder day]

Not sure where the “CPU temp” and “motherboard temp” sensors are being read from. The case has a front panel temperature LCD and two free-floating sensors which I’ve attached to the top of the hard drive and the chipset heatsink… but those are currently reading 26 and 47 degrees… so apparently not related.

To complicate things slightly, between taking the two sets of readings I also upgraded from Ubuntu 9.10 to Ubuntu 10.04 (more about that later). I’ve seen some reports on the internet from people who noticed the reported sensor temperatures dropping as a result of the upgrade. I did the upgrade a day before I cleaned and didn’t see any fall in reported temperatures afterwards so I don’t believe that that’s a factor in the improvement that I’m seeing.

So there you have it: a year’s worth of dust had clogged up my CPU heatsink and fans to elevate temperatures by approximately 15C. If it’s been over a year since you opened up YOUR computer’s case then maybe you should give it a spring cleaning too?

Last week I showed you how NOT to store your users’ passwords in your database: the biggest sin of all is storing them as plaintext and the ‘false sense of security’ solution is to apply a hashing algorithm to them.

We saw that we can use a common hashing algorithm (the algorithm I used is called MD5: http://en.wikipedia.org/wiki/MD5) to turn “donkey” into “9443b0fceb8c03b6a514a706ea69df0b” and I told you that there’s no programmatic way to turn that back into “donkey” – the hashing algorithm is one-way. However, if you did last week’s homework and pasted that ciphertext into a search engine you’ll have found you got many returns. Why?

A little history: when the commonly used hashing algorithms were created, they were designed to be computationally “expensive”. That means they take a lot of processor power (and hence time) to calculate. This was deliberate – a user only has to login occasionally so it didn’t matter if it took 2 or 3 seconds to check their password. The excellent side effect of this delay was that it prevented a hacker from trying to guess your password by brute-force. Even assuming you’d been silly and used a dictionary word as your password, a hacker couldn’t break into your account by trying every word in the dictionary as he’d be there for a very long time. A quick calculation with my machine’s dictionary says, taking 3 seconds per attempt, it would take 3.4 days to attempt every dictionary word. Unfortunately for hashing algorithms, computers have got very much faster in the last 20 years – even my little laptop can generate a hash in 0.04 seconds. Suddenly the time to run through the entire dictionary has shrunk to one hour and our apparent security has vanished.

Things get even worse though. If you have a dictionary word as your password and I have access to a hash of it, I can tell you your password in just 5 seconds. I paste the hash into a search engine – one click on “search” and I have your password. What’s happened is that hackers have done all the hard work up front – they’re already run entire dictionaries through the common hashing algorithms and they’ve posted the lists of words and hashes on the internet where search engines have found them and indexed them. So although it’s technically true that we can’t take a hash value and “unhash” it, hackers do have access to functionality that can perform a similar job – for single words.

“OK”, I hear you say, “but I’d never be stupid enough to just use a plain dictionary word as my password – I’ll put a number on the end of it”. Right then… that might help, but it might not… 8339e38c61175dbd07846ad70dc226b2 and 2484b2d1aec71de2ca87f88af401a6af are hashes of dictionary words with numbers on the end and both are indexed by Google (vote1234 and password99 in case you can’t be bothered checking). Although if your password is “aardvark50″ then you’re safe as its hash 0913c211b2eaa2a8b3b11fe53bdf9b4f doesn’t appear on the internet (until now of course because Google will index this blog post and your secret will soon be out!).

So how should we, as programmers, prevent our users’ passwords being cracked like this? The answer is surprisingly simple. We concatenate the password with some other information before we hash it.

The best approach is two-pronged. Firstly we concatenate with a fixed nonsense string eg “78g^&FB%V^&I” – this ensures that, however simple a password the user has entered, we’ve created something that’s pretty much guaranteed to never have existed as a string before in the history of the Internet. Secondly we also concatenate it with a piece of information that’s specific to that user on our site eg their username. This is just icing on the cake to make sure that the hashing is different for each user – so if two users use the same password then their hashes will be different. The procedure is the same as before: we apply this “super-hash” to the password that the user initially sets before we store it in our database and we apply the same “super-hash” to the user’s password attempt before we check it against the database entry.

So now, if user “smith” sets their password as “donkey”, the hash that we’re storing is the hash of “smithdonkey78g^&FB%V^&I”. Good luck finding an online hash dictionary that contains THAT!

Incidentally, my previous post is currently the second return on Google for “9443b0fceb8c03b6a514a706ea69df0b” (the hash of “donkey”) and I’ve actually had incoming traffic from that as a search term, so we KNOW that people are actually using search engines to crack hashed passwords like this. Consider yourself warned and make your code secure.