We saw that we can use a common hashing algorithm (the algorithm I used is called MD5: http://en.wikipedia.org/wiki/MD5) to turn “donkey” into “9443b0fceb8c03b6a514a706ea69df0b” and I told you that there’s no programmatic way to turn that back into “donkey” – the hashing algorithm is one-way. However, if you did last week’s homework and pasted that ciphertext into a search engine you’ll have found you got many returns. Why?

A little history: when the commonly used hashing algorithms were created, they were designed to be computationally “expensive”. That means they take a lot of processor power (and hence time) to calculate. This was deliberate – a user only has to login occasionally so it didn’t matter if it took 2 or 3 seconds to check their password. The excellent side effect of this delay was that it prevented a hacker from trying to guess your password by brute-force. Even assuming you’d been silly and used a dictionary word as your password, a hacker couldn’t break into your account by trying every word in the dictionary as he’d be there for a very long time. A quick calculation with my machine’s dictionary says, taking 3 seconds per attempt, it would take 3.4 days to attempt every dictionary word. Unfortunately for hashing algorithms, computers have got very much faster in the last 20 years – even my little laptop can generate a hash in 0.04 seconds. Suddenly the time to run through the entire dictionary has shrunk to one hour and our apparent security has vanished.

Things get even worse though. If you have a dictionary word as your password and I have access to a hash of it, I can tell you your password in just 5 seconds. I paste the hash into a search engine – one click on “search” and I have your password. What’s happened is that hackers have done all the hard work up front – they’re already run entire dictionaries through the common hashing algorithms and they’ve posted the lists of words and hashes on the internet where search engines have found them and indexed them. So although it’s technically true that we can’t take a hash value and “unhash” it, hackers do have access to functionality that can perform a similar job – for single words.

“OK”, I hear you say, “but I’d never be stupid enough to just use a plain dictionary word as my password – I’ll put a number on the end of it”. Right then… that might help, but it might not… 8339e38c61175dbd07846ad70dc226b2 and 2484b2d1aec71de2ca87f88af401a6af are hashes of dictionary words with numbers on the end and both are indexed by Google (vote1234 and password99 in case you can’t be bothered checking). Although if your password is “aardvark50″ then you’re safe as its hash 0913c211b2eaa2a8b3b11fe53bdf9b4f doesn’t appear on the internet (until now of course because Google will index this blog post and your secret will soon be out!).

So how should we, as programmers, prevent our users’ passwords being cracked like this? The answer is surprisingly simple. We concatenate the password with some other information before we hash it.

The best approach is two-pronged. Firstly we concatenate with a fixed nonsense string eg “78g^&FB%V^&I” – this ensures that, however simple a password the user has entered, we’ve created something that’s pretty much guaranteed to never have existed as a string before in the history of the Internet. Secondly we also concatenate it with a piece of information that’s specific to that user on our site eg their username. This is just icing on the cake to make sure that the hashing is different for each user – so if two users use the same password then their hashes will be different. The procedure is the same as before: we apply this “super-hash” to the password that the user initially sets before we store it in our database and we apply the same “super-hash” to the user’s password attempt before we check it against the database entry.

So now, if user “smith” sets their password as “donkey”, the hash that we’re storing is the hash of “smithdonkey78g^&FB%V^&I”. Good luck finding an online hash dictionary that contains THAT!

Incidentally, my previous post is currently the second return on Google for “9443b0fceb8c03b6a514a706ea69df0b” (the hash of “donkey”) and I’ve actually had incoming traffic from that as a search term, so we KNOW that people are actually using search engines to crack hashed passwords like this. Consider yourself warned and make your code secure.

But password storage is a dangerous area. Every month we hear about a high-profile web site being hacked into and all the user accounts made public together with their passwords. This is not good – especially as it’s fairly common for users to keep the same password across all of the sites they access.

Good web site security is about defence in depth. Yes, you set up security so that hackers hopefully can’t get access to your site’s files and databases. But you shouldn’t stop there. Any site whose password list has been published has made one other simple and easily avoidable mistake: they’ve made the mistake of storing their users’ passwords in plain text.

Storing passwords in plaintext is a dangerous mistake that’s easily avoided – there’s a much smarter way to do it and it involves something called a hashing algorithm. A hashing algorithm is a form of encryption which is ONE WAY i.e. you can convert the plaintext to the encrypted form (know as ciphertext) but you can’t convert it back again. For example, if you start with “donkey” and run it through a well-known hashing algorithm you end up with “9443b0fceb8c03b6a514a706ea69df0b”. In theory, there’s no easy way to go the other way and turn that back into “donkey”.

But how the heck does this help us with passwords? Surely we’re going to have to turn the encrypted password back to plaintext in order to check it? Nope – there’s a neater way of doing this.

When the user initially sets their password, we run the hashing algorithm on the plaintext password and generate a hashed version of it. We store that hashed version in our database. When a user attempts to login, we hash their password attempt and compare THAT to the ciphertext of the previously hashed password that we’ve stored. Because the hashing algorithm is repeatable, if the password they attempted to login with matches the password they setup originally, then the two hashed ciphertexts will match too and we’ll successfully validate their login.

There’s no excuse for not knowing about this design pattern – Unix & Linux systems have been handling user passwords in this way for the last 30 years.

When you implement a system like this, there’s one thing you CAN’T do. And that’s recover a password that a user’s forgotten. Remember that the hashing algorithm is one way. You can’t turn “9443b0fceb8c03b6a514a706ea69df0b” back into “donkey” when the user can’t remember their password. As a result of this you should, as a user, be very wary of any websites which offer to email you your password when you’ve forgotten it. If they can email your password to you then they’re not using a hashing algorithm to store it in their database and therefore their database is not secure should a hacker get access to it. As a programmer, if you use a hashing algorithm to safeguard your users’ passwords then all you can do if a user has forgotten their password is to generate a new one for them (or let them set a new one themselves). This is generally done via an email that you send them – either containing a new random password that you generated for them or (better still) a one-off link that gives them access to a special page on the website where they can set a new password.

OK then, we’re sorted are we? Everything’s secure and protected from the hackers? Unfortunately not.

DO NOT IMPLEMENT WHAT I’VE JUST DESCRIBED.

There’s a flaw and I’ll tell you in a few days what that flaw is. In the meantime, you might like to paste that ciphertext into your favorite search engine and wonder about what just happened.

I took the 12 week ‘XML For Web Applications’ COMP2899 course downtown. My first course at the downtown campus – it’s really nice there: very modern and shiny. Course was also excellent – interesting material… I never knew you could do so much with XML and there was so much XML capability built into every browser. I knew about basic XML, DTDs, XPATH and parsers already but the course also taught schemas, XSLT and web services – overall very interesting. The course required quite a lot of learning but the labs, assignments and tests were all very fair – basically just to show that you’d done and understood the lecture content. I got a 99% mark which I’m very pleased with – especially so because the final exam was closed book with no ‘cheat sheet’!

I took the XML course for several reasons. Partly because XML interests me – all applications need configuration data and free-format text files are a recipe for disaster. Partly because I’m trying to complete the Advanced Java Development Certificate program and none of the courses I still needed were running this term. There’s one required course which hasn’t run for at least 18 months! I emailed the part-time studies director and he recommended the XML course. The course isn’t on the Java program but apparently there’s a re-organization coming which will put it on there (although another 5 months have passed now and the XML course hasn’t been added to the Java program and the lost required course still hasn’t run).

Having looked at the XML course I noticed that it’s also part of the Web Application Software Development Certificate program. I looked at that program and was amazed… not only are all the courses on things I’m interested in, I’ve already done half of them! So now I have TWO goals.

Over the last year, I’ve come across small pieces of PHP in several places. Tweaking WordPress themes has exposed me to some, and the BCIT AJAX course has required writing some PHP to handle the server-side functionality but this has pretty much all been self-taught. So when I noticed that a PHP course, COMP1920, was part of the Web Development program and there was an accelerated version coming up, I signed up immediately.

The PHP course was really eye-opening. For starters, it was the standard 12 week syllabus condensed into 6 Saturdays – you do one ‘evening’ in the morning and the next ‘evening’ in the afternoon. The course itself started out at the basics as some of the students hadn’t even programmed before, let alone seen PHP. But with the workload doubled, I was very happy with the pace.

The course lecturer makes an incredible difference to any course and the PHP course reintroduced me to the best lecturer I’ve had at BCIT. Jason Harrison is a programmer’s programmer – he isn’t there to teach you the theory, the 20 different parameters you can use with a function, he’s there to teach you how to get results. Jason teaches the course as 80% programming and 20% business. One of the things that PHP is great for is rapidly developing web-based applications and so a lot of people make a lot of money from using it. It seemed that most of the students had signed up for the course with that in mind and so we were all as spellbound when Jason started offering advice about business strategy as when he introduced the fopen() function. Actually… maybe more so!

I’m used to lecturers emphasizing the evils of cheating and the need for students to complete their work on their own but Jason’s approach is the opposite. Yes, work that you hand in has to be written by you, but there’s nothing wrong with consulting other students for advice. After all, that’s what you’d do in the real world. In the PHP course, Jason takes it a stage further – there are sections of the course which you MUST complete with other students – some parts in pairs and the final assignment as a team. The final assignment was something I’ve never seen on a BCIT course. The brief was to form a team, research something related to the course material that might be of interest to the other students and then give a 30 minute presentation – complete with demonstration and class exercise.

The course work was great. Because we were working at double pace, the first half of the course was heavily loaded with labs to be submitted each week. The second half of the course had coding assignments, the final assignment and revision all falling over each other. This generated a terrific buzz – I was writing up our class exercise, struggling with PHP session management and guiding other students through their problems simultaneously. Again, just like the real world.

Everything came together wonderfully. The final coding assignment had two options: the easy option was marked to a maximum of 100%, the hard option was marked to a maximum of 115%. Unfortunately there was no overlap between the two projects… so you had to make a decision at the beginning and stick with it. I chose the hard option and got bogged down in session management for a bit but once I’d conquered that it came together well – I even had time to extend it beyond the requirements with a bit of personal flourish. Our presentation on email injection, form validation and CAPTCHA went very smoothly. I presented the class exercise on getting the other students to add a CAPTCHA test to an existing PHP form – went OK, most of the students managed to complete it and I think I answered all the questions well. It seems my Toastmasters experience showed through… I’d mentioned that I was in Toastmasters at the beginning of the course but not had any feedback. After my presentation I had THREE different people come up to me and ask me for more information. Because of the compressed timetable we had a short break after the presentations and then straight into the final exam, no time to rest on our laurels!

Overall I loved the course… content, lecturer, format all worked very well. Oh and I was very pleased with my mark as well… 100%

Jason also teaches an advanced PHP course but for some reason it’s only scheduled once a year. I’m itching to get on that course!