Top Down View » php

Writing a Web App: Do Your Research

Jon Jennings — Thu, 10 Mar 2011 16:15:30 +0000

I still remember many a lunch spent with work colleagues during the early days of widespread internet adoption where we’d brain-storm a great new internet-related idea that would make us all millionaires. We’d come rushing back to our desks after lunch and immediately search to see if anybody had beaten us to the idea.

Invariably they had. At which point we would abandon the idea.

But a couple of years ago I took an interesting course at BCIT on web application development and business. The tutor talked about various ideas that he’d had and projects that he’d built on-line. His approach was the exact opposite of ours. He said that if he can’t find someone that’s already implemented something similar to his idea then he walks away. His rationale being that its presence validates his idea… someone’s out there making money from the idea. If there’s NOBODY out there offering the service then this shows him that there’s no money to be made from it.

Of course this isn’t always the case… every big idea has someone who does it first and definitely not every website out there is profitable. But if you’re just looking for a web app that you can turn out and make a little money on, seeing an existing market is a good indicator as to the idea’s validity.

Having identified an existing implementation, you shouldn’t simply offer the same thing… your implementation has to differ in some important aspect – something that would make someone choose your site rather than another one. If the competitor’s site is full-featured and complex then make your site simple and easy-to-use. If their site is simple and rigid then make yours flexible and customizable. If they don’t offer a mobile version then make sure you offer one. If they offer subscription levels for 2, 10, 100 (users, accounts, MB – depending on your business model) then you offer subscriptions at 5, 20, 50.

So, in our case, is someone already offering an online vehicle fuel economy tracker? A quick web search shows that yes there are several out there – there’s definitely interest in the service we’re planning. Some sites are definitely better than others but amongst the collection that I investigated, I found some nice simple interfaces, some mobile sites, some good data output and graphs – however not necessarily all on the same site.

One cool feature I had already thought of but was suprised to find implemented was “text message fillups” – register your cellphone and you can send in a text message to record a fillup. Not sure I can implement that without spending money – maybe a later addition!

Amongst the flaws I noticed:

use of Flash for graphics. Definitely have to make sure that’s avoided
SEO on all these sites appears to be very spotty. Depending on exactly how you word your search for a fuel economy tracking website, you get wildly different results with sites that might have been top of the list for one search vanishing entirely from the front page for another search

In summary, yes, there appears to be a market for this application. I’m not sure that I’ve identified a unique niche at this point but we’ve definitely found an idea that people want to use.

Our next stage is to think about the application design…

This post is part of a series – read them all

Writing a Web App: The Idea

Jon Jennings — Fri, 04 Mar 2011 22:07:26 +0000

The best ideas for web applications are usually ones where you identify a need of your own – something that you personally want.

Something I’ve done ever since I first owned a car is to record my gas mileage – how many miles I get on each tankful and hence how many MPG (or l/100km) I’m getting. It’s interesting to watch how the numbers vary according to different factors:

how much of the tank has been used for city driving and how much has been used on the highway
summer driving vs winter driving
times when you’ve been consciously driving economically vs times when maybe your right foot’s been a little heavier

When I bought my first car, I typed these figures into a spreadsheet. The spreadsheet grew to be quite sophisticated with different stats, averages and graphs. It gave me everything I wanted. In 2000 when I bought my beautiful Palm Vx I switched to an MPG tracking app on there which meant I always had the numbers with me, but at the cost of using a tool which wasn’t entirely to my liking, didn’t give me all the features I wanted and locked my data away from me.

These days I still record the numbers but I haven’t been tracking them with anything for years. I have a couple of envelopes packed with old gas receipts but I have no statistics based on those figures.

So, what I want is a website where I can enter my gas purchases and track my MPG. Simple.

With the idea in mind, the next stage is research…

This post is part of a series – read them all

Writing a Web Application

Jon Jennings — Thu, 17 Feb 2011 18:27:37 +0000

I’m going to write a new web application from scratch. Just for fun & with no intention of making my fortune from it (but it wouldn’t hurt if I did).

I’ll be writing it in PHP – possibly plain PHP or possibly using a framework. I’ve done some work in the past with the Zend Framework, CakePHP and Symphony and not felt entirely satisfied with any of them so, if I do use a framework, I’ll be using Code Igniter. I’ve heard good things about CI, talked to a couple of people who love it and worked my way through the tutorials without being put off yet… it seems to fill a nice middle-ground between CakePHP and Symphony.

To make life more fun, I’ll be documenting my progress here, talking about what I’m doing and sharing some of the code.

This post is part of a series – read them all

Protecting Your Users’ Passwords – Part 2

Jon Jennings — Sat, 10 Apr 2010 02:39:40 +0000

Last week I showed you how NOT to store your users’ passwords in your database: the biggest sin of all is storing them as plaintext and the ‘false sense of security’ solution is to apply a hashing algorithm to them.

We saw that we can use a common hashing algorithm (the algorithm I used is called MD5: http://en.wikipedia.org/wiki/MD5) to turn “donkey” into “9443b0fceb8c03b6a514a706ea69df0b” and I told you that there’s no programmatic way to turn that back into “donkey” – the hashing algorithm is one-way. However, if you did last week’s homework and pasted that ciphertext into a search engine you’ll have found you got many returns. Why?

A little history: when the commonly used hashing algorithms were created, they were designed to be computationally “expensive”. That means they take a lot of processor power (and hence time) to calculate. This was deliberate – a user only has to login occasionally so it didn’t matter if it took 2 or 3 seconds to check their password. The excellent side effect of this delay was that it prevented a hacker from trying to guess your password by brute-force. Even assuming you’d been silly and used a dictionary word as your password, a hacker couldn’t break into your account by trying every word in the dictionary as he’d be there for a very long time. A quick calculation with my machine’s dictionary says, taking 3 seconds per attempt, it would take 3.4 days to attempt every dictionary word. Unfortunately for hashing algorithms, computers have got very much faster in the last 20 years – even my little laptop can generate a hash in 0.04 seconds. Suddenly the time to run through the entire dictionary has shrunk to one hour and our apparent security has vanished.

Things get even worse though. If you have a dictionary word as your password and I have access to a hash of it, I can tell you your password in just 5 seconds. I paste the hash into a search engine – one click on “search” and I have your password. What’s happened is that hackers have done all the hard work up front – they’re already run entire dictionaries through the common hashing algorithms and they’ve posted the lists of words and hashes on the internet where search engines have found them and indexed them. So although it’s technically true that we can’t take a hash value and “unhash” it, hackers do have access to functionality that can perform a similar job – for single words.

“OK”, I hear you say, “but I’d never be stupid enough to just use a plain dictionary word as my password – I’ll put a number on the end of it”. Right then… that might help, but it might not… 8339e38c61175dbd07846ad70dc226b2 and 2484b2d1aec71de2ca87f88af401a6af are hashes of dictionary words with numbers on the end and both are indexed by Google (vote1234 and password99 in case you can’t be bothered checking). Although if your password is “aardvark50″ then you’re safe as its hash 0913c211b2eaa2a8b3b11fe53bdf9b4f doesn’t appear on the internet (until now of course because Google will index this blog post and your secret will soon be out!).

So how should we, as programmers, prevent our users’ passwords being cracked like this? The answer is surprisingly simple. We concatenate the password with some other information before we hash it.

The best approach is two-pronged. Firstly we concatenate with a fixed nonsense string eg “78g^&FB%V^&I” – this ensures that, however simple a password the user has entered, we’ve created something that’s pretty much guaranteed to never have existed as a string before in the history of the Internet. Secondly we also concatenate it with a piece of information that’s specific to that user on our site eg their username. This is just icing on the cake to make sure that the hashing is different for each user – so if two users use the same password then their hashes will be different. The procedure is the same as before: we apply this “super-hash” to the password that the user initially sets before we store it in our database and we apply the same “super-hash” to the user’s password attempt before we check it against the database entry.

So now, if user “smith” sets their password as “donkey”, the hash that we’re storing is the hash of “smithdonkey78g^&FB%V^&I”. Good luck finding an online hash dictionary that contains THAT!

Incidentally, my previous post is currently the second return on Google for “9443b0fceb8c03b6a514a706ea69df0b” (the hash of “donkey”) and I’ve actually had incoming traffic from that as a search term, so we KNOW that people are actually using search engines to crack hashed passwords like this. Consider yourself warned and make your code secure.

Protecting Your Users’ Passwords

Jon Jennings — Wed, 31 Mar 2010 20:15:33 +0000

I’m currently working on a PHP-based web site that stores member details – username, address, password etc in a database. Nothing unusual here… literally millions of web sites on the internet are doing exactly the same thing.

But password storage is a dangerous area. Every month we hear about a high-profile web site being hacked into and all the user accounts made public together with their passwords. This is not good – especially as it’s fairly common for users to keep the same password across all of the sites they access.

Good web site security is about defence in depth. Yes, you set up security so that hackers hopefully can’t get access to your site’s files and databases. But you shouldn’t stop there. Any site whose password list has been published has made one other simple and easily avoidable mistake: they’ve made the mistake of storing their users’ passwords in plain text.

Storing passwords in plaintext is a dangerous mistake that’s easily avoided – there’s a much smarter way to do it and it involves something called a hashing algorithm. A hashing algorithm is a form of encryption which is ONE WAY i.e. you can convert the plaintext to the encrypted form (know as ciphertext) but you can’t convert it back again. For example, if you start with “donkey” and run it through a well-known hashing algorithm you end up with “9443b0fceb8c03b6a514a706ea69df0b”. In theory, there’s no easy way to go the other way and turn that back into “donkey”.

But how the heck does this help us with passwords? Surely we’re going to have to turn the encrypted password back to plaintext in order to check it? Nope – there’s a neater way of doing this.

When the user initially sets their password, we run the hashing algorithm on the plaintext password and generate a hashed version of it. We store that hashed version in our database. When a user attempts to login, we hash their password attempt and compare THAT to the ciphertext of the previously hashed password that we’ve stored. Because the hashing algorithm is repeatable, if the password they attempted to login with matches the password they setup originally, then the two hashed ciphertexts will match too and we’ll successfully validate their login.

There’s no excuse for not knowing about this design pattern – Unix & Linux systems have been handling user passwords in this way for the last 30 years.

When you implement a system like this, there’s one thing you CAN’T do. And that’s recover a password that a user’s forgotten. Remember that the hashing algorithm is one way. You can’t turn “9443b0fceb8c03b6a514a706ea69df0b” back into “donkey” when the user can’t remember their password. As a result of this you should, as a user, be very wary of any websites which offer to email you your password when you’ve forgotten it. If they can email your password to you then they’re not using a hashing algorithm to store it in their database and therefore their database is not secure should a hacker get access to it. As a programmer, if you use a hashing algorithm to safeguard your users’ passwords then all you can do if a user has forgotten their password is to generate a new one for them (or let them set a new one themselves). This is generally done via an email that you send them – either containing a new random password that you generated for them or (better still) a one-off link that gives them access to a special page on the website where they can set a new password.

OK then, we’re sorted are we? Everything’s secure and protected from the hackers? Unfortunately not.

DO NOT IMPLEMENT WHAT I’VE JUST DESCRIBED.

There’s a flaw and I’ll tell you in a few days what that flaw is. In the meantime, you might like to paste that ciphertext into your favorite search engine and wonder about what just happened.

BCIT Term Wrapup

Jon Jennings — Mon, 10 Aug 2009 18:50:02 +0000

A good term at BCIT this Spring. A very good term, both in terms (oh dear!) of course enjoyment and in terms of marks.

I took the 12 week ‘XML For Web Applications’ COMP2899 course downtown. My first course at the downtown campus – it’s really nice there: very modern and shiny. Course was also excellent – interesting material… I never knew you could do so much with XML and there was so much XML capability built into every browser. I knew about basic XML, DTDs, XPATH and parsers already but the course also taught schemas, XSLT and web services – overall very interesting. The course required quite a lot of learning but the labs, assignments and tests were all very fair – basically just to show that you’d done and understood the lecture content. I got a 99% mark which I’m very pleased with – especially so because the final exam was closed book with no ‘cheat sheet’!

I took the XML course for several reasons. Partly because XML interests me – all applications need configuration data and free-format text files are a recipe for disaster. Partly because I’m trying to complete the Advanced Java Development Certificate program and none of the courses I still needed were running this term. There’s one required course which hasn’t run for at least 18 months! I emailed the part-time studies director and he recommended the XML course. The course isn’t on the Java program but apparently there’s a re-organization coming which will put it on there (although another 5 months have passed now and the XML course hasn’t been added to the Java program and the lost required course still hasn’t run).

Having looked at the XML course I noticed that it’s also part of the Web Application Software Development Certificate program. I looked at that program and was amazed… not only are all the courses on things I’m interested in, I’ve already done half of them! So now I have TWO goals.

Over the last year, I’ve come across small pieces of PHP in several places. Tweaking WordPress themes has exposed me to some, and the BCIT AJAX course has required writing some PHP to handle the server-side functionality but this has pretty much all been self-taught. So when I noticed that a PHP course, COMP1920, was part of the Web Development program and there was an accelerated version coming up, I signed up immediately.

The PHP course was really eye-opening. For starters, it was the standard 12 week syllabus condensed into 6 Saturdays – you do one ‘evening’ in the morning and the next ‘evening’ in the afternoon. The course itself started out at the basics as some of the students hadn’t even programmed before, let alone seen PHP. But with the workload doubled, I was very happy with the pace.

The course lecturer makes an incredible difference to any course and the PHP course reintroduced me to the best lecturer I’ve had at BCIT. Jason Harrison is a programmer’s programmer – he isn’t there to teach you the theory, the 20 different parameters you can use with a function, he’s there to teach you how to get results. Jason teaches the course as 80% programming and 20% business. One of the things that PHP is great for is rapidly developing web-based applications and so a lot of people make a lot of money from using it. It seemed that most of the students had signed up for the course with that in mind and so we were all as spellbound when Jason started offering advice about business strategy as when he introduced the fopen() function. Actually… maybe more so!

I’m used to lecturers emphasizing the evils of cheating and the need for students to complete their work on their own but Jason’s approach is the opposite. Yes, work that you hand in has to be written by you, but there’s nothing wrong with consulting other students for advice. After all, that’s what you’d do in the real world. In the PHP course, Jason takes it a stage further – there are sections of the course which you MUST complete with other students – some parts in pairs and the final assignment as a team. The final assignment was something I’ve never seen on a BCIT course. The brief was to form a team, research something related to the course material that might be of interest to the other students and then give a 30 minute presentation – complete with demonstration and class exercise.

The course work was great. Because we were working at double pace, the first half of the course was heavily loaded with labs to be submitted each week. The second half of the course had coding assignments, the final assignment and revision all falling over each other. This generated a terrific buzz – I was writing up our class exercise, struggling with PHP session management and guiding other students through their problems simultaneously. Again, just like the real world.

Everything came together wonderfully. The final coding assignment had two options: the easy option was marked to a maximum of 100%, the hard option was marked to a maximum of 115%. Unfortunately there was no overlap between the two projects… so you had to make a decision at the beginning and stick with it. I chose the hard option and got bogged down in session management for a bit but once I’d conquered that it came together well – I even had time to extend it beyond the requirements with a bit of personal flourish. Our presentation on email injection, form validation and CAPTCHA went very smoothly. I presented the class exercise on getting the other students to add a CAPTCHA test to an existing PHP form – went OK, most of the students managed to complete it and I think I answered all the questions well. It seems my Toastmasters experience showed through… I’d mentioned that I was in Toastmasters at the beginning of the course but not had any feedback. After my presentation I had THREE different people come up to me and ask me for more information. Because of the compressed timetable we had a short break after the presentations and then straight into the final exam, no time to rest on our laurels!

Overall I loved the course… content, lecturer, format all worked very well. Oh and I was very pleased with my mark as well… 100%

Jason also teaches an advanced PHP course but for some reason it’s only scheduled once a year. I’m itching to get on that course!