Google is good. Too good. Every day, 10 hundred million billion searches are conducted on the website google.com. So many searches are conducted, it’s the main gate to the internet for almost everyone on the planet.

The problem is that even though Google is very very good, Google is and always will be an imperfect guardian of their gateway. Google, by their own admission, requires sites on the internet to follow certain rules they set if you wish to be listed in their search result pages.

In the old days of many search engines, optimization of your web pages was a fairly organic process. You listed keywords that matched what your site was about, you made sure the different search bots could access all your pages, you tried to keep things clean and simple and hoped for the best.

Today, everyone searches on Google. This means that you ought to be spending a lot of time thinking specifically about how to be placed highly and broadly on Google, if you would like Google to send millions of people to your site.

Unfortunately this means you need to do Search Engine Optimization. SEO is the worst thing ever invented. It’s destroying good web application development. Here’s why:

Let’s say you have no idea what SEO is and you just happen to decide to make a nice community site for people to talk to each other and publish articles about old cars. You sign up some activity with the community to talk to each other about cars and it’s slow but steady activity. Let’s also say that although you don’t have any idea what SEO is, you go ahead and follow those helpful web developer guidelines just to do the right thing.

After a while, if you are watching your servers, you will notice that Google will start sending people your way. At first it won’t be very many, but over time as your members publish more information on cars, link to your pages from the outside and link to other pages from the inside, Google will give you more love. You will one day wake up and notice that the visitors from Google are more numerous than from any other source. Sure a Digg link will have gotten you 50k hits last Thursday, but Google will reliably send you 10k hits every single day. And 10k daily this month will become 15k daily next month.

At this point you will look at your general efforts to promote your site. You will have spent a week emailing people and working the Digg system and coming up with something Digg-worthy for that one 50k Digg link. You may have spent 2 or 3 hundred dollars buying 1000 clicks on Google’s Adwords system. You will probably have spent a week developing cool new community features for your 500 users, maybe bringing in 100 more users. But over the space of a month, organic search referrals from Google will have referred 450,000 people to come see your site.

You will start to realize: Google drives traffic by the truckload. No one else can even come close to the amount of traffic you can get through Google. If you want a huge amount of traffic, the way to get it is not through community features, it is not through great writing and content development, but it is through optimizing the crap out of your site so that Google will send more and more searchers your way.

Now the most important thing to you is no longer, “how can I make my site better to use”. If your basic idea stays the same, the new features you make for the maybe 20% of people who will use them might bring in 500 new people a month. Also you will look at your previous marketing efforts: a week working on a story for Digg that you got lucky on and brought in 50k new visits. Hundreds and maybe thousands of dollars on Google Adwords for 1000 visits. Compared to those 450,000 Google search visits over the month, all of those other priorities shift.

Now you will read about SEO. The hundreds of crazy rules that Google uses to determine whether your site is good or not. Do you think having your site name in your page title is a good idea? Google doesn’t. Do you think that javascript widget you made for navigating your archives is really awesome, intuitive and innovative? Google disagrees, it thinks it’s a big black hole of nothing.

Now when you develop, your site will become oriented to how can this work for Google instead of to your original idea of your simple user community. And your user community might even die, but who cares, comparatively they are a tiny minority of your overall user base. You’re too busy dealing with scaling your servers to cope with the millions of hits coming from Google to care about those ten thousand monthly visits from loyal users.

That’s my story anyways. I originally designed SWiK to be a community site about open source, where people could come to help each other use and learn about using cool, useful and interesting projects. The small number of monthly visits from users interested in that idea very quickly became dwarfed by the 200,000 Google referred visits every day. That original idea morphed into two basic priorities: optimize our pages and links for Google, and make sure our incredibly overtaxed server doesn’t die serving that many visits, not to mention the 1 million pages requested every day by the search bots.

Prioritizing community building features that would bring in hundreds of visits versus Google features that would bring in hundreds of thousands of visits, what choice is there? I’m not really sure, but it seems like there has to be a better way.

If you want a good example of how Google optimization puts your development at odds with your users, check out this funny example of how Google would need to optimize their own page if they wanted to be found by their own search engine. It’s funny because it’s exactly true, and really unfortunate for what it means to build sites today under the reign of Google.

PS. Ironically I almost caught myself changing the title this post to better suit Google keywords. Did you know the more words you use in your blog titles, the less relevant each word becomes to a search? The search engine is stupid, it doesn’t know anything about the meaning of a page, it only knows how to quickly match words on the web to words you type into a box. Do I have to choose between dumbing down my writing for what a words-matching computer wants and writing what I think humans would prefer to read? What a crazy system this is, but it’s the reality of the internet.