SEO Myth: Dynamic URLs Can’t be Crawled

Myth – BUSTED!

OK Realtors, let’s see a quick show of hands – how many of you have been told, overheard or just assumed that the search engines cannot crawl and index pages if the URL contains a question mark?  Don’t get nervous, my hand is up too.  How many of you think it’s true?  According to Google, this is one of the most prevalent myths there are concerning what search engines can and cannot do.

As is true of most myths, it is based in fact.  When the first robots began to crawl the web they were rather dumb and not able to follow URLs if they had a “?” in them.  Because of this, dynamic pages were not included in the search indexes.  Thankfully, this changed a few years ago as the robots began to mature.  Unfortunately for us, that information did not trickle down to all the web site builders and owners and the myth continued to flourish.

In a post on the Google webmaster blog, they state that if you have a URL that reads something like “…?entertainment=2&artist=14” they can follow it and index the resulting content.  However, they also say that having URLs like this is not recommended.  Google suggests it is wiser, and more reader friendly to use a keyword rich URL like “/entertainment/music/punk/the-ramones/ .”  Reader friendly content…where have I heard that before?  The suggested format, often referred to as “permalinks,” also looks much better in the Search Engine Result Pages (SERPs) and visitors are more likely to click on a link if they see what appears to be relevant information.

What Does This Mean to You?

If you’ve been worried about “fixing” your site so pages get listed…relax (a little).  They are probably already in the indexes and the content is helping you get a better placement in the rankings.  But, as you have read from me already (and will again), your goal should be the reader, not the robot.  Your click-thrus from searches will increase if you use human-speak in your URLs, not geek-speak.

Still Want to Fix Those URLs?

Good, I hoped you would.  When you create your new, reader-friendly URLs, you cannot just delete the old ones.  And, you also cannot just leave them alone either.  Hey, I never said that all SEO would be easy.

If you delete all the old ones you lose all of the inbound link credit you have earned, lowering your search rankings and certainly frustrating anyone that gets a “file not found” error when they try to follow a link to your site.

If you leave all the old links alone, then you’ll probably get penalized by the search engines for having duplicate content on your site.

Fixing the URLs correctly will probably require the help of a web geek to create what’s called a “301 redirect.”  This is special code that will take your visitors to the correct place if they follow a now incorrect inbound link, and also tells the search engines where the new file is located (and to update the location in the index).  Depending on the size of your site, this may take some time as it will need to be done for every URL you are updating. If you are using a Content Management System (CMS), like WordPress, it may be a bit easier for you. There are plugins available that allow you to create and manage your 301 redirects from within your WordPress admin (Hey Lani & Benn…did you see that?).

I don’t think I need to say it, but I will anyway to to be sure – BEFORE you make any changes that will effect your entire site in one fell swoop, make and verify a back up.

How Important is it to Update my Dynamic URLs?

On a scale of 1 to 10, I’d say updating your dynamic URLs is only about a 4.  The engines can read your content so you are getting the full benefit of the page from that perspective.  However, there is the possibility of a slight increase in ranking if you were to use the key word rich URLs as suggested by Google.  If feel you have done all you can already and need to squeeze out a bit more SEO to nudge past the competition, then yes – you should do this.

The best thing is to not let your site get out of control with the dynamic URLs to begin with.  If you are new to this, as some of you have said you are, talk to your technology providers and figure out up front how to use permalinks on your site.