Changing .htaccess to accomodate Googlebot

[LeeU]

I have a Googlebot that keeps trying to access a page I don't have but it is close to an RSS feed that I do have (the ISP is listed as Google). Would it cause a problem adding a redirection to the actual page for the Googlebot? (I have no idea where it is getting the incorrect link from.) I just want to make sure that I'm not doing something that could be potentially destructive.

[Pethens]

No, that's a bad idea. Redirecting Googlebot without redirecting everybody else will cause more 404s when Google directs a lot of traffic to a page that doesn't exist. Also, this technique could be considered a type of cloaking, which is a black hat technique that can get you penalized in the search engine rankings, even though your intentions are good.

On the other hand, redirecting everybody without checking their user agent would be perfectly fine and might solve your problem. I would also put some effort into tracking down where Google is getting the link. It has to come from somewhere. You can use Yahoo site explorer to check for backlinks. Keep in mind that if Google found a link to this nonexistent page, others may find it as well and you want them to find your content.

Hope that helps,

Stephen

[LeeU]

I guessed I phrased it wrong. I meant that I would put a general 301 redirection in the .htaccess file, not just for the Googlebot. You're right, of course. It would be a bad idea just for the Googlebot.

I tried the Yahoo and Google site searches and could find nothing. I even tried Xenu but to no avail. I guess I'll just add the redirection.

Thanks for the help.

[Pethens]

Oops, misunderstanding. ;)

[LeeU]

No problem. I appreciate your input.

[BlueHost-Reviews]

You can use 301 redirection to redirect older page to newer one through .htaccess . It will keep your pages updated in Google Cache .