BlueHost servers not Google friendly?

[CAFEdePR]

I'd like to know if the BlueHost servers have been set up to block Google bots from crawling and indexing the websites in their servers.
I have other websites in other servers that are quickly indexed by google and other search engines yet the only website I have a problem with getting indexed is the one I host with BlueHost. I use the same techniques for getting indexed in all my websites.
When I do a search for the following phrases: puerto rico coffee or puerto rico arabica coffee or puerto rico cafe which should produce the results showing the website http://www.cafedepr.com which I host with BlueHost the website in nowhere to be found in the results yet some of my other websites that I host in other servers show-up in the results.
Has anyone else have the same problem?
Thanks in advance
Felix

[WebJDC]

noh google has my site indexed and i get plenty of google bots visiting my webpage.

[areidmtm]

it takes about a month for google to crawl your site and for your results to be indexed in searches

[dkinzer]

I'd like to know if the BlueHost servers have been set up to block Google bots from crawling and indexing the websites in their servers.
I have other websites in other servers that are quickly indexed by google and other search engines yet the only website I have a problem with getting indexed is the one I host with BlueHost. I use the same techniques for getting indexed in all my websites.
My experience suggests not. I created a new domain name when I first came to BlueHost and was suprised to see Googlebot crawling it within several hours. In that time, I did nothing to promote the crawling other than to put some simple content there.

Besides that, most of us would leave BlueHost as quickly as possible if they were blocking Googlebot access. According to my robot log, Googlebot crawled my site as recently as 10:40am yesterday.

You can create a robot log (actually, a log of accesses to /robots.txt) fairly simply. Add something like the following to your .htaccess file:

# rule for collecting statistics on requests for robots.txt
RewriteRule ^robots\.txt$ /cgi-bin/robots.cgi?%{REQUEST_URI} [L,NC]

Then, create a file /cgi-bin/robots.cgi that contains something like the following:

#!/usr/bin/perl
$web_root = "/home/mydomain/";
$log_file = $web_root . "logs/robots.log";
$robots_file = $web_root . "public_html/robots.txt";

$remote_host = "$ENV{'REMOTE_HOST'}";
$remote_addr = "$ENV{'REMOTE_ADDR'}";
$user_agent = "$ENV{'HTTP_USER_AGENT'}";
$referer = "$ENV{'HTTP_REFERER'}";
$document_name = "$ENV{'QUERY_STRING'}";

open (FILE, $robots_file);
@TEXT = <FILE>;
close (FILE);

&get_date;

&log_hits("$date $remote_host $remote_addr $user_agent $referer $document_name\n");

print "Content-type: text/plain\n\n";
if ($user_agent)
{
print @TEXT;
}
exit;

sub get_date
{
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isd st)=localtime();
$mon++;
$sec = sprintf ("%02d", $sec);
$min = sprintf ("%02d", $min);
$hour = sprintf ("%02d", $hour);
$mday = sprintf ("%02d", $mday);
$mon = sprintf ("%02d", $mon);
$year = scalar localtime;
$year =~ s/.*?(\d{4})/$1/;
$date="$year-$mon-$mday, $hour:$min:$sec";
}

sub log_hits
{
open (HITS, ">>$log_file");
print HITS @_;
close (HITS);
}

Note that you must change the string value that is assigned to the variable $web_root (on the second line) to match the home directory of your site. You may also want to change the value of $log_file (on the third line) or create the 'logs' directory in your home directory.

[1darkangel]

As has been said already bluehost is google friendly. I have never had a problem with them not coming around. Infact its been quite the reverse. For a while I suspected googlebot of having an affair with my site since I was catching it there everyday, around 5 times a day at least. It made hit after hit after hit. Honestly google bot was using up a ton of bandwith just to itself.

[magpie2419]

As has been said already bluehost is google friendly. I have never had a problem with them not coming around. Infact its been quite the reverse. For a while I suspected googlebot of having an affair with my site since I was catching it there everyday, around 5 times a day at least. It made hit after hit after hit. Honestly google bot was using up a ton of bandwith just to itself.

I have the same problem my site is hit daily by Google bot and it eats my bandwidth, but at least I get to be No 1 on the search page when you type in the key words for my site.

I would just sit tight and wait, make sure you have good back links and good content, having a quick look at youre site it could be you will get penalized for the amount of key words you use, keyword spam is frowned upon by Google.

[bboysteele]

I'd like to know if the BlueHost servers have been set up to block Google bots from crawling and indexing the websites in their servers.
I have other websites in other servers that are quickly indexed by google and other search engines yet the only website I have a problem with getting indexed is the one I host with BlueHost. I use the same techniques for getting indexed in all my websites.
When I do a search for the following phrases: puerto rico coffee or puerto rico arabica coffee or puerto rico cafe which should produce the results showing the website http://www.cafedepr.com which I host with BlueHost the website in nowhere to be found in the results yet some of my other websites that I host in other servers show-up in the results.
Has anyone else have the same problem?
Thanks in advance
Felix
This link should help you! http://www.google.com/support/webmasters/bin/topic.py?topic=8458 You also need to have a robots.txt file in your root directory before google will truelly index your site. If you don't have this file then a lot of times google just skips your site. The link I posted to this will take you to google where you can submit your website. They will ask you to verify that the site is yours by adding a html file to your site that they will then see. This stuff is not that hard to do. Almost all search engines have a way to submit your site to get indexed. People on this forum need to stop complaining about not getting indexed by google and other search engines. They need to start to use these search engines to find answers to these simple questions. Type in "How to get indexed by google" and you get the answer within the first three posts.... It's not that Hard!

[CAFEdePR]

Thank you so much to all that have taken their valuable time to respond to my question.

To bboysteele: Yes I have done everything that you mention including verifying, sitemap, robots.txt and more. I happen to prefer Google over most other search engines and am pretty familiar with their techniques and had not had to resort to asking this question until now.

BTW, can you tell me if BlueHost web servers supports the If-Modified-Since HTTP header feature? This feature allows your web server to tell Google bots and others whether your content has changed since your site was last crawled. Supporting this feature saves you bandwidth and overhead.

Thanks,
Felix

[aceofspades]

My website got crawled pretty easily mostly becuase I have an adsense accound. But anyways.......google works fine. Just need to do some SEO.

[bboysteele]

BTW, can you tell me if BlueHost web servers supports the If-Modified-Since HTTP header feature? This feature allows your web server to tell Google bots and others whether your content has changed since your site was last crawled. Supporting this feature saves you bandwidth and overhead.

Thanks,
Felix
I haven't heard anything in regards if Bluehost supports this feature. You probably need to contact them to find out or hope one of them answers this forum in their off time. When you find an answer, please let the rest of us know.

[thenewguy]

My website got crawled pretty easily mostly becuase I have an adsense accound.http://www.google.com/support/webmasters/bin/answer.py?answer=34428

[thenewguy]

When I do a search for the following phrases: puerto rico coffee or puerto rico arabica coffee or puerto rico cafe which should produce the results showing the website http://www.cafedepr.com which I host with BlueHost the website in nowhere to be found in the results yet some of my other websites that I host in other servers show-up in the results.I found you by searching
puerto rico arabica coffee
you were something like number 263 of about 50,200.

"Why doesn't my site show up for a specific keyword?":
http://www.google.com/support/webmasters/bin/answer.py?answer=34434

[tom.bcn]

Your hosting service doesn't really make any difference to your score in search engines. What makes a difference is having your site up for 1 month+, ensuring that you optimise your code for search engines, including a robots.txt file, using 'pretty URIs' (for blogs) and most importantly, by making sure other websites are linking to you.

[pmp6nl]

Googlebot is all over my site, so far this month I have had it hit my site 535 times. Just submit your site, have proper robot tags and let it go. Also you could try google sitemaps, it should help out.

[aurora]

I tried the script posted and I can't seem to get it to work.

Maybe my paths or folder locations are incorrect.

I looked in my root and I have an .htaccess there and one in public_html.

Which .htaccess should be modified and where should the log folder be?

sorry for the dumb questions.

[aurora]

This is what I've tried most recently.
I have the robots rewrite line in my .htaccess is the public_html folder
I have created a folder called "logs" and now have added an empty file named robots.log
both are chmod 666.
Nothing writes to the log file.

the paths all seem correct... what am I missing?

What permissions should be on the cgi file? I have it at 755.