File Count issue

[monterrey4ever]

Hi, is anyone having trouble with file count? Im getting a warning in cpanel regarding the number of files nearing the Excessive amount limit. However this warning is the result of an incorrect file count. It has happened once already, every file is being count twice. Last time I contacted support they re-ran the file count and it went back to normal ( around 90k files ), it is showing 180k+ files now and giving me a warning. Now Im totally paranoid, I dont want my account to get suspended all of the sudden for an error.
I thought it could be counting files inside the site backup compressed file. However I cant find the location of this file and delete it.
Support guy was nice and helpful last time but only resolved the situation temporary and I would like it to go away completely.

What do you think could be causing this?

[nitrocrzy]

sounds like a softlink causing you issues.... Support should have fixed that permanently when they found the problem. Sounds like time to chat with them again.
Bluehost doesn't back any site up over 4GB.

[cade]

Sorry to hijack this thread, monterrey4ever, I'll personally look into your issue and get back with you later this evening.

Bluehost doesn't back any site up over 4GB.

This is inaccurate.

Bluehost provides full, courtesy backups of every account regardless of size. There may be occasional kinks in backup generation, but there are no arbitrary size limits imposed. There was, however, at one time, a limit between 20-30 GB per account that was imposed for purely performance reasons (in essence, the system would get stuck generating backups for a 500 GB account and not make it through all users on the server in a timely manner). We've since overcome historical issues by largely throwing out CPanel's backup process and rolling our own solution that that monitors the file system for changes (via kernel patch), logs them, and then backs up changes to remote mounts during off periods.

I'm guessing that the 4 GB limit that nitrocrzy is referring to is actually the on-demand backup generation utility provided by CPanel-- we do limit the size of those backups as they tend to slam the file system (inevitably) during peak periods. There may be a solution to provide on-demand backups on the fly via queuing (in fact, our backup system has the capability) but it has yet to be implemented as a customer facing tool.

There is a delicate balance between performance and business priorities, one that the admin team at Bluehost takes very seriously. I know that they don't always sync up directly with what you need but I assure you that we are on your side.

[monterrey4ever]

Sorry to hijack this thread, monterrey4ever, I'll personally look into your issue and get back with you later this evening.

Thanks for taking your time to look into it.
I look forward to hearing from you about this.

[nitrocrzy]

Cade,

This was a statement made to me by BH support. When they started enforcing the 50K limit, I got caught out while on a vacation in Vietnam. I verbally granted support to delete any files they deemed a violation of the terms of service to restore my account. I got to thinking better of that and said to only delete site generated backup files in the form of .tgz or zip. Instead, support went thru and deleted all kinds of stuff, pictures, videos, and other site content. When I complained to support and requested everything to be restored. I was told "We don't back up any site over 4GB."

It's re-assuring that your telling me you guys have finally caught up with some of the newer technologies but because I was caught out before..... I'm going to continue to warn people not to trust BH is backing sites and be sure they have their own backups somewhere.

Sorry to hijack this thread, monterrey4ever, I'll personally look into your issue and get back with you later this evening.


I'm guessing that the 4 GB limit that nitrocrzy is referring to is actually the on-demand backup generation utility provided by CPanel-- we do limit the size of those backups as they tend to slam the file system (inevitably) during peak periods. There may be a solution to provide on-demand backups on the fly via queuing (in fact, our backup system has the capability) but it has yet to be implemented as a customer facing tool.

[videorob]

Having regular backups on your own machine is probably a good idea, anyway.

[nitrocrzy]

The cPanel states:

Reminder: The Terms of Service does not allow inappropriate use of our network for online storage. Please see "UNLIMITED" USAGE POLICIES AND DEFINITIONS for more information. Files uploaded in connection with your hosted site are always welcome.
Due to various causes, when your account grows larger than 30GBs or 50,000 files, our courtesy backup system no longer generates backups of any kind for your account. In order to restore the courtesy backups, you will need to bring your account below either 50,000 files or 30GBs. Please see our Terms of Service for more information.

So Cade.... according to this obscurely hidden remarks.. Sites are not backed up if they reach 50K files or if the site is over 30GB.

[Bob Barr]

The cPanel states:

Reminder: The Terms of Service does not allow inappropriate use of our network for online storage. Please see "UNLIMITED" USAGE POLICIES AND DEFINITIONS for more information. Files uploaded in connection with your hosted site are always welcome.
Due to various causes, when your account grows larger than 30GBs or 50,000 files, our courtesy backup system no longer generates backups of any kind for your account. In order to restore the courtesy backups, you will need to bring your account below either 50,000 files or 30GBs. Please see our Terms of Service for more information.

So Cade.... according to this obscurely hidden remarks.. Sites are not backed up if they reach 50K files or if the site is over 30GB.

From Cade's earlier reply:

There was, however, at one time, a limit between 20-30 GB per account that was imposed for purely performance reasons (in essence, the system would get stuck generating backups for a 500 GB account and not make it through all users on the server in a timely manner). We've since overcome historical issues by largely throwing out CPanel's backup process...

I suspect that those remarks were put in place at the earlier time Cade was referring to and just didn't get updated when Bluehost replaced the cPanel backup process with their own. (I've flagged Cade to this post for his confirmation whether or not this is the case.)

[monterrey4ever]

sounds like a softlink causing you issues.... Support should have fixed that permanently when they found the problem. Sounds like time to chat with them again.
Bluehost doesn't back any site up over 4GB.

I searched for symlinks in my home directory but didnt find anything unusual

[nitrocrzy]

Did you have a look at the file count report itself? I'd be interested in where all the files were being stored. I finally got around to looking at what happened to my site and it looks like Bluehost went through and deleted over 100K files without any notice whatsoever.... damm I'm getting tired of this.....