Upload from URL

[lazywolfy]

A friend of mine was kind enough to lend me some webspace to store some files until I got a bigger host, which I do now (thx bluehost).

I now need to transfer a ton of files to my new site...about 4-5GBs worth of zip files, but instead of manually downloading the files, then uploading them thru ftp, is it possible to just upload them to my new site via url from their current location?
Thanks

//edit
Using the file manager, I selected Upload and tried adding a url into the upload field but the upload fails and it doesn't upload it correctly. Is there another way or is this even possible?
Thanks

[Early Out]

Yes, but only if you have shell access. From a shell prompt, you can cd to the target directory on your BH site, then grab your zip files from the other server:

wget http://www.example.com/firstzip.zip

[lazywolfy]

Cool I knew there had to be a way, thanks

....although I've never used shell access before. :confused: lol
Thats the "SSH/Shell Access" icon in the control panel right? It can't be too hard....I'll give it a try. Any tips or details you can offer would be helpful. Thanks

[Early Out]

Truth be told, I've never had occasion to use it, so I've never requested it. But yeah, you're looking in the right place. Depending on which box you're on (there are apparently some that are misconfigured), you'll have to scan your driver's license and upload it to BH before they'll give you shell access.

The thing to remember about shell access is that you're flying without a net. You won't get any "are you sure?" prompts - it'll do whatever it's told, even if it's a disaster! :)

Anyone have a link to a good cheatsheet of shell commands?

[lazywolfy]

BH support gave me this info but any idea as to which commands I need to use? :cool: lol
http://helpdesk.bluehost.com/kb/index.php?x=&mod_id=2&id=228


Common SSH Commands or Linux Shell Commands:

ls : list files/directories in a directory, comparable to dir in windows/dos.
ls -al : shows all files (including ones that start with a period), directories, and details attributes for each file.

cd : change directory · · cd /usr/local/apache : go to /usr/local/apache/ directory
cd ~ : go to your home directory
cd - : go to the last directory you were in
cd .. : go up a directory

cat : print file contents to the screen
cat filename.txt : cat the contents of filename.txt to your screen

tail : like cat, but only reads the end of the file
tail /var/log/messages : see the last 20 (by default) lines of /var/log/messages
tail -f /var/log/messages : watch the file continuously, while it's being updated
tail -200 /var/log/messages : print the last 200 lines of the file to the screen

more : like cat, but opens the file one screen at a time rather than all at once
more /etc/userdomains : browse through the userdomains file.
hit Space to go to the next page, q to quit

pico : friendly, easy to use file editor
pico /home/burst/public_html/index.html : edit the index page for the user's website.

vi : another editor, tons of features, harder to use at first than pico
vi /home/burst/public_html/index.html : edit the index page for the user's website.

grep : looks for patterns in files
grep root /etc/passwd : shows all matches of root in /etc/passwd
grep -v root /etc/passwd : shows all lines that do not match root

touch : create an empty file
touch /home/burst/public_html/404.html : create an empty file called 404.html in the directory /home/burst/public_html/

ln : create's "links" between files and directories
ln -s /usr/local/apache/conf/httpd.conf /etc/httpd.conf : Now you can edit /etc/httpd.conf rather than the original. changes will affect the orginal, however you can delete the link and it will not delete the original.

rm : delete a file
rm filename.txt : deletes filename.txt, will more than likely ask if you really want to delete it
rm -f filename.txt : deletes filename.txt, will not ask for confirmation before deleting.
rm -rf tmp/ : recursively deletes the directory tmp, and all files in it, including subdirectories. BE VERY CAREFULL WITH THIS COMMAND!!!

last : shows who logged in and when
last -20 : shows only the last 20 logins
last -20 -a : shows last 20 logins, with the hostname in the last field

w : shows who is currently logged in and where they are logged in from.

netstat : shows all current network connections.
netstat -an : shows all connections to the server, the source and destination ips and ports.
netstat -rn : shows routing table for all ips bound to the server.

top : shows live system processes in a nice table, memory information, uptime and other useful info. This is excellent for managing your system processes, resources and ensure everything is working fine and your server isn't bogged down.
top then type Shift + M to sort by memory usage or Shift + P to sort by CPU usage

ps: ps is short for process status, which is similar to the top command. It's used to show currently running processes and their PID.
A process ID is a unique number that identifies a process, with that you can kill or terminate a running program on your server (see kill command).
ps U username : shows processes for a certain user
ps aux : shows all system processes
ps aux --forest : shows all system processes like the above but organizes in a hierarchy that's very useful!

file : attempts to guess what type of file a file is by looking at it's content.
file * : prints out a list of all files/directories in a directory

du : shows disk usage.
du -sh : shows a summary, in human-readble form, of total disk space used in the current directory, including subdirectories.
du -sh * : same thing, but for each file and directory. helpful when finding large files taking up space.

wc : word count
wc -l filename.txt : tells how many lines are in filename.txt

cp : copy a file
cp filename filename.backup : copies filename to filename.backup
cp -a /home/burst/new_design/* /home/burst/public_html/ : copies all files, retaining permissions form one directory to another.
find * -type d|xargs -i cp --verbose php.ini {} : copies your php.ini file into all directories recursively.

kill: terminate a system process
kill -9 PID EG: kill -9 431
kill PID EG: kill 10550
Use top or ps ux to get system PIDs (Process IDs)

EG:

PID TTY TIME COMMAND
10550 pts/3 0:01 /bin/csh
10574 pts/4 0:02 /bin/csh
10590 pts/4 0:09 APP

Each line represents one process, with a process being loosely defined as a running instance of a program. The column headed PID (process ID) shows the assigned process numbers of the processes. The heading COMMAND shows the location of the executed process.

Putting commands together
Often you will find you need to use different commands on the same line. Here are some examples. Note that the | character is called a pipe, it takes date from one program and pipes it to another.
> means create a new file, overwriting any content already there.
>> means tp append data to a file, creating a newone if it doesn not already exist.
< send input from a file back into a command.

grep User /usr/local/apache/conf/httpd.conf |more
This will dump all lines that match User from the httpd.conf, then print the results to your screen one page at a time.

last -a > /root/lastlogins.tmp
This will print all the current login history to a file called lastlogins.tmp in /root/

tail -10000 /var/log/exim_mainlog |grep domain.com |more
This will grab the last 10,000 lines from /var/log/exim_mainlog, find all occurances of domain.com (the period represents 'anything',
-- comment it out with a so it will be interpretted literally), then send it to your screen page by page.

netstat -an |grep :80 |wc -l
Show how many active connections there are to apache (httpd runs on port 80)

mysqladmin processlist |wc -l
Show how many current open connections there are to mysql

mysqldump -u username -p dbname > file.sql
MySQL Dump

tar -zxvf file.tar.gz
UnTAR file

mysql -u username -p database_name <file.sql
Importing MySQL database

which [perl]
Finding path to [perl]

[Eriksrocks]

Yes, but only if you have shell access. From a shell prompt, you can cd to the target directory on your BH site, then grab your zip files from the other server:

wget http://www.example.com/firstzip.zip

Because it's connecting to an external server, will you need a dedicated IP for that? :)

[Early Out]

Because it's connecting to an external server, will you need a dedicated IP for that? :)AFAIK, no - it should work from a standard account.

[Early Out]

BH support gave me this info but any idea as to which commands I need to use?Takes me back - it's been 20 years since I lived on a *nix system (at the time, AT&T System V Release 2), but it's a bit like riding a bicycle - you never forget it entirely.

All you'll need to move your stuff from the other host is the cd command (pretty much the same as the DOS cd command), to get to the directory where you want the retrieved files to land, and the wget command I referred to above, which just says, "go get this file!"

[lazywolfy]

hmmm i scanned my license and now have access to ssh but i must be doing something wrong. I login using CONNECT USING JAVA TELNET APPLET (REQUIRES JAVA) and it brings up a black dos-like screen and at the bottoms it says conected to xx.xx.xx.xxx ssh and there is a green button that says ONLINE but I can't type in anything...it won't let me type anything at all. lol
am i doing something wrong?

[Early Out]

No, you're not doing anything wrong. I got curious, requested SSH access, and I've run into exactly the same thing - the Java Telnet Applet doesn't work, and the SSHTerm application won't let me log in.

I installed PuTTY (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html), however, and that works like a champ - no difficulties at all. If you're on a typical WinXP platform, grab the "installer," the 9th item on that linked page.

[Eriksrocks]

I think that's broken. It was broken at least when I used it. Use Putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/). :)

[lazywolfy]

yep...that's exactly what I did and it's working like a charm...a helluva whole lot faster than manually downloading and uploading thru ftp! lol
thanks for the help

btw...if this helps anybody else looking to do the same thing

I used:
cd /home1/user/public_html/path/to/directory/

and once i got to that directory I just used

wget http://www.examplesite.com/folder/filename

and it transfers the files super fast.

it's just like dos, if you get stuck type help or help [name] (where [name] is the command...or w/e u call it) lol

[Eriksrocks]

Sweet. Another useful tip I'll store away in my brain. ;)

[lazywolfy]

what about uploading an entire folder as opposed to the contents one file at a time? will that work with "wget"?? guess I'll try it out.

[Early Out]

You can zip up a bunch of files into one big file, move the file, then unzip it - the zip program will preserve the folder structure. Sometimes, you can get away with wildcarding, and grab all the files in a folder, but I wouldn't count on it.

Just did a quick search for wget wildcard, and found a bunch of threads in various other forums in which people try all sorts of stuff to make that work. Conclusion? Zip the files you want to retrieve! :)

[lazywolfy]

i did a search too and some topics said you could do such a transfer using "-scp" or some other commands as long as both accounts had ssh access (which my old server doesn't) , but if only one account had ssh access most people recommended zipping the files and extracting them on the new server using ssh (or something like that lol).

I went ahead and just did it the long way cuz i'm lazywolfy! :p
so far so good!

thanks for the help.