cron job to download/save a webpage

Forum Moderators: phranque

Message Too Old, No Replies

cron job to download/save a webpage

animas

11:52 am on Aug 13, 2009 (gmt 0)

I am using godaddy hosting cron job.

I need to download/save a webpage(updated everyday) from a domain everyday and. The webpage should be named as DATE of download(eg. 2009-08-12.txt). Everyday pages will be save in a folder of my hosting account.

How to do this?

animas

3:35 pm on Aug 13, 2009 (gmt 0)

I used:

/usr/bin/wget –quiet –output-document /$HOME/html/folder/file.txt http://www.example.com/page.txt > /dev/null 2>&1

Here I am renaming page.txt as file.txt and putting it inside "folder".

But this doesn't seem to work. Any suggestion?

[edited by: phranque at 8:48 am (utc) on Aug. 16, 2009]
[edit reason] exemplified url [/edit]

JS_Harris

8:23 am on Aug 16, 2009 (gmt 0)

#1 - Set up a script/page that performs the exact task you want it to perform on page load.

#2 - Tell the cron job to load the page (give the page a hard to guess name too).

#3 - Since you're using godaddy you may need to wait for the cron job to become active (even if everything says it should be working fine it may still not run for an initial period of time, it will start working on its own eventually).

#4 - If it doesn't work with the parameters you specify contact godaddy and they will fix it on your behalf fairly quickly.

I've gone through the process of setting up an hourly cron job via Godaddy and the service was great but without their help there was nothing I could do to turn on the cron job, the right code did nothing. They turned it on without changing it.