how many of you block webarchive through htaccess or robots.txt?
MWpro
6:36 am on Nov 4, 2008 (gmt 0)
Is this generally a good idea, so that no one can have any proof of things that were on your websites years ago?
What is the preferable method for blocking?
phranque
7:55 am on Nov 4, 2008 (gmt 0)
the preferred method for preventing content archiving by well-behaved bots is the noarchive robots meta tag. a couple of recent WebmasterWorld threads have some useful information in that regard: [webmasterworld.com...] [webmasterworld.com...] the reason you might use this is time-sensitive content. there may also be IP related reasons for not allowing archive. you can't actually prevent someone from archiving content once they have gotten it.
