Re: [NTLK] Project Newtonberg?

From: Jon Glass (jonglass_at_usa.net)
Date: Mon Nov 12 2001 - 03:12:12 EST


on 11/12/01 1:41 AM, Grant [Tonic Bottle] Hutchinson at grant_at_splorp.com
wrote:

> Since the ISP that Rob had the Project Newtonberg site hosted with
> (aa.net) shut out _all_ robots and search engine spiders by using the
> robots.txt file, the Web Archive robots were never allowed to suck down
> and archive the pages from his site. The end result is that there are no
> copies of the site available anywhere (other than with Rob West himself).
> I checked Google's page cache as well... nada.

Aha. I understand now. I didn't know what the robot.txt file was. I assumed
it was a personal web archive. Sorry for being a newbie.

-- 
-Jon Glass
Krakow, Poland
<mailto:jonglass_at_usa.net>
<mailto:glasshaus5_at_aol.com>
"[The] law established by the Creator, which has existed from the beginning,
extends over the whole globe, is everywhere and at all times binding upon
mankind. . . . and is paramount to all human control." --Rufus King, signer
of the Constitution

-- This is the Newtontalk mailinglist - http://www.newtontalk.net To unsubscribe or manage: visit the above link or mailto:newtontalk-request_at_newtontalk.net?Subject=unsubscribe



This archive was generated by hypermail 2.1.2 : Sat Dec 01 2001 - 20:02:37 EST