|
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
> So... you're trying to locate publicly-accessible hosts providing
> services other than HTTP[S]
No, I'm looking for the fastest method to locate publicly available http
servers for a particular domain. I have no interest in non-http services.
> (which you could just get by descending
> publicly-accessible web pages with an HTTP client)?
Doesn't this assume there is a link from the main domain to the sub
domain? Specifically that somebody took the time to link to the sub
domain via a web page? (Just because a link is not present doesn't
preclude the possibility that a server is public). Assuming they did, I
need to find that particular webpage from hundreds of thousands of
possibilities.
It seems inefficient to send a spider out for this purpose and query every
page I find for links to potential sub domains. I was hoping the DNS
query could be used to quickly find the answer. Given a choice I'd think
it much more neighborly to query one dns server one time vs umpteen
million http head requests.
Since the answer is, apparently, that I need a spider I ask again... any
good books on spiders? Is the Spider Hacks (O'Reilly) book any good? Has
anyone seen the new O'Reilly book?
Sean C. Sheridan
scs@CampusClients.com
Campus Party, Inc.
444 North Third St.
Philadelphia, PA 19123
(215) 320-1810
(215) 320-1814 fax
http://www.CampusClients.com
http://www.CampusParty.com
___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug
|
|