how to prevent loading html file except for starting url

Author Message
Mark Greenberg 12/02/2011 09:39 am
Hi,
I want to add a large number of hmtl urls to a single project in order to explore and download all links from them, without a depth limit (such as java, redirected media files, etc.), but I don't want to explore any other htmls. Is there a way to do this?

Also, is there a way to automatically have added urls contain the username:password@... format.

I am dealing with a large number of pages, so I need a way to do this in batch, but I can't simply crawl the whole site
Oleg Chernavin 12/02/2011 10:41 am
Yes, sure. You may specify any number of URLs (even with password in the URL). Set Level to 1 and use URL Filters - Filename section to exclude most popular HTML extensions:

.php
.asp
.htm

and so on.

Best regards,
Oleg Chernavin
MP Staff