filter out URLs

Author Message
Stefan 07/04/2005 04:35 am
Is there any way to have OE create a list of URLs when spidering pages? (like the email extraction function you have implemented).

Thx

Stefan
Oleg Chernavin 07/04/2005 04:40 am
The list of the URLs to load is in the Queue. The URL that were rejected can be found in the Log.

Best regards,
Oleg Chernavin
MP Staff
Stefan 07/04/2005 04:45 am
Is there any way to save this only this list of URLs? I am actually looking for a way to only extract URLs from webpages for adding them to a database.

Thx

Stefan
Oleg Chernavin 07/04/2005 05:08 am
Unfortunately, there is no such feature yet. It will be not easy to implement, because OE would have to check every found URL against all other URLs stored in the file already. This will slowdown the work and consume a lot of memory.

Oleg.
Stefan 07/04/2005 05:11 am
Thank you for the info. I am now looking into other applications that might do what I need in this case.

Thx

Stefan
Oleg Chernavin 07/04/2005 06:32 am
OK. Please let me know what you find. If you will have no success, let me know and I will see what can be done.

Oleg.