I need help with the config

Author Message
Pablo Sebastian Fernandez 03/02/2007 07:52 am
Hi, i want download this Web:

http://cs.elderscrolls.com/constwiki/index.php/Main_Page

I want download this mini wiki complete, but only from http://cs.elderscrolls.com and not the others outside links. And i can do it, but i have a big problem, never finish the download, i have downloaded ~40.000 files and OE seis there are ~100.000 more files to download and that numbrer is uping.
Thanks to all
Oleg Chernavin 03/02/2007 08:06 am
Yes, you are downloading the entire site. Can you tell, what exact pages on the site you need?

Are all 140,000+ pages are the contents of the mini-wiki site or only some of them are really wanted?

Best regards,
Oleg Chernavin
MP Staff
Pablo Sebastian Fernandez 03/02/2007 08:22 am
> Yes, you are downloading the entire site. Can you tell, what exact pages on the site you need?
>
> Are all 140,000+ pages are the contents of the mini-wiki site or only some of them are really wanted?
>
> Best regards,
> Oleg Chernavin
> MP Staff


Well, i want all the site, but i dont think that site has that number of files, maybe i am wrong, the problems is when the OE is telling me that number of files it starts to act slowly. I want to know if i have a wrong config, maybe OE is repeating the files downloaded (i have marked dont load existing files). If u can check the files in that site, maybe i use a wrong url.

Thanks for answer
Oleg Chernavin 03/02/2007 08:52 am
Can you try to set Load from the starting directory in Project Properties - URL Filters - Directory. Also, use URL Filters - Filename to select Custom and add the following to the Excluded list:

category_talk

Oleg.
Pablo Sebastian Fernandez 03/02/2007 09:41 am
> Can you try to set Load from the starting directory in Project Properties - URL Filters - Directory. Also, use URL Filters - Filename to select Custom and add the following to the Excluded list:
>
> category_talk
>
> Oleg.


i can understand setting load from the starting directory, but what i am doing when i set the category_talk string in the filenames

thanks
Oleg Chernavin 03/02/2007 09:49 am
It disabled loading discussion pages - I think that too many pages can be in the forums on the site.

Oleg.
Pablo Sebastian Fernandez 03/02/2007 10:05 am
> It disabled loading discussion pages - I think that too many pages can be in the forums on the site.
>
> Oleg.


ok thanks, ill tray it, thanks to all
Oleg Chernavin 03/02/2007 02:47 pm
You are welcome!

Oleg.
Pablo Sebastian Fernandez 03/03/2007 12:01 pm
Hi again, well, actualy OE has downloaded ~175.000 files and continue downloading. I was checking what is downloading, and it isnt downloading external links, but it is loading "talk" links and i have the "category_talk" command added into filename, coueld anyone check for me an other command to block this type of links into the site

this is the site: http://cs.elderscrolls.com/constwiki/index.php/Main_Page
this is en exsample of talk topic: http://cs.elderscrolls.com/constwiki/index.php/Talk:AddTopic
(i taked it here: http://cs.elderscrolls.com/constwiki/index.php/Special:Recentchanges)

Thanks to all
Oleg Chernavin 03/04/2007 04:20 am
What about also adding

talk

to the Exclided filename keywords list?

Oleg.
Pablo Sebastian Fernandez 03/04/2007 05:09 am
> What about also adding
>
> talk
>
> to the Exclided filename keywords list?
>
> Oleg.

ok thanks ill try it, sorry for my ignorance in this kind of works, i dont know much about php and similars. Thanks :D
Oleg Chernavin 03/04/2007 09:39 am
This is fine - I am here to help.

Oleg.