Trying to download forum, but OE is downloading too much
- User Forums
- Offline Explorer Pro
- Trying to download forum, but OE is downloading too much
Author | Message | |
---|---|---|
Kols | 01/07/2011 09:15 pm | |
I'm trying to download a full forum, however the program is downloading too much irrelevant stuff that I don't need.
Basically all I want to download are the forum sections and all the threads. However this forum has many other links, such as: new posts, calendar, community, members, archives, blogs, etc...that I don't need. Also in each post it has this feature where you can click a buttom with a "thanks" link, and so what OE is doing is it's downloading each of the "thanks" in every single post of every thread. The structure of the forum is like this: Forum sections: www.website.com/forum/forumdisplay.php?f=123 Forum threads: www.website.com/forum/showthread.php?t=12345 I only want these two. Is there a way I can setup OE so that it only downloads these? If not is there a way I can setup OE so it doesn't download certain pages like for example the thanks links which look like this: www.website.com/forum/post_thanks.php?do=blablabla |
||
Oleg Chernavin | 01/08/2011 10:09 am | |
It is easy to do. Please use the Project Properties dialog - URL Filters - Filename section. Add the following keywords to the Included list:
forumdisplay.php showthread.php This should be enough. Best regards, Oleg Chernavin MP Staff |
||
Kols | 01/08/2011 10:13 am | |
Excellent, thank you. | ||
Oleg Chernavin | 01/08/2011 10:42 am | |
You are welcome!
Oleg. |
||
Kols | 01/08/2011 10:53 am | |
Well everything seems to be working well now, but I have one more question, this isn't that big of a deal but it'd still be nice to know how to filter this if possible.
I've set the file keyword filter up like you suggested and it works very well. I'm still getting a few pages that I don't really need like for example: forumdisplay.php?f=10&daysprune=-1&order=desc&sort=lastpost forumdisplay.php?f=10&daysprune=-1&order=desc&sort=voteavg forumdisplay.php?f=10&daysprune=-1&order=desc&sort=views forumdisplay.php?f=10&daysprune=-1&order=desc&sort=replycount forumdisplay.php?f=10&daysprune=-1&order=asc&sort=title forumdisplay.php?f=10&daysprune=-1&order=asc&sort=postusername forumdisplay.php?f=10&daysprune=-1&order=asc&sort=lastpost&pp=20&page=1 all I would need here is: forumdisplay.php?f=10 is there anyway to filter this as well? |
||
Oleg Chernavin | 01/08/2011 11:02 am | |
What about adding
daysprune to the Excluded list in the same section? Oleg. |
||
Kols | 01/08/2011 11:06 am | |
I tried that, didn't work.
However I paused or "suspended" the current download and THEN added the exclusion. Would I have to start from scratch for it to kick in? |
||
Oleg Chernavin | 01/08/2011 11:37 am | |
It is better to stop. Then select "Do not load existing files" in the Project Properties dialog. And start again. It will get only the missing files.
Oleg. |
||
Kols | 01/08/2011 12:11 pm | |
Yup, that did it. Looks like you can't suspend a project download, change options and have it start working with the new settings. Like you said you have to stop it first or just make a new project. But ya that worked. Thanks. | ||
Oleg Chernavin | 01/08/2011 12:42 pm | |
Actually, you can. And filtered out links will not appear in the queue. However the links that are already there will be not removed automatically.
Oleg. |
||
Kols | 01/08/2011 01:14 pm | |
Ah yes, thats sounds like what happened.
Thanks. |