Rip gallery from forum.

Author Message
Niklas 02/24/2007 10:01 am
Hi,
Forums such as http://forums.cgsociety.org/ has huge galleries of images that I would like to collect for myself (no really, it's just for myself because I use it for insperation and want to sort the images in folders), until now I have just pressed on every thread and manually downloaded the images but I quickly realized that is going to take forever.

So that's why I got interested in webspiders to do that automaticly for me.
But it turns out to be much harder than I though, let me try to explain:

I use this as my starting point: http://forums.cgsociety.org/forumdisplay.php?f=121 , the 3d gallery index. From here it takes me 1 clicks to get to the nearest image thread or two clicks if the thread is on page 2 or something, and another one to click on the image inside the thread to get the larger resolution image.

So I have set up Offline Explorer to go 3 clicks from the starting point. And only download images starting with the adress http://features.cgsociety.org/gallerycrits which is where all the gallery images are stored.
The saving filter part works great, it never saves anything unnessesary to my harddrive, the problem is just that it goes way off course to find these images (presses every link like Store, Ads, Members info etc) instead of staying inside the gallery forum which is where I know these images are, and thus takes incredible long time to get just a few images.

Is it somehow possible to limit offline explorer to only explore URL's that starts with http://forums.cgsociety.org/forumdisplay (pages 2,3,4 etc.) and http://forums.cgsociety.org/showthread ? That should mean that it stays on course.

Or if someone knows a completely different but better way would be nice :)
Oleg Chernavin 02/24/2007 10:10 am
It is easy to filter the files. Please use Properties - URL Filters - Filename section. Select Custom... and add to the Included list:

forumdisplay
showthread

File Filters - Images - Location box will be set to Load from all sites - this helps not to filter images.

Best regards,
Oleg Chernavin
MP Staff
Niklas 02/24/2007 02:43 pm
> It is easy to filter the files. Please use Properties - URL Filters - Filename section. Select Custom... and add to the Included list:
>
> forumdisplay
> showthread
>
> File Filters - Images - Location box will be set to Load from all sites - this helps not to filter images.
>
> Best regards,
> Oleg Chernavin
> MP Staff

Ooooh thanks for that!
I thought the file filters were only for which file to save to harddrive, not including which links to click, now it works thanks.
Oleg Chernavin 02/25/2007 09:52 am
File Filters and URL Filters determine which links to follow and download, while Content Filters work after downloading - they determine what to save and what to do if keywords are found inside loaded files.

Oleg.