Huge Commercial Web Site Download w/o the Commercial Aspect

McKenna
03/30/2015 12:31 pm
Hi, I'm planning to purchase an offline browser to download some recording sites for their databases purposes. Would any version of the Offline Explorer download, for instance http://www.arkivmusic.com/classical/main.jsp, without the cart, order, wishlist, etc. aspects, and only the find music lists which should occupy millions of pages? If so, how difficult would it be to set the software for such a task?
Oleg Chernavin
03/30/2015 07:04 pm
I browsed the site and I think that Offline Explorer Enterprise edition should manage to download it.

The settings - use http://www.arkivmusic.com/classical/main.jsp as the starting URL. Set Level to 10 first. Allow downloading from the starting server in URL Filters. Allow all directories in URL Filters.

URL Filters - Filename, add the following list to the Included keywords list:

album.jsp
-2
Drilldown?
label?
namelist?
musiclist?
albumgroup?
listpage.jsp?
/*composer*/*
/*conductor*/*
/*performer*/*
/*ensemble*/*

That should work.

Best regards,
Oleg Chernavin
MP Staff
McKenna
03/31/2015 10:45 am
Thanks for your swift response. I've tried it last night but after a while there were some 2million+ files on the queue and some 2GB of data accumulated without reaching the 4th level( main page-composer-beethoven-6th symphony, and more depth is "missing file"). I must be doing something wrong.
Oleg Chernavin
03/31/2015 07:39 pm
What links did you try to click from this "Symphony no 6" page? Maybe 2GBs of pages are not enough to get that amount of pages and data.

Oleg.