|Shaan||03/21/2007 04:18 am|
Downloading a private site. Options selected is "Within Server Only". No Levels selected. OE has d/ld practically a perfect copy of the site. Over 850MB. But quite a few of the links are not complete yet.
In the last 3 days...whenever I select the option to "Update Project" within a couple of minutes, OE freezes. Same thing usually happens when I select "Check Missing Links" but within 10-15 mins.
The site is a forum with posts and I have managed to get MOSTLY all posts till the 15th Mar. Update is not pulling in the rest. OE just freezes.
Have shut down all my spy-sweepers, anti-virus too. D/ld is on hi-speed line of over 6MB.
Any advice would be appreciated. Thank you for a wonderful product.
|Oleg Chernavin||03/21/2007 05:20 am|
|Do you use the latest 4.6 SR1 version? We fixed one bug before its release - it looks like what you are describing.
Also, please make sure that logging is not enabled (Ctrl+W to show the log window).
|Shaan||03/21/2007 05:29 am|
|Yep. Using SR-1. Logging is turned off. Well...I think it is turned off by default. Cuz when I get the logging window and select "update Project" there is no log generated in the window.
Any more advice would be appreciated.
|Oleg Chernavin||03/21/2007 06:03 am|
|OK. Can you please run Offline Explorer from the command line:
after it freezes, zip the log file and send to it to email@example.com.
|Shaan||03/25/2007 09:37 pm|
|Log sent as requested.
But I feel this program needs a serious overhaul for handling forums.
Today is the 4th day it has been running CONTINOUSLY to do an "Update Project". So far it has parsed 211112773 files. Downloaded showing as 21189567. Queue as 63353 files. Size as 27003kb. Speed is 0 b/s. And these numbers are still moving upwards!!!
So far...the size of the directory has gone over 1.1GB. CPU usage = 100%. This is on a AMD 3500 64. With 1GB ram. And I cannot do anything else cuz of the overload on the CPU. In the directory, I noticed that the program downloads everything as individual urls. So...every page that has the SAME info/layout is copied over and over and over again.
As a test...I manually downloaded the complete site just using the IE "Save As..." option and even though this took me time (a few hours compared to OE running for 4 days and not yet finished!!) the total size is less than 10MB. I agree this download is not a relection of the site but has been saved as individual pages but the speed and size in comparision are ridiculous.
Till date, OE has NEVER finished a complete download and informed me of the same.
Nice program...but I guess it is good for small sites with each page having diffeent info/layouts. Not cut out for a forum or bigger sites.
|Oleg Chernavin||03/26/2007 03:04 am|
|Yes, this is a problem when a site has different URLs for the same pages. Could you please give me several examples of such URLs that go to the same page? I will think on what can be done here.