I`m using OE Pro 3.2.1708. I`m downloading a huge site to archive it locally. I started the download last night, and now want to suspend it to file so I can continue the download this evening. I expect it will take several days to get the entire site this way.
At the moment, I`ve downloaded 51,547 files and have 2,141,805 in the queue. I can suspend the download via the F9 key with no problem - the download stops immediately.
When I try to suspend the download to a file, the progress bar displays but the download doesn`t stop (It`s not just waiting to finish those files being downloaded - OE continues to download new files as an older one finishes) and no files are created in the directory that I select to hold the suspended files. I see no CPU activity that would indicate that OE is processing something internally. The progress bar does not increment. After waiting for several minutes with no apparent OE activity (other than the download continuing). I can cancel the suspend progress bar and OE keeps right on downloading with no apparent adverse effects.
I`ve tried suspending with F9 first, but suspend to file acts as indicated above (of course with no files downloading now), it just sits there with no disk/cpu/progress bar activity.
How can I get Suspend to File to work properly?
At the moment, I`ve downloaded 51,547 files and have 2,141,805 in the queue. I can suspend the download via the F9 key with no problem - the download stops immediately.
When I try to suspend the download to a file, the progress bar displays but the download doesn`t stop (It`s not just waiting to finish those files being downloaded - OE continues to download new files as an older one finishes) and no files are created in the directory that I select to hold the suspended files. I see no CPU activity that would indicate that OE is processing something internally. The progress bar does not increment. After waiting for several minutes with no apparent OE activity (other than the download continuing). I can cancel the suspend progress bar and OE keeps right on downloading with no apparent adverse effects.
I`ve tried suspending with F9 first, but suspend to file acts as indicated above (of course with no files downloading now), it just sits there with no disk/cpu/progress bar activity.
How can I get Suspend to File to work properly?
I will try to reproduce the problem to fix it. Anyway, you can simply stop the download now. When you want to continue, just select that Project and press Ctrl-F5. Since you are loading the site the first time, Offline Explorer will be able to go through all downloaded files and load only missing ones.
Best regards,
Oleg Chernavin
MP Staff
Best regards,
Oleg Chernavin
MP Staff
> I will try to reproduce the problem to fix it. Anyway, you can simply stop the download now. When you want to continue, just select that Project and press Ctrl-F5. Since you are loading the site the first time, Offline Explorer will be able to go through all downloaded files and load only missing ones.
>
> Best regards,
> Oleg Chernavin
> MP Staff
Thanks for the quick response. If I stop the download I will lose the 2+ million files in the queue, won`t I? Will OE recreate the queue from the files its already downloaded?
A bit more info- I`ve left OE sit for over an hour trying to suspend to a file (after suspending with F9). No activity, no files created, progress bar still at zero.
>
> Best regards,
> Oleg Chernavin
> MP Staff
Thanks for the quick response. If I stop the download I will lose the 2+ million files in the queue, won`t I? Will OE recreate the queue from the files its already downloaded?
A bit more info- I`ve left OE sit for over an hour trying to suspend to a file (after suspending with F9). No activity, no files created, progress bar still at zero.
One other thing. When I press the abort button on the suspend to file progress dialog, I get a confirmation dialog that asks "Do you wnat to abort?". I was unsure exactly what would be aborted (the suspend process? the entire download?). I`d like to suggest that the dialog message be changed to "Do you want to abort suspending to file?" or some other verbage to make this clear.
> If I stop the download I will lose the 2+ million files in the queue, won`t I?
> Will OE recreate the queue from the files its already downloaded?
Offline Explorer will recreate the queue from links in all downloaded files for the Project, so it should be quite safe to stop the download now.
I will also change the wording of that message. Thank you!
Oleg.
> Will OE recreate the queue from the files its already downloaded?
Offline Explorer will recreate the queue from links in all downloaded files for the Project, so it should be quite safe to stop the download now.
I will also change the wording of that message. Thank you!
Oleg.
> > If I stop the download I will lose the 2+ million files in the queue, won`t I?
> > Will OE recreate the queue from the files its already downloaded?
>
> Offline Explorer will recreate the queue from links in all downloaded files for the Project, so it should be quite safe to stop the download now.
>
It appears that OE is downloading all of the files again. After stopping the download the other day, I restarted it this evening. Before I started downloading, I checked the number of files in the download directory and found 33,607. OE`s status bar is telling me that it has downloaded 23,924 files, but the download directory still contains 33,607 files! When I stopped the download, I had over 2 million files in the queue, but the queue count is now 232,647 and increasing rapidly.
It appears that OE is downloading all of the files that it already downloaded again! When I started downloading this evening, OE did not appear to scan my existing files for links - it started downloading files immediately and the number of queued files has been increasing steadily as the download progressed.
> > Will OE recreate the queue from the files its already downloaded?
>
> Offline Explorer will recreate the queue from links in all downloaded files for the Project, so it should be quite safe to stop the download now.
>
It appears that OE is downloading all of the files again. After stopping the download the other day, I restarted it this evening. Before I started downloading, I checked the number of files in the download directory and found 33,607. OE`s status bar is telling me that it has downloaded 23,924 files, but the download directory still contains 33,607 files! When I stopped the download, I had over 2 million files in the queue, but the queue count is now 232,647 and increasing rapidly.
It appears that OE is downloading all of the files that it already downloaded again! When I started downloading this evening, OE did not appear to scan my existing files for links - it started downloading files immediately and the number of queued files has been increasing steadily as the download progressed.
Followup to my previous message.
I left OE running overnight, downloading the site. When I left my computer last night, OE`s status bar indicated that it had downloaded 33,700 files. When I returned to my computer today, OE`s status bar indicates that it had downloaded 93,269 files. My download directory contains only 33,794 files! The status bar indicates a size of just over 1GB. My download directory contains 389MB. Something`s not right.
Also, when I returned to my computer, OE had stopped downloading files. All 10 channels (HTTP0: thru HTTP9:) had a status message "Looking up host...". I checked to make sure my ISDN line was up (it was), then pressed F9 to suspend the download. After a few minutes, when all 10 channels indicated that they were suspended, I pressed F9 again to resume downloading, hoping that his would fix the problem. It didn`t. OE`s been sitting for half an hour since, still indicating that it`s "Looking up host...". I was able to use IE to browse to the site with no problem, so it`s not that the site is down.
I still was displaying these forum messages in OE`s internal browser. I tried to add this message to the discussion from within OE, but got an exception (Code: 0xC0000005, Flags: 0x00000000, Record: 0x0000000000000000, Address: 0x000000001A409F45). As an aside, have you considered using madExcept or some other exception reporting tool? I use this in my code - it provides much better exception reports, including a stack trace and thread information, and allows me to mail the reports directly to you.
I left OE running overnight, downloading the site. When I left my computer last night, OE`s status bar indicated that it had downloaded 33,700 files. When I returned to my computer today, OE`s status bar indicates that it had downloaded 93,269 files. My download directory contains only 33,794 files! The status bar indicates a size of just over 1GB. My download directory contains 389MB. Something`s not right.
Also, when I returned to my computer, OE had stopped downloading files. All 10 channels (HTTP0: thru HTTP9:) had a status message "Looking up host...". I checked to make sure my ISDN line was up (it was), then pressed F9 to suspend the download. After a few minutes, when all 10 channels indicated that they were suspended, I pressed F9 again to resume downloading, hoping that his would fix the problem. It didn`t. OE`s been sitting for half an hour since, still indicating that it`s "Looking up host...". I was able to use IE to browse to the site with no problem, so it`s not that the site is down.
I still was displaying these forum messages in OE`s internal browser. I tried to add this message to the discussion from within OE, but got an exception (Code: 0xC0000005, Flags: 0x00000000, Record: 0x0000000000000000, Address: 0x000000001A409F45). As an aside, have you considered using madExcept or some other exception reporting tool? I use this in my code - it provides much better exception reports, including a stack trace and thread information, and allows me to mail the reports directly to you.
It looks that the directory where you save files is overloaded. Perhaps, the site has too many simularily-named files. To overcome this, please enable directory overload protection in the Options dialog | File Locations section.
I would suggest you to find a folder with several thousand files in it and remove it.
Oleg.
I would suggest you to find a folder with several thousand files in it and remove it.
Oleg.
> It looks that the directory where you save files is overloaded. Perhaps, the site has too many simularily-named files. To overcome this, please enable directory overload protection in the Options dialog | File Locations section.
>
> I would suggest you to find a folder with several thousand files in it and remove it.
>
> Oleg.
I already have directory overload enabled.
The portion of the site that I have downloaded contains 5,300+ directories. None of them appear to contain more than 800 files.
I seem to be missing around 60,000 files! Where do you think they might be?
Also, what do you think about OE appearing to download all of the files again?
>
> I would suggest you to find a folder with several thousand files in it and remove it.
>
> Oleg.
I already have directory overload enabled.
The portion of the site that I have downloaded contains 5,300+ directories. None of them appear to contain more than 800 files.
I seem to be missing around 60,000 files! Where do you think they might be?
Also, what do you think about OE appearing to download all of the files again?
Do you have the "Process HTML forms" box checked in the Project Properties | Advanced? If the site contains HTML forms with multiple choices, it is possible that Offline Explorer loads various combinations of all possible selections on a form. This will really result in thousands or even millions of URLs to be downloaded. They may also look the same way, because POST requests do not show requested form parameters in a URL - they are passed invisibly.
If so, I would suggest you either turn that option off or to add the following line to the URLs field:
Additional=ConvertPOSTToFileName
This will force to name the resulting files (not URLs) in a unique way, taking the posted parameters as a filename part.
Oleg.
If so, I would suggest you either turn that option off or to add the following line to the URLs field:
Additional=ConvertPOSTToFileName
This will force to name the resulting files (not URLs) in a unique way, taking the posted parameters as a filename part.
Oleg.
> Do you have the "Process HTML forms" box checked in the Project Properties | Advanced? If the site contains HTML forms with multiple choices, it is possible that Offline Explorer loads various combinations of all possible selections on a form. This will really result in thousands or even millions of URLs to be downloaded. They may also look the same way, because POST requests do not show requested form parameters in a URL - they are passed invisibly.
>
> If so, I would suggest you either turn that option off or to add the following line to the URLs field:
>
> Additional=ConvertPOSTToFileName
>
> This will force to name the resulting files (not URLs) in a unique way, taking the posted parameters as a filename part.
>
> Oleg.
I don`t think there are any HTML forms on the site al all, but I may be mistaken. At any rate, I added the "Additional" line you recommended, and then ran OE for a few hours this evening. A few minutes ago, OE stopped processing files with a "Looking up host" message in HTTP0: thru HTTP9:. Every so often, I see the "looking up host" message change to "Waiting 1 sec" message, then back to the "Waiting for host" message, so it`s not locked up. I am able to log into the web site in IE6, so I know I have DNS resolution to the site and it is responding.
When OE stopped, it clamins to have downloaded 62,648 files. It has 2,784,462 files in the queue, and claims to have downloaded 501,128 KB. When I started downloading this evening, I had 33,794 files in my download directory, which occupied 389MB.
So, OE had downloaded 62,648 files but added only 402 files to my project; it has also downloaded 392MB, but added only 3MB to my project. And it`s lost it`s mind in the process (it can no longer find the host) . Since I can`t suspend to file, my only choice is to stop the download! I`m getting a bit frustrated with this. I hope you can help.
>
> If so, I would suggest you either turn that option off or to add the following line to the URLs field:
>
> Additional=ConvertPOSTToFileName
>
> This will force to name the resulting files (not URLs) in a unique way, taking the posted parameters as a filename part.
>
> Oleg.
I don`t think there are any HTML forms on the site al all, but I may be mistaken. At any rate, I added the "Additional" line you recommended, and then ran OE for a few hours this evening. A few minutes ago, OE stopped processing files with a "Looking up host" message in HTTP0: thru HTTP9:. Every so often, I see the "looking up host" message change to "Waiting 1 sec" message, then back to the "Waiting for host" message, so it`s not locked up. I am able to log into the web site in IE6, so I know I have DNS resolution to the site and it is responding.
When OE stopped, it clamins to have downloaded 62,648 files. It has 2,784,462 files in the queue, and claims to have downloaded 501,128 KB. When I started downloading this evening, I had 33,794 files in my download directory, which occupied 389MB.
So, OE had downloaded 62,648 files but added only 402 files to my project; it has also downloaded 392MB, but added only 3MB to my project. And it`s lost it`s mind in the process (it can no longer find the host) . Since I can`t suspend to file, my only choice is to stop the download! I`m getting a bit frustrated with this. I hope you can help.
Can you please send me your Project settings? I will try to reproduce the problem to see what is wrong. Please select the Project, click Copy button on toolbar and then paste it either to this forum mesage or in an E-mail to support@metaproducts.com
Thank you!
Oleg.
Thank you!
Oleg.
> Can you please send me your Project settings? I will try to reproduce the problem to see what is wrong. Please select the Project, click Copy button on toolbar and then paste it either to this forum mesage or in an E-mail to support@metaproducts.com
>
> Thank you!
>
> Oleg.
Oleg, I`ve just e-mailed it to you. Thanks for working with me on this!
-Leroy
>
> Thank you!
>
> Oleg.
Oleg, I`ve just e-mailed it to you. Thanks for working with me on this!
-Leroy
> I`m using OE Pro 3.2.1708. I`m downloading a huge site to archive it locally. I started the download last night, and now want to suspend it to file so I can continue the download this evening. I expect it will take several days to get the entire site this way.
>
> At the moment, I`ve downloaded 51,547 files and have 2,141,805 in the queue. I can suspend the download via the F9 key with no problem - the download stops immediately.
>
> When I try to suspend the download to a file, the progress bar displays but the download doesn`t stop (It`s not just waiting to finish those files being downloaded - OE continues to download new files as an older one finishes) and no files are created in the directory that I select to hold the suspended files. I see no CPU activity that would indicate that OE is processing something internally. The progress bar does not increment. After waiting for several minutes with no apparent OE activity (other than the download continuing). I can cancel the suspend progress bar and OE keeps right on downloading with no apparent adverse effects.
>
> I`ve tried suspending with F9 first, but suspend to file acts as indicated above (of course with no files downloading now), it just sits there with no disk/cpu/progress bar activity.
>
> How can I get Suspend to File to work properly?
>
I`m experiencing similar issues to that of `casterle` last June (i.e., suspend to file not responding and downloads continuing to run or hang). I`ve followed the procedures that were suggested by `Oleg` but the main issue of being unable to suspend to file is still present ...progress bar stays at zero. What was the ultimate solution after `casterle` sent `Oleg` his/her project settings?
Thanks, -Josh E
PS: I`m using OE 3.5.1904 and am also downloading a fairly large site (100,000s of files).
>
> At the moment, I`ve downloaded 51,547 files and have 2,141,805 in the queue. I can suspend the download via the F9 key with no problem - the download stops immediately.
>
> When I try to suspend the download to a file, the progress bar displays but the download doesn`t stop (It`s not just waiting to finish those files being downloaded - OE continues to download new files as an older one finishes) and no files are created in the directory that I select to hold the suspended files. I see no CPU activity that would indicate that OE is processing something internally. The progress bar does not increment. After waiting for several minutes with no apparent OE activity (other than the download continuing). I can cancel the suspend progress bar and OE keeps right on downloading with no apparent adverse effects.
>
> I`ve tried suspending with F9 first, but suspend to file acts as indicated above (of course with no files downloading now), it just sits there with no disk/cpu/progress bar activity.
>
> How can I get Suspend to File to work properly?
>
I`m experiencing similar issues to that of `casterle` last June (i.e., suspend to file not responding and downloads continuing to run or hang). I`ve followed the procedures that were suggested by `Oleg` but the main issue of being unable to suspend to file is still present ...progress bar stays at zero. What was the ultimate solution after `casterle` sent `Oleg` his/her project settings?
Thanks, -Josh E
PS: I`m using OE 3.5.1904 and am also downloading a fairly large site (100,000s of files).
This problem should be fixed already. Can you please try the latest version of Offline Explorer Pro by replacing the old oe.exe file with a newer version:
http://www.metaproducts.com/download/betas/oep1978.zip
Oleg.
http://www.metaproducts.com/download/betas/oep1978.zip
Oleg.