I was recently downloading a site that had quite a few large files and wanted to pause the download immediately. Suspend to file WILL suspend immediately, BUT any partially downloaded files will be lost. A normal Suspend will not suspend until all the current files have been downloaded.
Can you improve the Suspend and Suspend to File features in OE so that it will suspend a download *immediately* while keeping any partially downloaded files. When this project is resumed, the download will continue from the same position (ie. using the partially downloaded files), much like the Pause/Resume feature of MetaProducts Download Express.
Can you improve the Suspend and Suspend to File features in OE so that it will suspend a download *immediately* while keeping any partially downloaded files. When this project is resumed, the download will continue from the same position (ie. using the partially downloaded files), much like the Pause/Resume feature of MetaProducts Download Express.
Frankly, I didn`t implement this, because Offline Explorer would keep its temporary files. Imagine that you do the Suspen to file frequently and there will be more and more such temp. files. The temporary directory will be simply oveloaded.
Best regards,
Oleg Chernavin
MP Staff
Best regards,
Oleg Chernavin
MP Staff
> Frankly, I didn`t implement this, because Offline Explorer would keep its temporary files. Imagine that you do the Suspen to file frequently and there will be more and more such temp. files. The temporary directory will be simply oveloaded.
> > Best regards,
> Oleg Chernavin
> MP Staff
But if they are stored in the temp folder then this folder often fills up with rubbish and so is emptied by the user reasonably regularly. If OE couldn`t find the partially downloaded file it would just start downloading it from the beginning again. I would like to see this as an option which the user could enable if they so wished. It could be disabled by default so the current behaviour would stay the same.
This is a big failing of OE because Suspends can take ages to actually suspend when there are a lot of downloads in progress over a slowish connection.
I hope you`ll consider it.
Thanks
> > Best regards,
> Oleg Chernavin
> MP Staff
But if they are stored in the temp folder then this folder often fills up with rubbish and so is emptied by the user reasonably regularly. If OE couldn`t find the partially downloaded file it would just start downloading it from the beginning again. I would like to see this as an option which the user could enable if they so wished. It could be disabled by default so the current behaviour would stay the same.
This is a big failing of OE because Suspends can take ages to actually suspend when there are a lot of downloads in progress over a slowish connection.
I hope you`ll consider it.
Thanks
I will think about it, but there are more problems. OE will have to keep an info on the last file modification and its previous size. So that if the file date or size changes on the server, it will have to redownload the file, rather than resume it.
Oleg.
Oleg.
The post by Tester reminded me about this feature and I was wondering if you had thought any more about this feature ?
Does Download Express not already keep the relevant information ? If so, then can`t you just use similar code for OE ?
As mentioned previously, this is a feature I`d very much like to see. If you have this feature disabled byu default, then any current users will not be affected by it.
Download Express keeps all these incomplete items in its list. Offline Explorer will be unable to show or track these items and they can be easily lost and kept permanently in the temp folder. Since this folder is being used by many applications, including Offline Explorer, this will only slow down the system.
This is my major concern about that.
Oleg.
This is my major concern about that.
Oleg.
> Offline Explorer will be unable to show or track these items and they can be easily lost and kept
> permanently in the temp folder.
Why can`t OE track these items ? For each project you will only have to keep a list of up to 500 partially downloaded items maximum (ie. the max. number of connections). Even with the maximum of 500 this would not be a vast amount of extra data to store with the project (eg. maybe 1K of info for every item).
When using this type of pause, you would essentially do a Suspend to file so the current state of the queue is kept. However, instead of asking the user to enter a filename, you should automatically associate it with the project. The list of info on partially downloaded items would also be associated with the project. This allows OE to keep control of partially downloaded items , thereby stopping the problem you mentioned of partially downloaded files being "lost" and littering the TEMP folder.
When resuming a paused download you would perform a type of resume from file along with reading in the list of partially downloaded items. Then before downloading each partial file simply do the checks to make sure the file hasn`t changed. If it has changed, just delete the partially downloaded file and start the download from the beginning, otherwise simply resume the file from the last postion.
NOTE: At the beginning of a download you would have to check if the server supports the ability to resume file downloads (like Download Express). If it can`t, then when a project is paused there is no point in keeping/storing the partially downloaded files (ie. just delete the partial files) or a list of partially downloaded items.
What do you think ?
> permanently in the temp folder.
Why can`t OE track these items ? For each project you will only have to keep a list of up to 500 partially downloaded items maximum (ie. the max. number of connections). Even with the maximum of 500 this would not be a vast amount of extra data to store with the project (eg. maybe 1K of info for every item).
When using this type of pause, you would essentially do a Suspend to file so the current state of the queue is kept. However, instead of asking the user to enter a filename, you should automatically associate it with the project. The list of info on partially downloaded items would also be associated with the project. This allows OE to keep control of partially downloaded items , thereby stopping the problem you mentioned of partially downloaded files being "lost" and littering the TEMP folder.
When resuming a paused download you would perform a type of resume from file along with reading in the list of partially downloaded items. Then before downloading each partial file simply do the checks to make sure the file hasn`t changed. If it has changed, just delete the partially downloaded file and start the download from the beginning, otherwise simply resume the file from the last postion.
NOTE: At the beginning of a download you would have to check if the server supports the ability to resume file downloads (like Download Express). If it can`t, then when a project is paused there is no point in keeping/storing the partially downloaded files (ie. just delete the partial files) or a list of partially downloaded items.
What do you think ?
OK. I will keep this in the to-do list. This should be probably made as - when a user stops a download, offer to keep the Project queue state and keep partial files.
Oleg.
Oleg.
Thanks Oleg. I look forward to this feature!
I would rather it was linked to the Suspend feature though instead of the Stop feature because that`s more intuitive (Stop means Stop, but Suspend implies that it will be resumed). If the user stops a download all partial files should be deleted along with the queue (ie. what OE currently does). If the user Suspend`s a download, then OE should offer to keep the Project queue state and keep partial files.
When the project is resumed (ie. the Download button is pressed) OE should give the option to resume or restart.
It should also be possible to set default values for the behaviour of the Download and Suspend buttons so you don`t have to keep on choosing the behaviour every time.
Thanks again!
I would rather it was linked to the Suspend feature though instead of the Stop feature because that`s more intuitive (Stop means Stop, but Suspend implies that it will be resumed). If the user stops a download all partial files should be deleted along with the queue (ie. what OE currently does). If the user Suspend`s a download, then OE should offer to keep the Project queue state and keep partial files.
When the project is resumed (ie. the Download button is pressed) OE should give the option to resume or restart.
It should also be possible to set default values for the behaviour of the Download and Suspend buttons so you don`t have to keep on choosing the behaviour every time.
Thanks again!
Yes, perhaps if the project was suspended previously, the Download should offer to start from the scratch or to continue.
Oleg.
Oleg.
What about an alternative,maybe simpler solution like
- suspend to file after all partial files have been fully downloaded??
Gunnar
> Yes, perhaps if the project was suspended previously, the Download should offer to start from the scratch or to continue.
>
> Oleg.
- suspend to file after all partial files have been fully downloaded??
Gunnar
> Yes, perhaps if the project was suspended previously, the Download should offer to start from the scratch or to continue.
>
> Oleg.
Yes, it is easy - just press F9 and when all downloads are complete, do the Suspend to File.
Oleg.
Oleg.
I`ve recently been trying to find a good website downloader and thought that I`d really found something useful with OE. The first website that I`ve tried to download shows that it is unsuitable for my purposes, or at least almost so, and I`m very disappointed because otherwise it has the best features I could find.
The problem is the subject of this thread and the ability to suspend and resume large downloads without having to start the file download completly over. While I can suspend OE I can`t close it to regain resources not can I shutdown the computer without having to restart the last downloading files completly again.
I understand the problem to be your choice of implementing "Suspend to file.." as an external file state which then isn`t monitored by OE and potentially the user. If you supported resuming the download, then the partial file "droppings" could pile up if the user frequently took advantage of suspending to different files the forgot about them.
But to protect all users is not a valid choice for a developer, you must let the strong survive. From a user perspective it is simple enough to add an option to resume the file downloads from the point of suspension. Default the option to restart the download rather than resume if you want to protect the user.
You might also consider an UI change with the suspend to file. You could leave your underlying code pretty much intact yet easily add a named suspend. Internally suspend would use what, today, is the filename to save and restore the state but you would save an association between the name and the disk state file. You could then have a view of suspended downloads, a much more user friendly approach because the user then does not have to remember the name and the location of a disk state file.
I congratulate you on having a very nice piece of software and while I find it does not match my current needs I wish you the best of luck in the future.
Thanks,
-dana
The problem is the subject of this thread and the ability to suspend and resume large downloads without having to start the file download completly over. While I can suspend OE I can`t close it to regain resources not can I shutdown the computer without having to restart the last downloading files completly again.
I understand the problem to be your choice of implementing "Suspend to file.." as an external file state which then isn`t monitored by OE and potentially the user. If you supported resuming the download, then the partial file "droppings" could pile up if the user frequently took advantage of suspending to different files the forgot about them.
But to protect all users is not a valid choice for a developer, you must let the strong survive. From a user perspective it is simple enough to add an option to resume the file downloads from the point of suspension. Default the option to restart the download rather than resume if you want to protect the user.
You might also consider an UI change with the suspend to file. You could leave your underlying code pretty much intact yet easily add a named suspend. Internally suspend would use what, today, is the filename to save and restore the state but you would save an association between the name and the disk state file. You could then have a view of suspended downloads, a much more user friendly approach because the user then does not have to remember the name and the location of a disk state file.
I congratulate you on having a very nice piece of software and while I find it does not match my current needs I wish you the best of luck in the future.
Thanks,
-dana
Thank you for the suggestion! Yes, the only way to know that a Project has been suspended to a file is to select the Project and look at the status bar below - it will show you the name of the file you suspended to.
I agree that we will have to add a screen that shows all suspended downloads for easier resume.
Regarding the site you cannot download - can you please tell me more about it? Perhaps, we will be able to find a way.
Oleg.
I agree that we will have to add a screen that shows all suspended downloads for easier resume.
Regarding the site you cannot download - can you please tell me more about it? Perhaps, we will be able to find a way.
Oleg.
Oleg,
Thanks for the reply. I`m still trying to find a mirror/downloader that works the way I`d like. I end up comparing them all to OE and except for the ability to resume, truely resume, OE still comes out on top.
I didn`t have a website that I physically couldn`t download. The problem was that the information I wanted was verbose such that it was taking days to download. (We recently moved and we`re stuck using 28.8K until the cable company adds internet service in Dec-Feb.. There`s no DSL in this rural community.)
I was forced to reboot my computer twice and both times I had to resume the download, restarting the download on a bunch of 2-10 M files. This ran overnight clearing a few of them off but then I had to reboot again around midday having to give up the download of 8-10 files in the 60-94% done range. Losing hours of download bytes is painful.
Many to most of the other spider downloaders I`m reviewing offer resume from last position.
Thanks,
-dana
Thanks for the reply. I`m still trying to find a mirror/downloader that works the way I`d like. I end up comparing them all to OE and except for the ability to resume, truely resume, OE still comes out on top.
I didn`t have a website that I physically couldn`t download. The problem was that the information I wanted was verbose such that it was taking days to download. (We recently moved and we`re stuck using 28.8K until the cable company adds internet service in Dec-Feb.. There`s no DSL in this rural community.)
I was forced to reboot my computer twice and both times I had to resume the download, restarting the download on a bunch of 2-10 M files. This ran overnight clearing a few of them off but then I had to reboot again around midday having to give up the download of 8-10 files in the 60-94% done range. Losing hours of download bytes is painful.
Many to most of the other spider downloaders I`m reviewing offer resume from last position.
Thanks,
-dana
I understand. Yes, the ability to stop a download of a file at 50% and continue it later from the same point is not added to Offline Explorer yet. The biggest issue we faced is that during huge downloads, Offline Explorer will have to keep too many incomplete files that will trash your temporary directory. It will be also necessary to have a way to view all these incomplete files and manage them.
Maybe we will add this in one of the future versions. Right now all our efforts are focused on 4.0 version development.
Oleg.
Maybe we will add this in one of the future versions. Right now all our efforts are focused on 4.0 version development.
Oleg.
Again, I believe that simply adding a view of the suspended partial downloads (the "suspend to files") provides the solution.
I know it`s not just as trivial as that but all the state information you need is there in the file or whatever persistent store you might use to replace the file.
You need to maintain a list of named suspensions.
You need to display the content of the named suspensions.
You need to allow a named to be deleted (as well as the partially downloaded files)
(You already have resume and suspend).
A would be nice is to also allow for pruning of the partial files in the suspended state.
Also, of course, is the code to resume a file download but here I assume you have that in the other products from which you can borrow.
Anyway, sorry to take up your time while your working hard on the new release but since you seem to be "relaxing" a bit by responding to emails and it`s been awhile since I did any major software design work I`d jump in a bit here.
I really like OE and I`ll definetly take a look when 4.0 comes out. (I might have purchased now but it seems like there isn`t an upgrade path, just a re-purchase of the product.) In the meantime I`ve purchased BlackWidow and DownloadStudio. Together they make a partial replacement solution for OE.
When I can finally get my local network provider to either put in cable modems or provide me with a satellite OE will probably be a very workable solution since I won`t have to suspend! Arg..... rural living is great as long as you don`t need things like the internet or cell phone coverage.
Take care and keep up the great work.
-dana
I know it`s not just as trivial as that but all the state information you need is there in the file or whatever persistent store you might use to replace the file.
You need to maintain a list of named suspensions.
You need to display the content of the named suspensions.
You need to allow a named to be deleted (as well as the partially downloaded files)
(You already have resume and suspend).
A would be nice is to also allow for pruning of the partial files in the suspended state.
Also, of course, is the code to resume a file download but here I assume you have that in the other products from which you can borrow.
Anyway, sorry to take up your time while your working hard on the new release but since you seem to be "relaxing" a bit by responding to emails and it`s been awhile since I did any major software design work I`d jump in a bit here.
I really like OE and I`ll definetly take a look when 4.0 comes out. (I might have purchased now but it seems like there isn`t an upgrade path, just a re-purchase of the product.) In the meantime I`ve purchased BlackWidow and DownloadStudio. Together they make a partial replacement solution for OE.
When I can finally get my local network provider to either put in cable modems or provide me with a satellite OE will probably be a very workable solution since I won`t have to suspend! Arg..... rural living is great as long as you don`t need things like the internet or cell phone coverage.
Take care and keep up the great work.
-dana
You convinced me that a suspended downloads manager should be a priority for 4.1 or 4.2 version. :-) Would you please remind me to work on that hard when 4.0 version is released?
Regarding the upgrade. We are sorry for not publishing on our site, but all people who purchase 3.9 version will be able to upgrade to any 4.1 version for free. In fact, they receive registration keys already that are working with all versions of Offline Explorer up to 4.9.
For other users who registered earlier, the upgrade is 50% of the regular price.
We will try to make this announcement on our site today.
Oleg.
Regarding the upgrade. We are sorry for not publishing on our site, but all people who purchase 3.9 version will be able to upgrade to any 4.1 version for free. In fact, they receive registration keys already that are working with all versions of Offline Explorer up to 4.9.
For other users who registered earlier, the upgrade is 50% of the regular price.
We will try to make this announcement on our site today.
Oleg.