Save & Resume

Author Message
Mark 03/29/2006 07:02 am
Many web-managers ask that daily download limits be set. OEP provides this option. The problem is that I don`t know how to restart under the following circumstances:

1) download byte-count is reached
2) OEP quits (OEP seems to exit without a message or report - is that right?)
3) many of the directories are not populated.

When I restart the project, OEP starts building the queue all over again. In some cases, OEP has to build more than 100,000 entries which takes a lot of time. Is there a way of resuming from the existing queue or working from the exisiting map ignoring invalid links as necessary?

I don`t know if "Suspend To File" or "Resume From File" are the functions to use here. If they are, I don`t know how to use them.


Oleg Chernavin 03/29/2006 07:02 am

Yes, I need to add the automatic Suspend to file feature if a limit is being faced. I plan to add it in a form of a new line in the URLs field. Something like:

Would this be fine for you?

Best regards,
Oleg Chernavin
MetaProducts corp.
Mark 03/29/2006 07:02 am

It`s not often that a company works so closely with their customers. What can I say except thanks again. For what it`s worth, here`s my 2 cents.

On the left side of OEP`s main window, there is a listbox with three tabs: Project, Map, and Queue. The Project tab represents a list of projects listing the project specifications. The Map tab displays the directory tree built by OEP for a given project. The Queue tab displays the list of download tasks built by OEP for a given project.

Enhancement #1:

I would add a tab to Project, Map and Queue, namely, Log. Log would display a cumulative history of messages from OEP pertaining to this project, e.g. start times, end times, number of links found, number of bytes downloaded, etc.

Enhancement #2:

Conceptually, Suspend To / Resume From File" refers to the Queue tab i.e. the pending list of download tasks, "Save Map To / Download Using Map" refers to the Map tab.

1) on the Properties > Project Page: add a checkbox: "Save Map To: ".
2) next to the checkbox, add a textbox to capture the name of the Map File.
3) add a new option to the Download sub-menu "Download Using Map".
4) If the download was stopped prior to the map file being completely built, then "Download Using Map" actually invokes "Resume From File" so that OEP could finish building the map. If there is a way to easily update the map with updates found during subsequent passes of the web-site, then great - however, I would be happy to assume responsibility for restarting the project from scratch to capture changes.

I would be using this feature to re-load missing files from a site, to re-start after a system crash, to re-start after a specified limit has been reached, etc.

Frankly, I can`t think of anything else for you to do <g> but I`m sure that others can.

I`ll eagerly await future versions of OEP. I have already recommended it everyone that I can. How do I send you a customer testimonial - via email? There doesn`t seem to be a page to enter it in.

Oleg Chernavin 03/29/2006 07:02 am

Thank you for your kind words.

Your first suggestion is already there. You can turn on the detailed log using View | Messages | Log window. Right-click the window to start logging, change logging detail level, etc. You can also dock it to the main screen - either under the Internal browser or at the bottom of the screen.

To get details on the last Project download, click Home button on the Internal browser toolbar and browse to the desired Project.

Regarding the second suggestion - you may notice a small black down-arrow next to the Download button on toolbar. Click on it to invoke a menu with the Project update modes. In fact, they are the same that you see in the Project Properties dialog. They were simply reworded to be more clear to understand.

I hope this is what you wanted to find.

Regarding testimonials - we really collect them and publish on Product Description pages on our site. So you can send them to us by E-mail. We will be very glad to publish them.

Thanks again!

OK. I will also start working on the SuspendOnLimit feature now.

Mark 03/29/2006 07:02 am
At this point, I am convinced that you must have a time machine so that you can make features appear retroactively in my version of OEP. Barring that scenario, I don`t have any other plausible explanation as to how features just seem to appear in my copy of OEP <g>.

Seriously, I saw the log window option but nothing ever appears in it for me. I just did a download and it is still blank. However, I did find the Statistics sub-menu item which has exactly the information that I was suggesting you put in so thanks again.

BTW, I sent the confirmation that I received from the Isidor Awards to I hope that this helps. I told all of my friends to vote for you as well.


Oleg Chernavin 03/29/2006 07:02 am

Regarding the Log - it is not working until you uncheck Pause. This was done to prevent memory usage when it is not necessary.