To start with I am using OE Pro 5.4.2972.
I have a few questions regarding the above processes.
1. I read in another thread that the 'Suspend to File' option in my version doesn't work properly. In other words if you click the button whilst OE is downloading, any in-progress files will be lost. Is this true?
2. I also read that a work-around for this is to 'Suspend' the project first, then 'Suspend to File'. This will stop the project first thus preventing loss of data. Does this fix the problem?
3. Also, I noticed the last time I used it that there was little indication that the project had successfully been suspended. The screen just looked exactly the same as before. I was expecting to see an OK button or something. How do I confirm that my project has 'Suspended' or 'Suspended to File' so I can safely shut down my machine?
Many thanks,
Wozza.
I have a few questions regarding the above processes.
1. I read in another thread that the 'Suspend to File' option in my version doesn't work properly. In other words if you click the button whilst OE is downloading, any in-progress files will be lost. Is this true?
2. I also read that a work-around for this is to 'Suspend' the project first, then 'Suspend to File'. This will stop the project first thus preventing loss of data. Does this fix the problem?
3. Also, I noticed the last time I used it that there was little indication that the project had successfully been suspended. The screen just looked exactly the same as before. I was expecting to see an OK button or something. How do I confirm that my project has 'Suspended' or 'Suspended to File' so I can safely shut down my machine?
Many thanks,
Wozza.
4. Also in the View/Statistics window there is a green bar at the bottom. What does this show? Is it the percentage of the project that has finished downloading? If I start to suspend will it show the progress of the suspend?
1. I made an improvement of this feature a couple months ago, so it should be working properly in the latest 5.8 version. Can you please download and install it?
http://www.metaproducts.com/download/opsetup.exe
2. Yes, this will surely help. Because it waits for all current downloads to complete, but other files from the queue will not start the download.
3. The indication is simple - the Project will have a red "stop" sign on its icon. And when you select the Project, the status bar of the program will show something like "Suspended to file c:\dir\file.wdq - Last download from ... to ...".
4. The bar shows a progress of scanning the Project Map to display details about actual amount of files that belong to the Project. And the breakdown of Web pages, images and other files summary.
Best regards,
Oleg Chernavin
MP Staff
http://www.metaproducts.com/download/opsetup.exe
2. Yes, this will surely help. Because it waits for all current downloads to complete, but other files from the queue will not start the download.
3. The indication is simple - the Project will have a red "stop" sign on its icon. And when you select the Project, the status bar of the program will show something like "Suspended to file c:\dir\file.wdq - Last download from ... to ...".
4. The bar shows a progress of scanning the Project Map to display details about actual amount of files that belong to the Project. And the breakdown of Web pages, images and other files summary.
Best regards,
Oleg Chernavin
MP Staff
Thanks for your reply Oleg,
Regarding question 2, when you say 'other files from the queue will not start the download', do you mean I will never get the whole download once I have suspended to file?
I will explain my problem in more depth: the site I am downloading takes longer than a working day to download so it is critical I am able to suspend the project so I can continue where I left off the following morning. I work for a government department so the equipment has to be locked away every evening so I can't leave it running overnight.
Thanks,
Regarding question 2, when you say 'other files from the queue will not start the download', do you mean I will never get the whole download once I have suspended to file?
I will explain my problem in more depth: the site I am downloading takes longer than a working day to download so it is critical I am able to suspend the project so I can continue where I left off the following morning. I work for a government department so the equipment has to be locked away every evening so I can't leave it running overnight.
Thanks,
No. Suspend does this (not Suspend to File). It waits for all current downloads and then pauses all other downloads. Suspend to File is more equal to record the current queue state to a file and stop the selected Project download.
Oleg.
Oleg.
So can you tell me, is there currently a way to save the download state so I can start again the following day EXACTLY where I left off?
Yes, Suspend to File in the 5.8 version should do exactly what you need - record the queue and download state, so you can continue the next time. It should not require to Suspend before you do Suspend to File.
Oleg.
Oleg.
Thankyou, but just to confirm, when I 'Suspend' then 'Suspend to File' using my current 5.4, is this the same as using 'Suspend to File' on its own in 5.8? If so, there is no need for me to upgrade.
Sorry to keep asking these questions but I need to get permission from my bosses before I can download new software.
Thanks,
Sorry to keep asking these questions but I need to get permission from my bosses before I can download new software.
Thanks,
5.4 version has some issues - as I remember - if the parsing queue contained files to process, the Suspend to File could hang or not properly work with these files. This was fixed in 5.5 SR1 version.
Oleg.
Oleg.
OK it looks like I will have to upgrade then.
It is interesting you say it will 'hang'. What do you mean by that because I have just lost two days work. I came back to my machine and it said 'Offline explorer has encountered a problem and needs to close'. I had to click through a lot of these messages before it closed and it never told me what the problem was. This happened a few days ago aswell but I just restarted.
Not what I needed.
Is that the 'hang' you are talking about?
Thanks,
It is interesting you say it will 'hang'. What do you mean by that because I have just lost two days work. I came back to my machine and it said 'Offline explorer has encountered a problem and needs to close'. I had to click through a lot of these messages before it closed and it never told me what the problem was. This happened a few days ago aswell but I just restarted.
Not what I needed.
Is that the 'hang' you are talking about?
Thanks,
Not quite sure right now. As I remember, Offline Explorer showed a Wait... dialog while it was processing the files and it was hanging infinetely without inability even to Abort.
Oleg.
Oleg.
OK I have now upgraded to 5.8 and 'Suspend to File' seems better.
However, I have another question. Sometimes OE still will randomly crash for a reason I don't know. I have found that deleting the folder 'RECYCLER' every now and again will help get the download running again. Is this safe to do?
Thanks,
However, I have another question. Sometimes OE still will randomly crash for a reason I don't know. I have found that deleting the folder 'RECYCLER' every now and again will help get the download running again. Is this safe to do?
Thanks,
Yes, this is safe, but it may indicate some problem with the filesystem - can you please make a disk check?
I also made a new version that should create an error log once the crash happens - c:\download\oe.log.
Please download it here:
http://www.metaproducts.com/download/betas/oep3212.zip
Unzip the file and replace the old oe.exe file with the new one.
When it crashes, please post in the forum and send the log file to support@metaproducts.com. Thank you!
Oleg.
I also made a new version that should create an error log once the crash happens - c:\download\oe.log.
Please download it here:
http://www.metaproducts.com/download/betas/oep3212.zip
Unzip the file and replace the old oe.exe file with the new one.
When it crashes, please post in the forum and send the log file to support@metaproducts.com. Thank you!
Oleg.
Hi Oleg,
I performed the checkdisk and it did actually get rid of the errors.
However, I have now finished this job so I won't be sending you any error logs until next time in a month or two.
Thanks,
Wozza.
I performed the checkdisk and it did actually get rid of the errors.
However, I have now finished this job so I won't be sending you any error logs until next time in a month or two.
Thanks,
Wozza.
No problem! Whenever anything happens, let me know. Thank you!
Oleg.
Oleg.
Oleg,
Can you tell me why I keep getting the following issue?
I am part-way through my download, and I have just checked the data that has downloaded so far. My destination folder is called 'Scrape'.
When I look at it I seem to have this nesting issue where the Scrape folder is being copied to itself multiple times. I had this exact same problem last time. This is what it looks like:
E:\Scrape\Local Files\Scrape\Local Files\Scrape\Local Files
Within each Scrape folder I end up with copies of the folders that sit in my root folder E. The folders in E are:
Aug2010 - This is the master data
Working - This is the folder the download is being taken from. It is a shortened version of Aug2010
Scrape - This is the destination folder for the download
As you can see I am not downloading from the internet but from a folder on a hard-drive that contains the web site I need. The web-site is too big initially, so I shorten it into Working, then do the 'download' from that folder.
I have been through my project settings and I can see nothing that would even need to reference these root directory folders.
So why I they being copied in this way? I am sure this makes the project go even slower.
Thanks,
Can you tell me why I keep getting the following issue?
I am part-way through my download, and I have just checked the data that has downloaded so far. My destination folder is called 'Scrape'.
When I look at it I seem to have this nesting issue where the Scrape folder is being copied to itself multiple times. I had this exact same problem last time. This is what it looks like:
E:\Scrape\Local Files\Scrape\Local Files\Scrape\Local Files
Within each Scrape folder I end up with copies of the folders that sit in my root folder E. The folders in E are:
Aug2010 - This is the master data
Working - This is the folder the download is being taken from. It is a shortened version of Aug2010
Scrape - This is the destination folder for the download
As you can see I am not downloading from the internet but from a folder on a hard-drive that contains the web site I need. The web-site is too big initially, so I shorten it into Working, then do the 'download' from that folder.
I have been through my project settings and I can see nothing that would even need to reference these root directory folders.
So why I they being copied in this way? I am sure this makes the project go even slower.
Thanks,
By the way, Local Files is where the copy of the root folder ends up.
My URL in the Project Properties is set as follows (This is obviously the home page the download begins from):
file://E:/Working/defenceintranet/Military/Navy/default.htm
This is the Folder download directory:
E:\Scrape\
file://E:/Working/defenceintranet/Military/Navy/default.htm
This is the Folder download directory:
E:\Scrape\
I need to look at the files. Frankly, I didn't understand the issue. Can we arrange a TeamViewer session on your computer?
Oleg.
Oleg.
Unfortunately, the computer is not connected to the internet. I do have the version of OE that creates a log file, however. See my above posts. Would that help?
Otherwise i will just have to try and explain again. Please let me know what you want to do.
Thanks,
Wozza
Otherwise i will just have to try and explain again. Please let me know what you want to do.
Thanks,
Wozza
I need to see how it reproduces. Maybe you can have a small set of files that causes the same issue? It would be great if I could download an archive with such files and make a Project that "downloads" them here from hard disk to see what exact code makes the duplicate folders.
Oleg.
Oleg.
I only have one set of files that does it. Also, this is government work so there is no way I could let you see it anyway.
I will try and explain it again.
If I right-click the Default folder in the Project window and select Properties the 'Folder download directory' is set to:
E:\Scrape\
If I right-click the project folder that sits within the Default folder and select Properties, the Addresses (URLs) box is set to:
file://E:Working/defenceintranet/Military/Navy/default.htm
Now as far as I can make out neither of these settings references the other folders that sit in the root of my E hard-drive, but within 'E:\Scrape\' OE creates copies of all the folders. It does it twice so they are nested within each other. If you have an email I could send you an image showing you what it looks like.
So I'm not sure if I am even getting what I need because which of the three scrape folders is actually recieving the correct data?
Is there a setting somewhere that I may have ticked accidentally to make it do this?
I will try and explain it again.
If I right-click the Default folder in the Project window and select Properties the 'Folder download directory' is set to:
E:\Scrape\
If I right-click the project folder that sits within the Default folder and select Properties, the Addresses (URLs) box is set to:
file://E:Working/defenceintranet/Military/Navy/default.htm
Now as far as I can make out neither of these settings references the other folders that sit in the root of my E hard-drive, but within 'E:\Scrape\' OE creates copies of all the folders. It does it twice so they are nested within each other. If you have an email I could send you an image showing you what it looks like.
So I'm not sure if I am even getting what I need because which of the three scrape folders is actually recieving the correct data?
Is there a setting somewhere that I may have ticked accidentally to make it do this?
My project now seems to have stopped when there are still 170,519 files in the queue! I have a green down pointing arrow on the project title and the message at bottom left says downloading to make it seem that everything is still OK. However there is no activity in the main screen and the 'downloading' message has disappeared from the Download Progress bar.
I am beginning to think your software is just broken.
I am beginning to think your software is just broken.
After a shutdown and restart things have come back again. I have also opened the Log window so I can see stuff moving.
The logging has now stopped again and I have a blank logging window. Did it reach the 5000 line limit? I changed it to 500000 but it hasn't started again.
Any ideas?
Any ideas?
No, I don't think some setting matters. Only maybe please see if the Project Properties dialog - Parsing section has checked or unchecked Explore All Possible Directories? But I doubt it may influence.
Please send me the files set to reproduce such download. I will write you right now to the E-mail address you left in this forum.
Regarding the log - the logic is simple here - once it reaches the limit, it clears the log and starts again.
Oleg.
Please send me the files set to reproduce such download. I will write you right now to the E-mail address you left in this forum.
Regarding the log - the logic is simple here - once it reaches the limit, it clears the log and starts again.
Oleg.
Oleg,
Like I said to you before, I can't send you the files because it is government work. I could however send you a screenshot of the folder structure that is being created if you feel that would help.
Regarding the Log. That is not what happens on my copy. When mine gets to the end of the limit it simply stops and I can't get it going again. Not without a restart anyway.
Like I said to you before, I can't send you the files because it is government work. I could however send you a screenshot of the folder structure that is being created if you feel that would help.
Regarding the Log. That is not what happens on my copy. When mine gets to the end of the limit it simply stops and I can't get it going again. Not without a restart anyway.
The 'Explore all possible subdirectories' box was unchecked. I have now checked it to see if it helps.
No, "all directories" will not make a difference, I think. It is something in the links in the files. This is why I need some subset to reproduce.
Oleg.
Oleg.
I uninstalled the software and recreated the project from scratch. This time everything seems OK. I will let you know if I have any other problems.
This is good. But it would be great for me to know why it was failing before that...
Oleg.
Oleg.
I am now getting another problem. The 'Suspend Project to file...' is no longer working. Previously this would generate a file of many Mb in size. Now it creates a file of 1Kb, which doesn't work when I try to resume from it.
Any ideas?
Any ideas?
Is it with one Project or with any of them? I can send a version that logs what goes on to understand the reason.
Oleg.
Oleg.
I only have one project so I can't test others.
http://www.metaproducts.com/download/betas/OEP3250A.ZIP
Please replace oe.exe file with this one. Run the download, suspend it to file and send me the log file from c:\download\oe.log
Oleg.
Please replace oe.exe file with this one. Run the download, suspend it to file and send me the log file from c:\download\oe.log
Oleg.