Huge descr.wd3

Author Message
Janet 10/02/2008 02:06 pm
Hi, Oleg. Is there a way to disable the creation of descr.wd3 files in Offl Exp Ent. I know what these files are used for and they seem to be quite nonessential unless someone wants to update the project. The problems is that after I download 60000 small files (say 500 bytes each) in one folder the size of the descr.wd3 file becomes 6 MB. So when I keep downloading these small files in 7 channels the Offl Exp Ent stops responding for the reason that is quite obviuos. Offl Exp Ent makes Windows overwrite descr.wd3 file each time the new file is downloaded. And that happens every 1/20 of the second (actually that would have happened if descr.wd3 were small). In reality the download speed is around 1 tiny file per 3 seconds (that is the speed limit with which the system on ANY desktop would cope with the overwrite process). The problem is exacerbated by the fact that such a large number of files in one folder makes it virtually impossible to delete the descr.wd3 file manually once in a while (it takes 5 minutes for explorer or any other file manager to open such a folder).

If there''s no way to disable the creation of descr.wd3 files in the current version of Offl Exp Ent, I think it would be great if such an option appeared in future versions. Because one of the stongest points of Offl Exp Ent is the ability to download huge number of files, but this just collapses if many files are to be downloaded in the same folder. By the way, I''m not the only one who experiences this problem (it has been discussed on a bunch of forums).

Janet.
Oleg Chernavin 10/02/2008 04:07 pm
I think, it is not descr.wd3 files that slow down the download. Can you enable directory overload protection (Advanced - Files button)? If this will not help, I will add an option to skip using descr.wd3 and we will see if it helps to improve the speed.

Best regards,
Oleg Chernavin
MP Staff
Janet 10/04/2008 04:34 pm
> I think, it is not descr.wd3 files that slow down the download. Can you enable directory overload protection (Advanced - Files button)? If this will not help, I will add an option to skip using descr.wd3 and we will see if it helps to improve the speed.
>
> Best regards,
> Oleg Chernavin
> MP Staff


Thanks for the advice, Oleg. Well, enabling directory overload protection solved the problem. BUT WHAT WAS A PRICE!!! It took me a while to figure out was wrong. I hope you wouldn''t mind if criticize a little the way the directory overload protection is implemented. But let me start with pros of the method:
1. the method (the creation of folders that contain small number of files) is simple.
2. it solves the problem.

cons:
1. files are downloaded not where they should have been downloaded. That means if you partially download a project with, say, 40000 files and then start it again, all the files get downloaded all over again (even though ''do not download existing files'' is checked).
2. the links that are translated offline DO NOT point to the real location of the files but where they should have been downloaded (I have checked only the location of pictures. Clickable links may be ok). Consequently the project is not browsable (at least what concerns the pictures).
3. it is not easy to figure out what have happened to the files. They migth be in any of the folders.
4. not creating the descr.wd3 file would be soooo migth easier and more elegant.

Well, that is my opinion. Summing up I think that the current method is unnecessery sophisticated, messes up the location of the files and make the updating of the project impossible.

That would be great if there were an option to skip using descr.wd3.

Regards.
Janet.
Janet 10/04/2008 06:05 pm
I wanted to correct myself. Not all projects download the existing files again when they are updated. Some projects are ok, some are not. Frankly, I don''t what, but something does go wrong.

Could I mention some other annoying things/bugs in Off Expl En 5.1?
Janet.
Oleg Chernavin 10/05/2008 06:23 am
Yes, overload protection is not perfect. The links may stop working, because when I load HTML file, I don''t know which files will appear in the original directory and which - in overloaded ones.

This is currently solved by two ways - internal server automatically feeds correct files to Web browser when you browse inside Offline Explorer. And Export feature also corrects such links when you copy downloaded files.

However in (1) case, Offline Explorer checks all overloaded directories for all previously downloaded files. If you have a Project that fails to do this, let me know more details, I will fix it shortly.

In any case, I added the way to skip creating descr.wd3 files. here is the updated oe.exe (Pro version):

http://www.metaproducts.com/download/betas/OEP2842.ZIP

Run it from command line:

oe.exe /SkipDescr

I am really open to critics, it helps me to improve Offline Explorer. So, please let me know any opinion - both negative and positive. I appreciate this.

Thank you!

Oleg.