parsing status

Author Message
Rene 12/16/2005 02:36 pm

Is it possible to somehow make a parsing status dialog box, wich can be opened like the log window. Sometimes the downloading stops and parsing takes a long time. It would be handy to see if OEP is still working, or just hangs!

Thanks for all the effort and the new version!
Oleg Chernavin 12/16/2005 03:03 pm
You can use the Log to show parsing messages.

Best regards,
Oleg Chernavin
MP Staff
Rene 12/16/2005 03:53 pm
Hmmm, ok. I saw some parsing stuff in the log dialog.
What can I do when I`m downloading a website, and there`s nothing in the queue left to download, but it still shows me that it`s parsing 3 files??
Oleg Chernavin 12/16/2005 04:12 pm
It looks like it hangs. Please let me know, if that Project has Script Calculations box checked or not in the Properties - Advanced?

Rene 12/16/2005 04:14 pm
Nope, not checked!
Oleg Chernavin 12/16/2005 04:25 pm
Can you please try the latest version:

Please replace the old oe.exe file with the above one (unzip it first) and in the new version please uncheck the GZIP compression box in the Options dialog | Speed section. Click the OK button and try to download that Project which hanged before. Would the hang occur again?

Rene 12/16/2005 05:14 pm
That didn`t help. I installed version 3.9, same problem. So I guess nothing wrong in the new version. I deleted the project, not the files, and added it again. Same bad result. I moved all files and started downloading from scratch. Seems to work now!
Any idea what coul be the problem? Maybe file checking, crc etc?
Anyway, I`ll let OEP try to download and see what happens!
Oleg Chernavin 12/17/2005 04:42 am
I am more than willing to understand this myself. It is a rare problem and I cannot reproduce it.

Rene 12/17/2005 01:14 pm
The new download worked fine. I`ll do an update tomorrow and will let you know if it worked out!!
Rene 12/18/2005 02:52 pm
The update didn`t work. The first few parsing files are ok, but after a short while the parsing queue fills up.
When I tel OEP not to download Video files it all works out fine!?
Looks like it`s somehow checking for files wich takes too much time.

Also, very often buttons and text on the Connections Panel fade out to black while parsing!

The lay-out of the pages are mostly:

Page1 - Subject1
subject1-movie and subject1-pictures

Page2 - Subject2
subject2-movie and subject2-pictures

Page3 - Subject3
subject3-movie and subject3-pictures

etc. till approxamatly page 50
Oleg Chernavin 12/19/2005 09:55 am
Can you please turn on Log (Ctrl+W) and see what last file starts parsing? Please copy me its URL.

Rene 12/19/2005 10:58 am
Sorry Oleg, but I can`t give you the URL, it`s a member site.
But it tells me that the last file is a MOV file! After 5 minutes it goes on with parsing other MOV en WMV files:

QUEUE - 12/19/2005 4:49:39 PM - Parsing (54) http://www.***.com/members/mov/
QUEUE - 12/19/2005 4:50:25 PM - Parsing end.
QUEUE - 12/19/2005 4:50:25 PM - Parsing files added.

Is this correct? Shouldn`t OEP add these files to the queue, instead of parsing them?
Rene 12/19/2005 10:59 am
ps: the above file loaded pretty fast as you can see
Oleg Chernavin 12/20/2005 05:07 am
MOV files can contain links. This is why Offline Explorer parses them. I think, I should optimize that, anyway. Can you please tell me the size of exactly that MOV file?

Rene 12/20/2005 07:46 am
Allright, now I get it. So that`s also why it`s slowing down my computer in almost a non workable state.
Anyway, the MOV file is 150MB, and took about 5 minutes. I guess OEP is in a lot of trouble when it`s parsing 1 page that contains like 35 links to MOV and WMV files!!
Parsing speed is random though:
149MB MOV file: 1 minute (wich first was 5 minutes)
127MB MOV file: 4 minutes
181MB WMV file: 4 minutes
179MB WMV file: 18 seconds
Oleg Chernavin 12/20/2005 08:02 am
I think, I need to improve the code to parse only the MOV files that are less than 10 MBs. This will be a good solution.

Rene 12/20/2005 08:27 am
Maybe, and don`t forget the WMV files!

Maybe you can also put a check box in the file filter options to enable/disable checking for links in media files.
Also maybe doing some prioritizing, when e big media file is being parsed, pause downloading and parsing other files, so the queue doesn`t get overloaded.

I didn`t know though that MOV files can contain links. If it`s a link that shows Quicktime where the file can be streamed from, shouldn`t the MOV file be just a few kb`s?
Oleg Chernavin 12/20/2005 08:35 am
Yes, there were relatively small MOV files that contained the links. I don`t know if real big files will contain links or not. WMV files should not get parsed. At least, Offline Explorer checks their contents and if it is a real WMV file, it is not parsed.

Rene 12/20/2005 08:55 am
Ok, but as you can see it also took some minutes to "check" a WMV file.
Oleg Chernavin 12/20/2005 08:59 am
Does the link to a WMV file look like mms:// or http:// ?

Rene 12/20/2005 09:14 am
Oleg Chernavin 12/20/2005 09:21 am
OK. Optimized this as well.

Rene 12/20/2005 09:46 am
Ok, anxiously waiting for the new release!
Oleg Chernavin 12/20/2005 10:02 am
You can try it already:

Rene 12/20/2005 10:25 am
It looks like it`s faster. But again it hangs parsing a MOV file with a size of 215MB. Parsing now for 6 minutes.
Oleg Chernavin 12/21/2005 12:53 am
Can you give me the log lines about it?

Rene 12/21/2005 07:12 am
I sent you an email (!
please email me directly if you need more info concerning the website!
Oleg Chernavin 12/22/2005 04:24 am
OK. I already received the E-mail. I will work on it. Thank you!