Hi. I`m starting to use Offline Explorer Enterprise since about a week, and have colleted the following questions:
1) Sometimes the download process finishes (or seems to be finished), with no download activity, but the icon indicator does not change to finished; also, the Report says it is still downloading. And stays there for hours.
2) What is the difference between "Download Missing Files" and "Update Project" ?
3) I don`t want to download pages that have not changed; then I have checked "Download only Modified and new files", and also marked "Check file size". However, after I finish the update of a site, I start a Update Download again to verify that nothing is to be downloaded, but ... it starts to download files again. And I don`t think they have changed so many files in such a short time!
4) Trying to find the optimum (faster) configuration:
- What is best: to set a "small" number of connections (10), or a "large" one (like 30 or 40) ?
- What is best: to download a single website at a time (sequence), or to fire many sites at once ?
Thank you,
JJ
1) Sometimes the download process finishes (or seems to be finished), with no download activity, but the icon indicator does not change to finished; also, the Report says it is still downloading. And stays there for hours.
2) What is the difference between "Download Missing Files" and "Update Project" ?
3) I don`t want to download pages that have not changed; then I have checked "Download only Modified and new files", and also marked "Check file size". However, after I finish the update of a site, I start a Update Download again to verify that nothing is to be downloaded, but ... it starts to download files again. And I don`t think they have changed so many files in such a short time!
4) Trying to find the optimum (faster) configuration:
- What is best: to set a "small" number of connections (10), or a "large" one (like 30 or 40) ?
- What is best: to download a single website at a time (sequence), or to fire many sites at once ?
Thank you,
JJ
1. Perhaps Offline Explorer processes downloaded files - please see in the Status Bar if there is a Parsing (XX) message.
2. Update Project checks all files whether they have changed on the server since the last download. Download missing files will not check all downloaded files - it will simply detect all links that were not yet downloaded and gets them.
3. Many servers have dynamic pages nowdays and they report them as just newly updated. You can easily recognize them - .asp, .aspx, .php, etc. This is a problem, but such servers may even not supply the file size.
4. From my experience, it doesn`t matter much on fast connections - whether you download 10 or 30 files at a time. Downloading several Web sites at a time is always better, because you balance load on several servers.
Best regards,
Oleg Chernavin
MP Staff
2. Update Project checks all files whether they have changed on the server since the last download. Download missing files will not check all downloaded files - it will simply detect all links that were not yet downloaded and gets them.
3. Many servers have dynamic pages nowdays and they report them as just newly updated. You can easily recognize them - .asp, .aspx, .php, etc. This is a problem, but such servers may even not supply the file size.
4. From my experience, it doesn`t matter much on fast connections - whether you download 10 or 30 files at a time. Downloading several Web sites at a time is always better, because you balance load on several servers.
Best regards,
Oleg Chernavin
MP Staff
Dear Oleg,
Thank you very much for your answers.
Now I will focus only on point 1). These are the symptoms I have:
- Yes, I have a Parsing (1049) message on the bottom left status bar.
- The download seems to be finishing, but never finishes: it has 30 threads,
27 with "Download Complete", 2 with "Connection Timed out", and 1 with "File
is probably incomplete, retrying".
- The "Queue" folder is empty.
- Seems that there is no internet activity, but the hard disk is working a lot.
- If I try to suspend the project, the status bar says "Suspending", and stays there.
- The site is ASP (www.lanacion.com.ar), and I`m downloading it with a level=3 deep.
Reards,
JJ
> 1. Perhaps Offline Explorer processes downloaded files - please see in the Status Bar if there is a Parsing (XX) message.
> > 2. Update Project checks all files whether they have changed on the server since the last download. Download missing files will not check all downloaded files - it will simply detect all links that were not yet downloaded and gets them.
> > 3. Many servers have dynamic pages nowdays and they report them as just newly updated. You can easily recognize them - .asp, .aspx, .php, etc. This is a problem, but such servers may even not supply the file size.
> > 4. From my experience, it doesn`t matter much on fast connections - whether you download 10 or 30 files at a time. Downloading several Web sites at a time is always better, because you balance load on several servers.
> > Best regards,
> Oleg Chernavin
> MP Staff
Thank you very much for your answers.
Now I will focus only on point 1). These are the symptoms I have:
- Yes, I have a Parsing (1049) message on the bottom left status bar.
- The download seems to be finishing, but never finishes: it has 30 threads,
27 with "Download Complete", 2 with "Connection Timed out", and 1 with "File
is probably incomplete, retrying".
- The "Queue" folder is empty.
- Seems that there is no internet activity, but the hard disk is working a lot.
- If I try to suspend the project, the status bar says "Suspending", and stays there.
- The site is ASP (www.lanacion.com.ar), and I`m downloading it with a level=3 deep.
Reards,
JJ
> 1. Perhaps Offline Explorer processes downloaded files - please see in the Status Bar if there is a Parsing (XX) message.
> > 2. Update Project checks all files whether they have changed on the server since the last download. Download missing files will not check all downloaded files - it will simply detect all links that were not yet downloaded and gets them.
> > 3. Many servers have dynamic pages nowdays and they report them as just newly updated. You can easily recognize them - .asp, .aspx, .php, etc. This is a problem, but such servers may even not supply the file size.
> > 4. From my experience, it doesn`t matter much on fast connections - whether you download 10 or 30 files at a time. Downloading several Web sites at a time is always better, because you balance load on several servers.
> > Best regards,
> Oleg Chernavin
> MP Staff
This means that Offline Explorer still has 1049 files to process (extract links and change them for offline browsing). Perhaps this process is slow because there are some directories overpopulated (have too many files). Windows filesystems slows down significantly in this case. To overcome it, please allow Directory Overload protection in the Options dialog - File Locations section. It is better to erase such overloaded directories after that and redownload files.
Oleg.
Oleg.
Hello.
I have tryed again with the Directory Overload protection (with previous erase of all)
Same result: the download process stops without finishing.
I have a "Parsing (925)" indication on the bottom left.
There is no internet nor hard disk activity.
I have 29 threads with "Download Complete", and one with "File is probably incomplete, retrying".
I hope you can have a hint on this...
Thank you very much,
JJ
I have tryed again with the Directory Overload protection (with previous erase of all)
Same result: the download process stops without finishing.
I have a "Parsing (925)" indication on the bottom left.
There is no internet nor hard disk activity.
I have 29 threads with "Download Complete", and one with "File is probably incomplete, retrying".
I hope you can have a hint on this...
Thank you very much,
JJ
What version of Offline Explorer do you have now?
Oleg.
Oleg.
It is Offline Explorer Enterprise 4.0.2272
> What version of Offline Explorer do you have now?
> > Oleg.
> What version of Offline Explorer do you have now?
> > Oleg.
Can I ask you to try the latest version of oe.exe file?
http://www.metaproducts.com/download/betas/oep2282.zip
Please replace the old oe.exe file with the new one from the ZIP archive. Then let me know if the new version hangs on parsing or not.
Oleg.
http://www.metaproducts.com/download/betas/oep2282.zip
Please replace the old oe.exe file with the new one from the ZIP archive. Then let me know if the new version hangs on parsing or not.
Oleg.
Hello again. Sorry, but the problem is still there. I have tryed oep2282 and oep2283.
Download stops and then all the sequence of downloads is stopped.
Regards, JJ
> Can I ask you to try the latest version of oe.exe file?
> > http://www.metaproducts.com/download/betas/oep2282.zip
> > Please replace the old oe.exe file with the new one from the ZIP archive. Then let me know if the new version hangs on parsing or not.
> > Oleg.
Download stops and then all the sequence of downloads is stopped.
Regards, JJ
> Can I ask you to try the latest version of oe.exe file?
> > http://www.metaproducts.com/download/betas/oep2282.zip
> > Please replace the old oe.exe file with the new one from the ZIP archive. Then let me know if the new version hangs on parsing or not.
> > Oleg.
Can you please turn on logging (Ctrl+W) and let me know which file starts to be parsed and copy/paste its URL.
Thank you!
Oleg.
Thank you!
Oleg.
Hello. I have turned on logging, and started the download of the site with the problem (the site that does not finish). The first message logged is:
QUEUE - 27/12/2005 02:34:20 p.m. - Analizando (0) http://www.lanacion.com.ar/
Is this what you need ?
Also, for your information, I have copied and pasted the project:
[Object]
OEVersion=Pro 4.0.0.2283
Type=0
IID=7012
Caption=La Nacion (nivel 3)
URL=http://www.lanacion.com.ar/
Lev=3
Weekday=257
LimTSize=10000
LimNumber=5000
LimTime=100
FMGroup=2
CheckSize=True
FTText.Exts=htmlhtmaspaspxjspstmstmlidcshtmlhtxtxttextxspxmlrxmlcfmwmlphpphp3
FTImages.Exts=gifjpgjpegtiftiffxbmfifbmppngipxjp2j2cj2kwbmplwf
FTVideo.Exts=mpgavianimpegmovfliflcvivrmramrvasfasxwmvm1vm2vvob
FTAudio.Exts=wavriffmp3midmp2m3uravocwmaape
FTArchive.Exts=ziparcgzzarjlhalayleirarcabtarpakacejarpdf
FTUDef.Exts=jscssssivbsdtdxslswfclass
FTText.B=ooxooo
FTImages.B=ooxoxo
FTVideo.B=xoxooo
FTAudio.B=xoxooo
FTArchive.B=xoxooo
FTUDef.B=ooxooo
FTOther.B=xoxooo
FTSizes=0,0,20,0,0,0,0,0,0,0,0,0,0,0,1,1,3,3,0,1,0
RProt=127
LastStart=110:182:7:110:51:231:226:64:
LastEnd=175:155:240:110:51:231:226:64:
S304=137
SPar=1
SLast=304
LFiles=137
Stopped=True
Copies=10000
CopiesDate=YYYYMMDDHHNN
ImgDim=0,0,0,0
PrevURL=http://www.lanacion.com.ar/
SkipURLs=https://www.lanacion.com.ar
ConvertRSS=True
Thank you.
> Can you please turn on logging (Ctrl+W) and let me know which file starts to be parsed and copy/paste its URL.
> > Thank you!
> > Oleg.
QUEUE - 27/12/2005 02:34:20 p.m. - Analizando (0) http://www.lanacion.com.ar/
Is this what you need ?
Also, for your information, I have copied and pasted the project:
[Object]
OEVersion=Pro 4.0.0.2283
Type=0
IID=7012
Caption=La Nacion (nivel 3)
URL=http://www.lanacion.com.ar/
Lev=3
Weekday=257
LimTSize=10000
LimNumber=5000
LimTime=100
FMGroup=2
CheckSize=True
FTText.Exts=htmlhtmaspaspxjspstmstmlidcshtmlhtxtxttextxspxmlrxmlcfmwmlphpphp3
FTImages.Exts=gifjpgjpegtiftiffxbmfifbmppngipxjp2j2cj2kwbmplwf
FTVideo.Exts=mpgavianimpegmovfliflcvivrmramrvasfasxwmvm1vm2vvob
FTAudio.Exts=wavriffmp3midmp2m3uravocwmaape
FTArchive.Exts=ziparcgzzarjlhalayleirarcabtarpakacejarpdf
FTUDef.Exts=jscssssivbsdtdxslswfclass
FTText.B=ooxooo
FTImages.B=ooxoxo
FTVideo.B=xoxooo
FTAudio.B=xoxooo
FTArchive.B=xoxooo
FTUDef.B=ooxooo
FTOther.B=xoxooo
FTSizes=0,0,20,0,0,0,0,0,0,0,0,0,0,0,1,1,3,3,0,1,0
RProt=127
LastStart=110:182:7:110:51:231:226:64:
LastEnd=175:155:240:110:51:231:226:64:
S304=137
SPar=1
SLast=304
LFiles=137
Stopped=True
Copies=10000
CopiesDate=YYYYMMDDHHNN
ImgDim=0,0,0,0
PrevURL=http://www.lanacion.com.ar/
SkipURLs=https://www.lanacion.com.ar
ConvertRSS=True
Thank you.
> Can you please turn on logging (Ctrl+W) and let me know which file starts to be parsed and copy/paste its URL.
> > Thank you!
> > Oleg.
Well, I need last lines like that. If the parsing hangs, the Log should show the last file that started parsing, but didn`t end.
Oleg.
Oleg.
Hello. Tyed again. The download finished with "Download Complete" in all the threads. However the project is still reported as "Loading", and interrupts the download sequence.
The last logged line was:
HTTP4 - 29/12/2005 02:02:48 p.m. - 100% of 35579 bytes of http://www.lanacion.com.ar/Archivo/nota.asp?nota_id=764698.
I have also copied the state of all the 20 finished threads. Click here to see the image:
http://www.lablar.com/Captura.jpg
Regards.
The last logged line was:
HTTP4 - 29/12/2005 02:02:48 p.m. - 100% of 35579 bytes of http://www.lanacion.com.ar/Archivo/nota.asp?nota_id=764698.
I have also copied the state of all the 20 finished threads. Click here to see the image:
http://www.lablar.com/Captura.jpg
Regards.
Can you please try the following updated oe.exe file:
http://www.metaproducts.com/download/betas/oep2294.zip
Replace the older oe.exe file with the new one. Please let me know if the bug reproduces or not.
Thank you!
Oleg.
http://www.metaproducts.com/download/betas/oep2294.zip
Replace the older oe.exe file with the new one. Please let me know if the bug reproduces or not.
Thank you!
Oleg.
Downloaded and installed oep2924, but ... same result: site downloaded, but does not finish the project.
Here you have some behaviour that perhaps may help:
After finishing the (bad) website, I pressed STOP to force it to finish the project. On the report, it now shows "Stopped". In this state, if I start any other download (sites that have not showed the problem), it does not download! It shows "Download complete", but didn`t download anything.
The only way to start a new download is to close OE and start again, and now the download goes normal.
It looks like a problem on the internal state machine ...??
Regards,
JJ
> Can you please try the following updated oe.exe file:
> > http://www.metaproducts.com/download/betas/oep2294.zip
> > Replace the older oe.exe file with the new one. Please let me know if the bug reproduces or not.
> > Thank you!
> > Oleg.
Here you have some behaviour that perhaps may help:
After finishing the (bad) website, I pressed STOP to force it to finish the project. On the report, it now shows "Stopped". In this state, if I start any other download (sites that have not showed the problem), it does not download! It shows "Download complete", but didn`t download anything.
The only way to start a new download is to close OE and start again, and now the download goes normal.
It looks like a problem on the internal state machine ...??
Regards,
JJ
> Can you please try the following updated oe.exe file:
> > http://www.metaproducts.com/download/betas/oep2294.zip
> > Replace the older oe.exe file with the new one. Please let me know if the bug reproduces or not.
> > Thank you!
> > Oleg.
Can you please find the last QUEUE lines in the log and send me few of them?
Oleg.
Oleg.
Hello Oleg, I`m here again.
The QUEUE tab finishes EMPTY after download.
Also, the download progress finishes with a Download Complete.
Everything seems to be OK, BUT...
- The STOP button continues active, meaning that the software thinks there is a download in course.
- If there was a SEQUENCE, it is stopped.
- If you mannualy start a new download for a different site, it does not work.
There is a problem here. I think you may easily duplicate it by downloading
www.lanacion.com.ar with a Level Limit of 3.
Thank you,
JJ
> Can you please find the last QUEUE lines in the log and send me few of them?
> > Oleg.
The QUEUE tab finishes EMPTY after download.
Also, the download progress finishes with a Download Complete.
Everything seems to be OK, BUT...
- The STOP button continues active, meaning that the software thinks there is a download in course.
- If there was a SEQUENCE, it is stopped.
- If you mannualy start a new download for a different site, it does not work.
There is a problem here. I think you may easily duplicate it by downloading
www.lanacion.com.ar with a Level Limit of 3.
Thank you,
JJ
> Can you please find the last QUEUE lines in the log and send me few of them?
> > Oleg.
I mean that I need the last lines of the log with the QUEUE messages.
Oleg.
Oleg.
Sorry but I don`t understand what you want.
I have the full log file of the site with the problem; I can email it to you (tell me your email address)
Thanks.
> I mean that I need the last lines of the log with the QUEUE messages.
> > Oleg.
I have the full log file of the site with the problem; I can email it to you (tell me your email address)
Thanks.
> I mean that I need the last lines of the log with the QUEUE messages.
> > Oleg.
Please zip the log and send it to support@metaproducts.com. Thank you!
Oleg.
Oleg.
Did you receive the LOG file ?
JJ
> Please zip the log and send it to support@metaproducts.com. Thank you!
> > Oleg.
JJ
> Please zip the log and send it to support@metaproducts.com. Thank you!
> > Oleg.
Yes and I will work on it soon. Thank you!
Oleg.
Oleg.
Today I sent you a new e-mail with more on this issue.
JJ
> Yes and I will work on it soon. Thank you!
> > Oleg.
JJ
> Yes and I will work on it soon. Thank you!
> > Oleg.
Did you send it directly to my E-mail address?
Oleg.
Oleg.
Same problem here...
Program show parsing 678 files on status bar....... and stays like this indefinitely...
When i stop the project and choose download missing files........it says "parsing (678)" and doesn''t change status.
The icon of the project is green (downloading).
Do not overload directories activated....
Program show parsing 678 files on status bar....... and stays like this indefinitely...
When i stop the project and choose download missing files........it says "parsing (678)" and doesn''t change status.
The icon of the project is green (downloading).
Do not overload directories activated....
What version do you use? And please let me know other details - the site you are downloading, number of files downloaded by the time it stalls, number of files in the Queue, downloaded files size.
Oleg.
Oleg.
> What version do you use? And please let me know other details - the site you are downloading, number of files downloaded by the time it stalls, number of files in the Queue, downloaded files size.
>
> Oleg.
Ok....
The version is 4.9.2694 SR2
The site is: http://www.forumnow.com.br/vip/foruns.asp?forum=94106 (IE cookies activated, IgnoreLogoutLinks activated...)
Actually, as it''s a huge forum site, i stopped downloading it sometimes. I think i''ve lost the map and choose CTRL + F5 to rebuild it.
Now it always downloads a small number of files, the queue stays with 678 files and nothing happens. The icon is green (downloading). The project has already downloaded aproximately 100.000 files, but i know it hasn''t downloaded everything in the site, as i want (the 678 files in queue confirms it).
>
> Oleg.
Ok....
The version is 4.9.2694 SR2
The site is: http://www.forumnow.com.br/vip/foruns.asp?forum=94106 (IE cookies activated, IgnoreLogoutLinks activated...)
Actually, as it''s a huge forum site, i stopped downloading it sometimes. I think i''ve lost the map and choose CTRL + F5 to rebuild it.
Now it always downloads a small number of files, the queue stays with 678 files and nothing happens. The icon is green (downloading). The project has already downloaded aproximately 100.000 files, but i know it hasn''t downloaded everything in the site, as i want (the 678 files in queue confirms it).
P.S.:
Sorry....
I gave wrong information.
The correct information is:
The version is 4.9.2694 SR2
The site is: http://www.forumnow.com.br/vip/foruns.asp?forum=94106 (IE cookies activated, IgnoreLogoutLinks activated...)
Actually, as it''''s a huge forum site, i stopped downloading it sometimes. I think i''''ve lost the map and choose CTRL + F5 to rebuild it.
Now it says in says in status bar: DOWNLOADED=0 ----- PARSING (678) ----- QUEUE=0 SIZE=0. BUT NOTHING HAPPENS. The icon is green (downloading). The project has already downloaded aproximately 100.000 files, but i know it hasn''''t downloaded everything in the site, as i want.
> > What version do you use? And please let me know other details - the site you are downloading, number of files downloaded by the time it stalls, number of files in the Queue, downloaded files size.
> >
> > Oleg.
>
> Ok....
>
> The version is 4.9.2694 SR2
>
> The site is: http://www.forumnow.com.br/vip/foruns.asp?forum=94106 (IE cookies activated, IgnoreLogoutLinks activated...)
>
> Actually, as it''''s a huge forum site, i stopped downloading it sometimes. I think i''''ve lost the map and choose CTRL + F5 to rebuild it.
>
> Now it always downloads a small number of files, the queue stays with 678 files and nothing happens. The icon is green (downloading). The project has already downloaded aproximately 100.000 files, but i know it hasn''''t downloaded everything in the site, as i want (the 678 files in queue confirms it).
>
>
Sorry....
I gave wrong information.
The correct information is:
The version is 4.9.2694 SR2
The site is: http://www.forumnow.com.br/vip/foruns.asp?forum=94106 (IE cookies activated, IgnoreLogoutLinks activated...)
Actually, as it''''s a huge forum site, i stopped downloading it sometimes. I think i''''ve lost the map and choose CTRL + F5 to rebuild it.
Now it says in says in status bar: DOWNLOADED=0 ----- PARSING (678) ----- QUEUE=0 SIZE=0. BUT NOTHING HAPPENS. The icon is green (downloading). The project has already downloaded aproximately 100.000 files, but i know it hasn''''t downloaded everything in the site, as i want.
> > What version do you use? And please let me know other details - the site you are downloading, number of files downloaded by the time it stalls, number of files in the Queue, downloaded files size.
> >
> > Oleg.
>
> Ok....
>
> The version is 4.9.2694 SR2
>
> The site is: http://www.forumnow.com.br/vip/foruns.asp?forum=94106 (IE cookies activated, IgnoreLogoutLinks activated...)
>
> Actually, as it''''s a huge forum site, i stopped downloading it sometimes. I think i''''ve lost the map and choose CTRL + F5 to rebuild it.
>
> Now it always downloads a small number of files, the queue stays with 678 files and nothing happens. The icon is green (downloading). The project has already downloaded aproximately 100.000 files, but i know it hasn''''t downloaded everything in the site, as i want (the 678 files in queue confirms it).
>
>
You will have to wait. Parsing means getting links from the downloaded page. It has to get through all pages on your hard disk to get the links to download further.
Oleg.
Oleg.