HTTPS download of a protected subtree

Author Message
Lufe 10/27/2003 08:39 am
Hello, I am in charge of testing this product to see if it can be used in our company, so far I can say you guys have made a wonderful job ;)

One of the main things we need, though, is to download a site from our providers where the protocol is https. We need to enter a username and password to see the contents of the site.

The site is https://www.sec.sisclear.com/bpspec/default.htm

So, if I enter this url in the project properties, all pages which are not secure are downloaded. The ones which need an authorization are not, and a message appears (see below):

--------------------------------------------------------------------------
"Document not found
This page is not accessible offline. Possible reasons: either it was an invalid link on the server or Project settings do not allow the page to be downloaded. In some cases, increasing Project Level setting should help.


Click here to go online:
https://www.sec.sisclear.com/bpspec/vol4/updates/next/updates.htm

Download the missing link now and add it to the selected Project."
--------------------------------------------------------------------------

I`ve tried to put my username and password on the "Project Properties">Passwords, and all other options, and it doesn`t work.
If I try to manually "Download the missing link now and add it to the selected Project", it works, after I give the username and password.

The problem is that there are hundreds of other links, and it`s not possible to go into each one of them and repeat this process. After I gave the
username and password for the first one, I tried again to start downloading the whole project, but no new pages come.

If you can please e-mail me, I will give you the username and password so you could see what the problem is. The goal is to download the whole
subtree, starting at the url listed above, and "force" the links which are not shown to be downloaded, after the password is entered. I noticed
that if I enter the username/password in one link, the other links don`t ask for it anymore (exactly as happens with a browser).

What is keeping the Offline Explorer from downloading the whole thing?

Thank a lot for the help, I hope this is solved so we can buy the product :)
Oleg Chernavin 10/27/2003 10:11 am
Can you try the following - browse to the site in the Internal browser of Offline Explorer Pro, find a link that requires authorization and log on there. Then keep the site logged on, start downloading it with Offline Explorer Pro. It will use cookies of the logged session. This helps on most of protected sites.

Please only make sure that Offline Explorer doesn`t follow logout links (if they are on the site).

Best regards,
Oleg Chernavin
MetaProducts corp.
Lufe 10/27/2003 11:38 am
> Can you try the following - browse to the site in the Internal browser of Offline Explorer Pro, find a link that requires authorization and log on there. Then keep the site logged on, start downloading it with Offline Explorer Pro. It will use cookies of the logged session. This helps on most of protected sites.
>
> Please only make sure that Offline Explorer doesn`t follow logout links (if they are on the site).
>
> Best regards,
> Oleg Chernavin
> MetaProducts corp.

I have tried that. I think that somehow it bumps in a place where the cookie is invalidated, or something. I`ve tried to add the url where it asks for the password to the url filter, so that it doesn`t go there again, but it doesn`t work.

I have sent you an e-mail with the username and pass, so if you could test I`d be most grateful.
Oleg Chernavin 10/28/2003 07:08 am
Yes, I got that E-mail. I will work on it.

Oleg.