Setting Cookies

Author Message
Dave 11/29/2005 10:52 am
I am in the process of trying to inventory a website. I have been working with OE to try and download the website for analysis, however I have run into a problem. I have found that the website does not download the website completely, and that there are some links that I can browse normally, that are not downloaded. I am convinced that this is the result of a problem with OE being able to properly use the cookies.

The website uses HTML Forms authentication, and after analyzing the website traffic, I find the following information, after you authenticate you get redirected to a SSL encrypted website called:

GET /mydoc.jpg HTTP/1.1
Accept: */*
Referer: http://www.mywebsite.com/
Accept-Language: en-us
Accept-Encoding: gzip, deflate
If-Modified-Since: Wed, 06 Feb 2002 19:20:04 GMT
If-None-Match: "50a8c4843afc11:aa7"
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; InfoPath.1)
Host: www.mywebsite.com
Connection: Keep-Alive
Cookie: SITESERVER=ID=d96af3fa733ef3081b9ed4304b399718; ASPSESSIONIDASDCQTBS=OOFGPNPDBGMPBMJICIGLNFPB

I have tried specifying in the project the following information:

https://secure.mywebsite.com/
Cookie=SITESERVER=ID=d96af3fa733ef3081b9ed4304b399718; ASPSESSIONIDASDCQTBS=OOFGPNPDBGMPBMJICIGLNFPB

However even after trying this approach I am unable to download the other documents in the website that I know are missing. Is the above information formed correctly for the Cookie? The site does redirect to another website protected with SSL, and I believe that the cookie is used for both websites.

Any ideas as how I can inventory this html forms protected website?

Thanks,
Dave
Oleg Chernavin 11/29/2005 11:02 am
It is easy to download this type of site.
You need to browse to the logon page of the site using the internal Browser of Offline Explorer Pro .

If you need to download the site immediately and only once, you can proceed with the logon and begin downloading the desired pages using Offline Explorer Pro . The program will use the session cookies of the logged on site from the internal browser.

You can also record the logon form contents in a Project, so that Offline Explorer Pro will know how to log itself on whenever you wish to download the site. This is useful when you want to schedule the site download or perform it later, or if you want to update the downloaded site in the future.

Once you have entered your username and password on the logon page in the internal browser, press and hold the Alt + Ctrl keys on your keyboard, click the Logon (or Submit) button in the Web form and release the keyboard buttons. You should get a new Project that contains the Web form information recorded in the URL field.

Adjust the Project settings as you wish (set the Level and other parameters) and click the OK button to save the Project. You may begin downloading at any time.

Note: The form recording method is supported only in the Pro and Enterprise editions of Offline Explorer.

Best regards,
Oleg Chernavin
MP Staff
Dave 11/29/2005 11:22 am
> It is easy to download this type of site.
> You need to browse to the logon page of the site using the internal Browser of Offline Explorer Pro .
>
> If you need to download the site immediately and only once, you can proceed with the logon and begin downloading the desired pages using Offline Explorer Pro . The program will use the session cookies of the logged on site from the internal browser.
>
> You can also record the logon form contents in a Project, so that Offline Explorer Pro will know how to log itself on whenever you wish to download the site. This is useful when you want to schedule the site download or perform it later, or if you want to update the downloaded site in the future.
>
> Once you have entered your username and password on the logon page in the internal browser, press and hold the Alt + Ctrl keys on your keyboard, click the Logon (or Submit) button in the Web form and release the keyboard buttons. You should get a new Project that contains the Web form information recorded in the URL field.
>
> Adjust the Project settings as you wish (set the Level and other parameters) and click the OK button to save the Project. You may begin downloading at any time.
>
> Note: The form recording method is supported only in the Pro and Enterprise editions of Offline Explorer.
>
> Best regards,
> Oleg Chernavin
> MP Staff

Thanks Oleg for the quick response!

I did find the forum post regarding holding Alt + Ctrl when clicking on the login page, however I am not presented with a new project box that contains the web form information. I think the problem may be the result of an additional Window being launched after authenticating to the website. After you click on the Login button, another Internet Explorer window opens outside of OE, that contains the authenticated web.

Any thoughts on how this may affect the process of creating the Project, or maybe there is something that I`m doing wrong.

Thanks,
Dave
Oleg Chernavin 11/29/2005 11:38 am
Yes, popup window is a problem here. To overcome this, please right-click the Logon page (with the form to enter your username/password), select Properties and copy the page address. Then paste it to the Internal browser of Offline Explorer and browse to it. Now you will be able to use Alt+Ctrl.

Oleg.