Configuring Offline Explorer Pro for Form based authentication sites

Author Message
Imagespy 11/15/2004 07:41 pm
Dear Admin,
I am trying to download pictures from a form based login/password authentication site. This site is huge and all the pictures have got thumbnails and .html links. I have been successful in login into this site and browse it online and offline however I need to automate the process of downloading images from this protected site. I understand that it is quite simple if I would like to download all the pictures however that is not the aim. The following flow-chart would hopefully clarify what I am trying to do.

Steps:
1. Browse the website with IE
2. Choose the hyperlinks of thumbnails of different sets of pictures and paste in Notepad
3. Paste all these hyperlinks in Offline Explorer Pro
4. Find a way to automate entering the login and password of these links (form based)
5. Download all hyperlinks 1 level below of these pasted hyperlinks (hopefully these links would contain the hyperlinks of actual images). At this stage I am only interested downloading just the hyperlinks not the actual images.
e.g:
http://www.anysite.com/members/fullpic/sunrise/page1.html
http://www.anysite.com/members/fullpic/sunrise/page2.html
http://www.anysite.com/members/fullpic/sunrise/page3.html
http://www.anysite.com/members/fullpic/sunrise/image01.jpg
http://www.anysite.com/members/fullpic/sunrise/image02.jpg
http://www.anysite.com/members/fullpic/sunrise/image03.jpg
http://www.anysite.com/members/fullpic/sunrise/image04.jpg
http://www.anysite.com/members/fullpic/sunrise/image05.jpg
http://www.anysite.com/members/fullpic/sunrise/image06.jpg
http://www.anysite.com/members/fullpic/sunrise/image07.jpg
http://www.anysite.com/members/fullpic/sunrise/image08.jpg
....................
http://www.anysite.com/members/fullpic/sunrise/image91.jpg
http://www.anysite.com/members/fullpic/sunrise/image92.jpg
http://www.anysite.com/members/fullpic/sunrise/image93.jpg

http://www.anysite.com/members/thumbs/sunrise/image01.jpg
http://www.anysite.com/members/thumbs/sunrise/image02.jpg
http://www.anysite.com/members/thumbs/sunrise/image03.jpg
http://www.anysite.com/members/thumbs/sunrise/image04.jpg
http://www.anysite.com/members/thumbs/sunrise/image05.jpg
http://www.anysite.com/members/thumbs/sunrise/image06.jpg
http://www.anysite.com/members/thumbs/sunrise/image07.jpg
http://www.anysite.com/members/thumbs/sunrise/image08.jpg
..............................
http://www.anysite.com/members/thumbs/sunrise/image91.jpg
http://www.anysite.com/members/thumbs/sunrise/image92.jpg
http://www.anysite.com/members/thumbs/sunrise/image93.jpg

6. Copy all these newly downloaded links in an excel spreadsheet to sort
7. Find all links which contain the hyperlinks for the actual images
e.g:
http://www.anysite.com/members/fullpic/sunrise/image01.jpg
http://www.anysite.com/members/fullpic/sunrise/image02.jpg
http://www.anysite.com/members/fullpic/sunrise/image03.jpg
http://www.anysite.com/members/fullpic/sunrise/image04.jpg
http://www.anysite.com/members/fullpic/sunrise/image05.jpg
http://www.anysite.com/members/fullpic/sunrise/image06.jpg
http://www.anysite.com/members/fullpic/sunrise/image07.jpg
http://www.anysite.com/members/fullpic/sunrise/image08.jpg
........................
http://www.anysite.com/members/fullpic/sunrise/image91.jpg
http://www.anysite.com/members/fullpic/sunrise/image92.jpg
http://www.anysite.com/members/fullpic/sunrise/image93.jpg
8. Choose every 3rd hyperlink of image (only want to download one-third of actual number of images)
e.g.:
http://www.anysite.com/members/fullpic/sunrise/image01.jpg
http://www.anysite.com/members/fullpic/sunrise/image04.jpg
http://www.anysite.com/members/fullpic/sunrise/image07.jpg
9. Overwrite the old project with these new hyperlinks (which are the hyperlinks for actual images) and start downloading these images.

I am seeking help in regards to step 3, 4, 5, 6 & 9.

Can this be done with Offline Explorer Pro?

You assistance in this regard is highly apprecia
Oleg Chernavin 11/17/2004 01:41 am
If the image links have this kind of simple numbering, you could use URL Macros to download them all using a single line in the Project`s URLs field:

http://www.anysite.com/members/fullpic/sunrise/image{:01..99}.jpg

Best regards,
Oleg Chernavin
MP Staff
Imagespy 11/17/2004 02:49 pm
Many thanks Oleg. However is it posible to just download the hyperlinks in any particular page rather than actual imgaes so that I can analyze all the hyperlinks.

Kind regards

Oleg Chernavin 11/18/2004 06:25 am
Yes, this is possible, but I cannot give you a particular advice without looking at the actual site. Can you give me the URL? If it is not public, you can contact me using support@metaproducts.com .

Oleg.