|mulig||11/20/2006 04:03 am|
i need your help to handle orkut.com
i have an account, and i know OE can simulate clicking on links on the pages it crawls. My goal is to search for friends in a cetrtain country, and from the results page, visit all pages and clink on add friend button automatically.
it is also nice to be able to crawl the images from these pages.
OE keeps looping in a login process. Please help
|Oleg Chernavin||11/20/2006 07:01 am|
|There are two kinds of password-protected Web sites. One type asks for a username and password in a standard Windows-type dialog (BASIC and NTLM authentication,) while the other type requires you to logon directly on a Web page.
1. Web sites that require BASIC and NTLM-authentication.
To download this type of site, either specify the username and password in the Project Properties dialog | Advanced | Password section, or type them directly in the URL, such as:
Notice that a colon separates the password from the username and the @ symbol separates them from the server name.
Some sites use NTLM authentication, which looks like the above, but with a third box labeled "DOMAIN" along with the username and password. Enter the domain name along with the username in the same field in the Project Properties dialog | Passwords this way:
The backslash symbol separates the domain name from the username. When you are done with the Project Properties changes, click the OK button to save them.
Note: NTLM authentication is supported only in the Pro and Enterprise editions of Offline Explorer.
2. Authentication in a Web form.
It is easy to download this type of site.
You need to browse to the logon page of the site using the internal Browser of Offline Explorer Pro .
If you need to download the site immediately and only once, you can proceed with the logon and begin downloading the desired pages using Offline Explorer Pro . The program will use the session cookies of the logged on site from the internal browser.
You can also record the logon form contents in a Project, so that Offline Explorer Pro will know how to log itself on whenever you wish to download the site. This is useful when you want to schedule the site download or perform it later, or if you want to update the downloaded site in the future.
Once you have entered your username and password on the logon page in the internal browser, press and hold the Alt + Ctrl keys on your keyboard, click the Logon (or Submit) button in the Web form and release the keyboard buttons. You should get a new Project that contains the Web form information recorded in the URL field.
Adjust the Project settings as you wish (set the Level and other parameters) and click the OK button to save the Project. You may begin downloading at any time.
Note: The form recording method is supported only in the Pro and Enterprise editions of Offline Explorer.
The same way with Ctrl+Alt could be used on the site search page to load the search results.