Hello,
I'm trying to download a phpBB-powered forum, but after a few downloaded pages, the site logs me off and all the following downloaded pages are just login forms.
What I do:
- I go to the website in internal browser and log in, then start the download.
What happens:
- OE succesfully downloads several pages, but then the forum logs the user off and the following pages are just login forms.
What I've tried:
- (Un)checking "Use MS Internet Explorer cookies".
- (Un)checking "Use alternative connection method".
- Downloading really slow (1 connection, 30 seconds delay).
- Downloading really fast (100 connections, no delay).
- Setting alternate "Download method" ("Open every page in the Internal Browser and save").
- Change identity to various browsers.
The most successul combination that I've tried was:
- Download method: Open every page in the Internal Browser and save
- Use alternative connection method: unchecked
- Use MS Internet Explorer cookies: checked
This combination managed to download 303 pages before I was logged out, while all the others managed to download no more than 10.
My humble guess is that it somehow related to session ID checking. I can send you the website and login info if you are interested in this problem.
Thank you.
I'm trying to download a phpBB-powered forum, but after a few downloaded pages, the site logs me off and all the following downloaded pages are just login forms.
What I do:
- I go to the website in internal browser and log in, then start the download.
What happens:
- OE succesfully downloads several pages, but then the forum logs the user off and the following pages are just login forms.
What I've tried:
- (Un)checking "Use MS Internet Explorer cookies".
- (Un)checking "Use alternative connection method".
- Downloading really slow (1 connection, 30 seconds delay).
- Downloading really fast (100 connections, no delay).
- Setting alternate "Download method" ("Open every page in the Internal Browser and save").
- Change identity to various browsers.
The most successul combination that I've tried was:
- Download method: Open every page in the Internal Browser and save
- Use alternative connection method: unchecked
- Use MS Internet Explorer cookies: checked
This combination managed to download 303 pages before I was logged out, while all the others managed to download no more than 10.
My humble guess is that it somehow related to session ID checking. I can send you the website and login info if you are interested in this problem.
Thank you.
Some more details I forgot:
- "Ignore logout, delete links and captcha images" box is checked
- I'm downloading a list of specific pages with "Level limit" set to 0 (like [...]/viewforum.php?f=1&start_letter=&start={:0..6560|80}), so there isn't any crawling
- "Ignore logout, delete links and captcha images" box is checked
- I'm downloading a list of specific pages with "Level limit" set to 0 (like [...]/viewforum.php?f=1&start_letter=&start={:0..6560|80}), so there isn't any crawling
Stefan,
Yes, please send me the details to support@metaproducts.com. I will try to see if there is a way to download the forum.
You may also try to change the macro to download less pages with the "Browser" method. For example, load 200 pages. Then logout/logon manually, change macro to next 200 pages and load them. And so on.
It might be faster than waiting while I am looking for a solution.
Best regards,
Oleg Chernavin
MP Staff
Yes, please send me the details to support@metaproducts.com. I will try to see if there is a way to download the forum.
You may also try to change the macro to download less pages with the "Browser" method. For example, load 200 pages. Then logout/logon manually, change macro to next 200 pages and load them. And so on.
It might be faster than waiting while I am looking for a solution.
Best regards,
Oleg Chernavin
MP Staff
I think, I found the way to overcome the logout for this forum.
Download and install TOR Browser:
https://www.torproject.org/projects/torbrowser.html.en
Start it, then run Offline Explorer Pro, open its Options dialog - Proxy Servers section and add TOR proxy. Click OK button.
Now logon in the Internal browser to the site online. In the Internet tab of Offline Explorer set 1 connection and Delay between downloads to 2-10 for a random delay.
Then start the Project download. It downloaded all pages up to 6560 with no single logout with these settings.
Oleg.
Download and install TOR Browser:
https://www.torproject.org/projects/torbrowser.html.en
Start it, then run Offline Explorer Pro, open its Options dialog - Proxy Servers section and add TOR proxy. Click OK button.
Now logon in the Internal browser to the site online. In the Internet tab of Offline Explorer set 1 connection and Delay between downloads to 2-10 for a random delay.
Then start the Project download. It downloaded all pages up to 6560 with no single logout with these settings.
Oleg.