Downloading Forums

CJ
01/29/2007 07:22 pm
I've been trying to figure out how to get OE to download the forums on SOE's website, before they delete them next week.

The situation.

The portion of their forums i want to down load is:

http://soe.lithium.com/board?board.id=game_guides

which is page 1. but page 1 also has this url

http://soe.lithium.com/swg/board?board.id=game_guides&page=1

so... I mapped the site and got all the pages from page1 to page1883.

Problem is, when i run it, it skips entirely over long series of pages and then picks up somewhere else.

Each forum has a list of threads, and replies to those threads.

I just want to download all the pages and the top 2 levels of each thread.

But I'm also downloading the print preview stuff, notifiy moderator, and user/profile id stuff.

----------------------------------------------------------------------------------------------------------

In the address list for the URLS, my URLS look like this....

http://soe.lithium.com/swg/board?board.id=game_guides&page=1
http://soe.lithium.com/swg/board?board.id=game_guides&page=2
http://soe.lithium.com/swg/board?board.id=game_guides&page=3
http://soe.lithium.com/swg/board?board.id=game_guides&page=5
http://soe.lithium.com/swg/board?board.id=game_guides&page=5
.
.
.

Should they have ending puntuation??

But this program doesn't seem to hit each individual page, it seems to bounce all over the place.

I'd be happy with this psuedo download:

Game_Guides:
page1: Each Thread and the first page in that thread
page2: each thread and the first page in that thread
.
.
.
page1883: each thread and the first page in that thread.

--------------------------------------------------------------------------------------------------------------------
This is the url's to the first page so you can get an idea. (Tree Structure)

http://soe.lithium.com/swg/board?board.id=game_guides&page=1
|____http://soe.lithium.com/swg/board/message?board.id=game_guides&message.id=297713
(First Thread of Page 1)
|___http://soe.lithium.com/swg/board/message? board.id=game_guides&message.id=297713&view=by_date_ascending&page=2
(Second Page of 1st Thread)

|____http://soe.lithium.com/swg/board/message?board.id=game_guides&message.id=312289
(second thread of Page 1)
|____ No Second Page
Oleg Chernavin
01/30/2007 05:37 am
As I understand, you want to load in depth first. To do this, please add the following line to the Project's URLs field:

Additional=DepthFirst

Once the top page is loaded, Offline Explorer will load its topics and only then it will continue with the next top-level page.

Best regards,
Oleg Chernavin
MP Staff
CJ
01/30/2007 10:47 am
As my URL List goes....

does: Additional=DepthFirst

appear once, or does it follow each susequent URL in my list.?

What do I specify the limit to? Just uncheck the box?



> As I understand, you want to load in depth first. To do this, please add the following line to the Project's URLs field:
>
> Additional=DepthFirst
>
> Once the top page is loaded, Offline Explorer will load its topics and only then it will continue with the next top-level page.
>
> Best regards,
> Oleg Chernavin
> MP Staff
Oleg Chernavin
01/30/2007 10:54 am
Just once in the URLs field. Uncheck Limit to make it unlimited. Check it and set a limited value otherwise.

Oleg.
CJ
01/30/2007 11:06 am
BTW URLS and Commands in the Addresses(URLs) Field, is there some type of punctuation that terminates each line?

> Just once in the URLs field. Uncheck Limit to make it unlimited. Check it and set a limited value otherwise.
>
> Oleg.
Oleg Chernavin
01/30/2007 11:15 am
Each command or URL should be on a separate line. This is the only limitation.

Oleg.
KGM
01/30/2007 12:44 pm
Hello,
I've a problem viewing the applet from the address below using the demo version of offline explorer enterprise:
http://math.hws.edu/javanotes/c12/s5.html

Please help
Oleg Chernavin
01/30/2007 01:17 pm
Did you try the AutoSave method with it?

Oleg.
KGM
01/31/2007 03:10 am
> Did you try the AutoSave method with it?
>
> Oleg.

yes I tried that already but still nothing. i just noticed today that as I roll the mouse through the blank applet place holder It shows that applet not found in the status bar below, the message quickly disappears though.

Any idea

thanx in advance
Oleg Chernavin
01/31/2007 09:43 am
There is a kind of error on the site - the same error happens if you browse this page online.

Oleg.
KGM
01/31/2007 10:02 am
> There is a kind of error on the site - the same error happens if you browse this page online.
>
> Oleg.

NO, I'm sure you're you are using java version 1.4 or earlier. try it on java runtime version 6 (in general try it on the latest java runtime)

I think offline explorer doesn't support the latest version of java. Any ideas, again, will be appreciated

Thanks
Oleg Chernavin
01/31/2007 10:10 am
Do you browse it with FireFox? If so, try to open the downloaded page using that browser? Internal browser of Offline Explorer is MS IE.

Oleg.
KGM
02/02/2007 11:02 am
> Do you browse it with FireFox? If so, try to open the downloaded page using that browser? Internal browser of Offline Explorer is MS IE.
>
> Oleg.

I use Internet exlorer and I wouldn't like to use firefox. Any Idea on Internet explorer. Please also remember what I said on java run time
Oleg Chernavin
02/02/2007 04:09 pm
OK. Where I can download and install that Java version?

Oleg.
KGM
02/03/2007 11:02 am
> OK. Where I can download and install that Java version?
>
> Oleg.

get it from
http://java.com/en/download/index.jsp
Oleg Chernavin
02/05/2007 07:58 am
There is only Version 5.0 Update 10 on that page. You were talking about version 6. Is there a page with the link to get it?

Oleg.
KGM
02/05/2007 11:11 am
> There is only Version 5.0 Update 10 on that page. You were talking about version 6. Is there a page with the link to get it?
>
> Oleg.

Here's the version 6 link
https://sdlc5e.sun.com/ECom/EComActionServlet;jsessionid=556DBF06D2123797FA27732B40DF64BB
Oleg Chernavin
02/05/2007 01:02 pm
OK. I installed it. It is strange, but when I view that page in the Internal browser, applet fails. If I browse it in an external MS IE - it works. Internal browser is simply the emmbedded MS IE.

Oleg.
KGM
02/06/2007 02:45 am
> OK. I installed it. It is strange, but when I view that page in the Internal browser, applet fails. If I browse it in an external MS IE - it works. Internal browser is simply the emmbedded MS IE.
>
> Oleg.

You now see what I had told you when we started. So what can I do cause the applets dont load when I export in chm either?

Thanks
Oleg Chernavin
02/06/2007 06:12 am
Well, I don't know how it works. This java machine is quite tricky - the same browser and different browsing environments - it may work or not. HTML Help is the same MS IE - no difference. I don't know why it doesn't work. This looks very strange.

Oleg.
KGM
02/06/2007 11:26 am
> Well, I don't know how it works. This java machine is quite tricky - the same browser and different browsing environments - it may work or not. HTML Help is the same MS IE - no difference. I don't know why it doesn't work. This looks very strange.
>
> Oleg.

So who's the expert to solve this, you or someone else at MP? Please let me me know as soon as possible.

Thanks
Oleg Chernavin
02/06/2007 11:45 am
If you need that java applet to be working in CHM, you probably have to contact Sun to allow applets working under this kind of environment. The links to the applet archive are correct there.

Oleg.