downloading pages from php server

Author Message
Martin 08/04/2007 02:32 am
How to download site www.literotica.com.
There is for example a page with a story http://www.literotica.com/stories/showstory.php?id=308003 and the next chapter is on http://www.literotica.com/stories/showstory.php?id=308003&page=2
I made macro: http://www.literotica.com/stories/showstory.php?id={:1..300000} and http://www.literotica.com/stories/showstory.php?id={:1..3000000}&page={:1..100} and i download strange files with php?id={:1..3000000}&page={:1..100} in extensions.
so i put only 100 files in the poject. i dowloadet them and i have 2 folders with files. one is named www.literotica.com and www.literotica.com_3A81. Both of them contains files showstory.php@id=1 ..100 with extensions File PHP@ID=1 to 100 ect. Those files should by HTML ( i can chcange the extensions in other aplication). Another thing is that some files from the first folder works correctly but others are corrupted. and in the second folder those files which didnt work in the first, works ok. Files have different sizes.
How to resolve this problem??
Oleg Chernavin 08/04/2007 06:55 am
Parhaps, the site is located on two servers - www.literotica.com and www.literotica.com:81. This is why there are two folders.

To add .htm extension, please use File - Export with standard extensions box checked. If will be unable to find out what links work and for what reason, please let me know more details to look at this.

Best regards,
Oleg Chernavin
MP Staff
Martin 08/06/2007 03:14 am
The option with "Export" works well. But files in two folders have different sizes - one is 1KB and in the second folder has 23KB. Both of them works ok.
But how to make macro to download all stories. Now first of all i put one url co excell for example www.page.1, i made 100 of them, next copy and paste to notepad and to the end i copy them from notepad and paste in the project. so when i want download only 100 pages there is no problem - only when i have good urls. But i want to download 300000 url, and some of them has "=2" aded to the url. With the macro it could be easier.


Oleg Chernavin 08/07/2007 05:06 am
But why this macro doesn't work?

http://www.literotica.com/stories/showstory.php?id={:1..3000000}&page={:1..100}

It should generate all URLs for the download. It could be too many and you may run out of memory, so you could split it into several projects:

http://www.literotica.com/stories/showstory.php?id={:100000..200000}&page={:1..10}

Oleg.
Chris 08/21/2007 12:52 pm
> But why this macro doesn't work?
>
> http://www.literotica.com/stories/showstory.php?id={:1..3000000}&page={:1..100}
>
> It should generate all URLs for the download. It could be too many and you may run out of memory, so you could split it into several projects:
>
> http://www.literotica.com/stories/showstory.php?id={:100000..200000}&page={:1..10}
>
> Oleg.

Hey could you guys explain to me how to go about doing this? I've been through the php posts and so far still nothing. If i click on a page under the map, i get an error page stating "expected a semi-colon. Please correct error and refresh to load"???

Chris
Oleg Chernavin 08/23/2007 09:34 am
Can you give a sample of such page?

Oleg.