Download link changes

Author Message
Allen 11/01/2005 09:35 pm
Hello Oleg, I am currently trying to figure out how to visit this URL and download certain pages
http://www.vbulletin.com/forum/search.php?&do=getdaily

I am able to download all pages I need by excluding "&nojs=1"

and including

search.php?&do=getdaily

&pp=30&page=*

What is happening is each time I visit http://www.vbulletin.com/forum/search.php?&do=getdaily It redirects to a unique URL so I geta popup telling me and the project start URL gets changed to

http://www.vbulletin.com/forum/search.php?searchid=393108
OriginalURL=http://www.vbulletin.com/forum/search.php?&do=getdaily

Now what happens is the next day it goes to http://www.vbulletin.com/forum/search.php?searchid=393108 and it downloads this single page and the "searchid" has expired, is there some way for it each day to visit http://www.vbulletin.com/forum/search.php?&do=getdaily and download all the pages I have filtered eg; &pp=30&page=* without invoking the popup and asking me for confirmation?
Oleg Chernavin 11/02/2005 07:34 am
What about also including:

search.php?searchid=

?

Best regards,
Oleg Chernavin
MP Staff
Allen 11/03/2005 02:47 am
That works almost perfect Oleg but now I download a few extra files and I am not sure how to filter

I want these files
---------------------------------------------------------------------------------------------
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30&page=5
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30&page=9
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30&page=8
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30&page=7
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30&page=6
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30&page=4
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30&page=3
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30&page=2
http://www.vbulletin.com/forum/search.php?searchid=398539&pp=30
---------------------------------------------------------------------------------------------

I do not want these 2 files to download, they are duplicates of the last one above

http://www.vbulletin.com/forum/search.php?searchid=398539
http://www.vbulletin.com/forum/search.php?&do=getdaily

---------------------------------------------------------------------------------------------

This is what I am using now...


[Object]
OEVersion=Enterprise 4.0.0.2232
Type=0
IID=9788
Caption=http://www.vbulletin.com/forum/search.php?&do=getdaily
URL=http://www.vbulletin.com/forum/search.php?&do=getdaily
Lev=1000000
Weekday=257
LimTSize=10000
LimNumber=5000
LimTime=100
FMGroup=1
pswMethod=4
FTText.Exts=aspcfmhtmhtmlhtxidcjspphprxmlshtmlstmstmltexttxtwmlxmlxsp ooxxoooxooooooooo
FTImages.Exts=gifjpgjpegtiftiffxbmfifbmppngipxjp2j2cj2kwbmplwf
FTVideo.Exts=mpgavianimpegmovfliflcvivrmramrvasfasxwmvm1vm2vvob
FTAudio.Exts=wavriffmp3midmp2m3uravocwmaape
FTArchive.Exts=ziparcgzzarjlhalayleirarcabtarpakacejar xxxxxxxxxxxxxx
FTUDef.Exts=aspcssdtdjsssivbsxsl xxxxxxx
FTText.B=ooxxoo
FTImages.B=xoxooo
FTVideo.B=xoxooo
FTAudio.B=xoxooo
FTArchive.B=xoxooo
FTUDef.B=xoxooo
FTOther.B=xoxooo
FTSizes=0,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
RSrvsBx=3
RPathBx=1
RFileBx=2
RFileIn=search.php?&do=getdailysearch.php?searchid= ox
RFileEx=&nojs=1 x
RProtBx=1
RProt=1
LastStart=147:68:225:206:96:224:226:64:
LastEnd=115:89:154:208:96:224:226:64:
S200=11
SAbr=1
SPar=11
SSav=11
SLast=200
SSiz=154971
SMdf=11
LFiles=11
LSize=154971
ImgDim=0,0,0,0
PrevURL=http://www.vbulletin.com/forum/search.php?&do=getdaily
SkipURLs=





> What about also including:
>
> search.php?searchid=
>
> ?
>
> Best regards,
> Oleg Chernavin
> MP Staff
Allen 11/03/2005 02:48 am
Note that searchid=* changes each time you login to a different number

Thanks again for the help Oleg!
Oleg Chernavin 11/03/2005 09:58 am
Well, the ...&do=getdaily file has to be downloaded, because it is the first URL you download. If you disallow it, how the download will start?

Oleg.
Allen 11/03/2005 11:40 am
Okay..I thought perhaps there was a way to start at that url but not download that page..appreciate the help!

> Well, the ...&do=getdaily file has to be downloaded, because it is the first URL you download. If you disallow it, how the download will start?
> > Oleg.
Oleg Chernavin 11/04/2005 08:09 am
You can try to use DeleteAfterParsing= command in the URLs field to remove that file after downloading it and following links in it.

Oleg.
Allen 11/05/2005 06:50 am
I tried using this in properties but cannot seem to get file to delete

http://www.vbulletin.com/forum/search.php?&do=getdaily
DeleteAfterParsing=search.php@&do=getdaily

*Is this the proper way or am I missing something? This is the exact name of the file after downloading and when I check propertys of the file I see

Type of file: PHP@&DO=GETDAILY File

**These are basically php pages being downloaded from a forum

> You can try to use DeleteAfterParsing= command in the URLs field to remove that file after downloading it and following links in it.
> > Oleg.
Oleg Chernavin 11/07/2005 06:10 am
What about:

http://www.vbulletin.com/forum/search.php?&do=getdaily
DeleteAfterParsing=*&do=getdaily

Oleg.
Allen 11/07/2005 01:30 pm
No this does not work and file is not deleted
Oleg Chernavin 11/08/2005 05:36 am
OK. I just changed the code, so it will now work. I plan to release 4.0 Beta 2 version this week.

Oleg.
Allen 11/08/2005 12:06 pm
Thank you Oleg, appreciated.
Oleg Chernavin 11/08/2005 12:09 pm
You are welcome!

Oleg.