Can I use "?" sign in filename filter?

Author Message
leecom 12/19/2004 08:53 pm
Hello,
I want to download a sub-forum in a asp site, the entry like this:
http://www.test.com/club/list.asp?boardid=123

and the items in the page of this sub-forum like this:
http://www.test.com/club/dispbbs.asp?boardID=123&ID=27547&page=1

this sub-forum also includes a lot of pages, like this
http://www.test.com/club/list.asp?boardid=123&page=2&selTimeLimit=&action=&topicmode=0


So I try to adds some filters, URL Filters-->filename-->include
list.asp?boardid=123
dispbbs.asp?boardID=123

but the OE just downloads few pages

and I try this:
list.asp*boardid=123
dispbbs.asp*boardID=123

But again just several pages is downloaded.


As this site has a lot of asp links in the page, such as: permisson.asp, action.asp, reply.asp
If I use "boardID=123" as the filename filter, I can get the things I needed but also get a great deal of garbage, more than 30,000 pages. So I have to combine dispbbs.asp and boardID=123.






Oleg Chernavin 12/20/2004 08:39 am
I just tried this on our forum and it worked well. For example, I used the following starting URL:

http://www.metaproducts.com/mp/mpSupport_User_Forums_Topic.asp?topic=7

URL Filters | Filename | Included keywords list:

message.asp?id=9008

This loaded only the specified message. You can enable logging (Ctrl-W) and filter Rejected URLs there to see exact reason why a URL was not loaded.

Best regards,
Oleg Chernavin
MP Staff
leecom 12/20/2004 12:08 pm
Thank you, the logging mechanism is great useful. However I don`t solve this problem yet.

In the log I find the following page is rejected:

Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/dispbbs.asp?boardID={$boardid}&ID={$topicid}&page={$page}

As the variable {$boardid} isn`t be replaced by the value 123, so OE reject it. This link should be like this:
http://www.test-test.com/club/dispbbs.asp?boardID=123&ID=27729&page=1
http://www.test-test.com/club/dispbbs.asp?boardID=123&ID=24873&page=1
http://www.test-test.com/club/dispbbs.asp?boardID=123&ID=27726&page=1
... ...
These links are items of the page in the sub-forum, they are the very things I need. Why does the value not be put in the links?
leecom 12/20/2004 12:09 pm
HTTP0: Connecting to host www.test.com...
HTTP0: Host www.test.com connected. Waiting for http://www.test.com/club/list.asp?boardid=123.
HTTP0: GET /club/list.asp?boardid=123 HTTP/1.0
HTTP0: Authorization: Basic c2xvdHpAQXJnZW50aW5hLmNvbToyMzYwMjI2MQ==
HTTP0: Accept: */*
HTTP0: Accept-Language: en-us
HTTP0: User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322)
HTTP0: Host: www.test.com
HTTP0: Cookie: www%2Etest%2Dtest%2Ecom%2Fclub%2F=userid=11747&usercookies=3&userhidden=2&password=57374Z8LLG536McG&userclass=%C6%BD%C3%F1&username=leecom&StatUserID=7056541831; www%2Etest%2Dtest%2Ecom%2Fclub%2FKill=kill=0; ASPSESSIONIDSSABBQSR=JNELDJBDICJJGBBCINCDMAHB
HTTP0: Transferring data from http://www.test.com/club/list.asp?boardid=123.
HTTP0: HTTP/1.1 200 OK
HTTP0: Server: Microsoft-IIS/5.0
HTTP0: Date: Mon, 20 Dec 2004 16:36:10 GMT
HTTP0: X-Powered-By: ASP.NET
HTTP0: Connection: Keep-Alive
HTTP0: Content-Length: 50633
HTTP0: Content-Type: text/html
HTTP0: Cache-control: private
HTTP0: 2% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 5% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 8% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 11% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 13% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 16% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 19% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 22% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 25% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 28% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 31% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 34% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 36% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 39% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 42% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 45% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 48% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 51% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 54% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 56% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 62% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 68% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 76% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 82% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 90% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 98% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: 100% of 50633 bytes of http://www.test.com/club/list.asp?boardid=123.
HTTP0: Download complete.
leecom 12/20/2004 12:09 pm
QUEUE: Parsing (0) http://www.test.com/club/list.asp?boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.dvbbs.net/download.asp
Rejected URL (URL Filters | Filename | Included files keywords): http://www.efutest.com/
Rejected URL (URL Filters | Filename | Included files keywords): http://www.qi-pao.com/
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/<b
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/admin_boardset.asp?boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/AllPaper.asp?boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/announcements.asp?action=showone&boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/bbseven.asp?boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/boardhelp.asp?boardID=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/BoardPermission.asp?boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/BoardPermission.asp?boardid=123&action=Myinfo
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/boardstat.asp?action=lastbbsnum&boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/boardstat.asp?action=lasttopicnum&boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/boardstat.asp?boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/boardstat.asp?reaction=online&boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/boardstat.asp?reaction=onlineinfo&boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/boardstat.asp?reaction=onlineUserinfo&boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=hidden&userid=11747
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=0&boardid=123
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=0
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=1
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=10
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=12
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=13
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=14
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=15
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=16
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=2
Rejected URL (URL Filters | Filename | Included files keywords): http://www.test.com/club/cookies.asp?action=stylemod&skinid=2&boardid=123&cssid=3
Rejected URL (URL Filters | Filename | Included files
Oleg Chernavin 12/20/2004 12:25 pm
Can you please send me your Project settings to support@metaproducts.com ? Select the Project, click the Copy button on toolbar and then paste it to the E-mail message.

Thank you!

Oleg.