"URL filters" keywords problem

Mike12
06/13/2013 02:34 pm
My main Address (URL) in Project Properties is "www.site.com/". It contains lots of links to files from different domains, like "a.site.com/", "b.site.com/", "c.site.com/" etc.
The link for the files I need redirects to "a.site.com/" (that's what it says on Address Bar in Browser; however this site can only be accessed via the main site).
Finally, the actual address of the files I need is "1.site.com/file<#0-100>.txt". This site also can only be accessed via the main site.

How can I use server keywords to download the whole contents of "1.site.com/"?
My problem is: if I Include the "a" server keyword, so as not to get "b.site.com/" or "c.site.com/" OE wouldn't download "1.site.com/", which doesn't contain an "a"... and the other way around if I Include "1" which isn't part of "a.site.com/".

Sorry for the long post but I've tried everything to no avail.
Oleg Chernavin
06/13/2013 02:37 pm
I see two ways from your description:

1. Allow downloading from the starting domain. This will enable all *.site.com servers.

2. Use a direct link to the files if they are sequentally numbered:

http://1.site.com/file{:0..100}.txt
Referer=http://a.site.com/

Best regards,
Oleg Chernavin
MP Staff
Mike12
06/13/2013 04:00 pm

Well there are some problems:

1) If by "Allow downloading from the starting domain" you mean my Address (URL) should be "a.site.com/". It's impossible (I get a message, smth. like Forbidden, Directory Access Denied etc.). It can only be accessed via www.site.com.

2) "This will enable all *.site.com servers". By this you mean the domains like "b.site.com/", "c.site.com/" etc? Because it is exactly what I want to avoid. I only want "a.site.com"/, not some unrelated ad-sites. And I can't get it directly.

3) "Use a direct link to the files if they are sequentially numbered". Unfortunately, many of them aren't.
Oleg Chernavin
06/13/2013 04:03 pm
OK. But what about allowing three keywords:

www.site.com (for all pages that main contain useful links)
a.site.com
1.site.com

?

It is very hard to give good advices without looking at the real URLs. Perhaps, I would be able to make a much better solution.

Oleg.
Mike12
06/13/2013 04:55 pm
It worked better so far, only one problem left: "a.site.com/" has A LOT of directories, and only one of them- "a.site.com/img/" serves me to get the real files- "1.site.com/file<#0-100>.txt".
I need to filter the other directories out. I tried to put "img" in the Included dir list, but then "1.site.com/file<#0-100>.txt" doesn't match it.

(Of course I could put "a.site.com/img/" directly in the Address (URL) field, but as I said the site only allows downloading through "www.site.com".)
Oleg Chernavin
06/13/2013 04:57 pm
OK. Clear URL Filters - Servers - Included list. And add to the URL Filters - Directory - Included:

http://www.site.com/*
http://a.site.com/img/
http://1.site.com/*

Oleg.