Macro Enhancement

Author Message
Steve Sieloff 06/09/2004 11:47 pm
Oleg --

I have recently run into the need of being able to vary 2 different parts of a URL string simultaneously with different values.

For instance, a site has a directory of companies from A to Z. The A search returns 25 pages of results, the B search 55 pages, the C search returns 28 pages ... Z search returns 2 pages. The URL has 2 name pairs that need to be modified: search.asp?name={:a..z}&...&pagecount={:25, 55,...2} or to show the 1-to-1 relationship better
search.asp?name={:a,b,c,...z}&...&pagecount={:25,55,28,...2}
.
What I want to be able to do is for each of the 26 values in the first macro use the corresponding array member in the 26 individual values in macro 2.

For now, I can set 26 separate projects or set a default value for pagecount like pagecount=55 to handle the largest search result ... both are inefficient as 1 is a waste of duplicated project setups and the other wastes server time in attempting to download non existent pages.

You set up the #1n {:1..101|10#1} {:#1+10} macro for me to vary 2 separate fields (or reuse the same value elsewhere) ... and you added the option to read values within a macro from a local file {:c:\file.txt} ... any ideas here?

Thanks!

Steve
Steve Sieloff 06/09/2004 11:50 pm
Oleg --

I should have added that, for reasons outside of this forum, the Project will not work using Filename filters in combination with Level limit unchecking (my normal approach to this type of site crawl for retriving an unlimited number of results pages)

Thanks again,

Steve

> Oleg --
>
> I have recently run into the need of being able to vary 2 different parts of a URL string simultaneously with different values.
>
> For instance, a site has a directory of companies from A to Z. The A search returns 25 pages of results, the B search 55 pages, the C search returns 28 pages ... Z search returns 2 pages. The URL has 2 name pairs that need to be modified: search.asp?name={:a..z}&...&pagecount={:25, 55,...2} or to show the 1-to-1 relationship better
> search.asp?name={:a,b,c,...z}&...&pagecount={:25,55,28,...2}
> .
> What I want to be able to do is for each of the 26 values in the first macro use the corresponding array member in the 26 individual values in macro 2.
>
> For now, I can set 26 separate projects or set a default value for pagecount like pagecount=55 to handle the largest search result ... both are inefficient as 1 is a waste of duplicated project setups and the other wastes server time in attempting to download non existent pages.
>
> You set up the #1n {:1..101|10#1} {:#1+10} macro for me to vary 2 separate fields (or reuse the same value elsewhere) ... and you added the option to read values within a macro from a local file {:c:\file.txt} ... any ideas here?
>
> Thanks!
>
> Steve
Oleg Chernavin 06/10/2004 03:08 am
Steve,

I am afraid it will be quite complex to add. Do you have some other way to do that? You can simply take a URL:

search.asp?name=a&...&pagecount=25

and copy it 26 times in the URLs field. If this is a one-time task, it is easier to do this way.

Oleg.
Steve Sieloff 06/10/2004 08:16 am
Oleg --

Yes, I have and will continue to work around ... just adding to a wish list! :-)

Steve

> Steve,
>
> I am afraid it will be quite complex to add. Do you have some other way to do that? You can simply take a URL:
>
> search.asp?name=a&...&pagecount=25
>
> and copy it 26 times in the URLs field. If this is a one-time task, it is easier to do this way.
>
> Oleg.
>
Oleg Chernavin 06/10/2004 08:28 am
OK. I will try.

Oleg.