Changing queue sequencing

Author Message
ssieloff 11/23/2005 05:21 pm
Oleg --

Happy Thanksgiving and thanks for the continued great work and support of OE Pro!!!

I am starting to run into several sites where the sequencing of navigational links (series of next pages) is dependent upon a hash key/serach ID in order to resolve the next set of links to dispaly ... page 2`s 20 links are dependent upon page 1`s search ID ... each search generates a new, unique ID for tracking the results and presenting them on mutiple pages.

OE Pro currently stacks the queue first in, first out putting next page navigational links at the bottom of the queue in order that they were encountered/parsed. The problem is that other searches are executed (via the already queued links/URLs) and by the time these are executed the search ID is now where near what it was and the results, if any, are invalid.

Is it possible to instruct OE to do a depth first crawl (or loading) of the queue ... that is, have OE Pro parse page 1 ... stack links found to be executed next (page 2) ... retrieve and parse page 2 causing any page 3 links to be added next (and not to bottom of queue) ... then, when there are no further links revert back to next link of original queue.

I see this working as the main project loads a primary queue of desired links/searches ... each top link then loads a secondary queue of links in order of processing and OE exhausts the secondary queue of all its links before proceeding to next link in primary queue.

Does any of this make sense to you?

I can provide examples if you like of sites that have this type of navigational style when presenting multi-page search results ...

Thanks,

Steve
Oleg Chernavin 11/25/2005 03:07 am
As far as I understand you, adding the following line to the Project`s URLs field will help:

Additional=DepthFirst

Best regards,
Oleg Chernavin
MP Staff
ssieloff 11/25/2005 10:34 am
Oleg --

Thanks for the assistance ... I will try this and let you know if it helps ... sorry to bother you when you had it covered via an additional parm ...

> As far as I understand you, adding the following line to the Project`s URLs field will help:
>
> Additional=DepthFirst
>
> Best regards,
> Oleg Chernavin
> MP Staff
Oleg Chernavin 11/25/2005 01:02 pm
That`s not a problem!

Oleg.