Restrict Connections per Server

Author Message
Brad Konia 11/30/2004 06:07 pm
Offline Explorer allow you to limit the total number of simultaneous connections, however I don`t see any way to limit the number of connections per server (or per domain). This would be very useful as it would prevent OE from overloading individual servers, yet would still allow you to utilize the full bandwidth of your Internet connection.

Any chance of getting this feature implemented in the next release?
Oleg Chernavin 12/01/2004 03:57 am
You can add the following line to the Project`s URLs field:

Channels=3

This will limit the number of connections the Project uses. Another way to limit connections is to use the right-click context menu in the Queue tab.

This limits connections per whole Project, not per a particular server, but since Projects load mostly one of the servers, this should work for you.

Best regards,
Oleg Chernavin
MP Staff
Brad Konia 12/01/2004 09:08 am
> This limits connections per whole Project, not per a particular server, but since Projects load mostly one of the servers, this should work for you.

Oleg,

I know that you can limit connections for a project. However, it`s a pretty broad generalization to state that "Projects load mostly one of the servers." Personally I use OE for spidering the web, not for downloading individual websites. Therefore, limiting the number of connections for the project as a whole is of little value.

I notice in your Feature Comparison matrix that you listed "Server Overload Protection" as a feature, but you have "No" for all three versions of OE. Does this mean that you`re planning to add this feature soon? I`m considering purchasing the Enterprise Edition, but I`m pretty disturbed by the lack of this feature.
Oleg Chernavin 12/01/2004 09:15 am
I plan to include this feature, but I am not sure when I will do it. It is not too easy to implement it. The main reason I placed the feature in the comparison chart was to make this information as honest and as complete as possible.

Is it really urgent and important for you to have this feature?

Oleg.
Brad Konia 12/01/2004 09:40 am
> Is it really urgent and important for you to have this feature?

Well, I don`t want to throw off your whole development schedule, but the sooner the better :-)

Seriously though, I think this is a pretty important feature, not just for me, but for anyone who is doing any kind of web spidering. Many servers will ban your IP address if you make too many requests within a certain time period.

I`ve tried out a lot of different web spidering products and I feel that OE is the best one I`ve come across. However, paradoxically it`s also one of the few products that doesn`t support limiting connections by server.
Oleg Chernavin 12/01/2004 12:04 pm
OK. I added a temporary solution as a new ChannelsPerServer= Project Properties URLs field parameter. I hope it will satisfy your needs. I will also think on how to integrate this feature in the Options dialog in one of the future releases.

Thank you!

We plan to release 3.5 version this week with the new command implemented there.

Oleg.
Brad Konia 12/01/2004 12:58 pm
> OK. I added a temporary solution as a new ChannelsPerServer= Project Properties URLs field parameter. I hope it will satisfy your needs. I will also think on how to integrate this feature in the Options dialog in one of the future releases.

> We plan to release 3.5 version this week with the new command implemented there.

Thank you! That will be very helpful.
Oleg Chernavin 12/02/2004 05:28 am
Good!

Oleg.