|Edwards||07/26/2004 09:55 pm|
|How to force the application to obay robot.txt?
I hope there is a feature or parameter to make it respect this file.
Web masters get very angry if you don`t.
Your application has a very good reputation, and having a feature like that will add value.
|Oleg Chernavin||07/27/2004 07:26 am|
|Sorry, but we haven`t added this feature yet. We still have it in our plans, anyway.