1) The problem: In the "File Modification Check" settings, I use the check "Skip files on levels higher than...1"; after using OEP for some minutes, downloading that or other sites, I find the setting (radio button) in "Download only modified and new files". That is: the configuration was changed ! I may repeat the setting to "Skip files on levels higher than...1", and after a some runs, randomly it is set again in "Download only modified and new files".
Is it possible that if the configuration file (webdown.dat) is being used by a running download project, and if I change the configuration of another project, it may produce the above error ??
2) The question: If I set a file filter to exclude for example "redirect.html", when OEP finds a redirect.html file will it "execute but not save it" ? That is: the filters mean that that files will not be SAVED, but it WILL be CRAWLED ?
Thank you very much,
2. All filters except Content Filters do not allow even downloading a file (crawling) when you disable certain URLs using them.
I have posted more information about my questiones:
> 1. This is possible if you have one or more Projects selected and use Ctrl+F5, Alt+F5 or Shift+F5 hotkeys
> or the submenu of the Download button, which change the selected Projects settings and start the download.
The situation was like this: I have had 1 (one) project downloading. At the same time, I was reviewing the settings of some other projects (not running). In about 40% of these reviewed projects, I have found that the File Modification Check settings were changed (and I didn't change them).
I'm very worried about this, because now I don't feel confident that my project's settings are kept along the time, and I don't want to spend time everyday checking all the project'ts settings...
> 2. All filters except Content Filters do not allow even downloading a file (crawling) when you disable
> certain URLs using them.
Then I don't fully understand how OEP works. I have a project that downloads a news site in which most of the links go to a "redirect.html=variable number" file. After downloading it for several weeks, I realized that I have had about 175,000 redirect.html files stored in the project's folder (and also the related content files) !!!.
So I put "redirect.html" in the excluded filenames.
Now I don't have more redirect.html files stored, but I'm wondering if I am downloading all the site's content...
Thank you very much for your always excellent service !
Thank you very much.
2. In this case you have to use DeleteAfterParsing= command in the URLs field of the Project.
> 1. Do these 40% of projects have the same setting or they are different, but changed from the original?
All the projects are different because they crawl a different URL. However, many of the settings of one project are derived from another project. Is this a problem ?
Please note that when I report that I found a project's setting changed was from a previous "known" state; they were not derived from a "wrong" master project.
> 2. In this case you have to use DeleteAfterParsing= command in the URLs field of the Project.
> Frankly, I have no idea why this may happen except that you selected several Projects and used Ctrl+F5 or Alt+F5 or Shift+F5 keys to start them - these keys change Project File Modification Check setting.