clean old (non-working) links in the OE download databases?
|Nick||12/21/2010 10:06 am|
I'm curious if it's possible to clean old non-working links in the project databases.
Let me describe what I mean:
I -for example- want to download pictures from image boards like 4chan.
After some time OE wants to download (or check) lotsa files when I start the project, so it might say 50000 files in queue.
Image board threads are short-lived, after some time the same thread-html-file will disappear and never come back. But OE still tries to load it to see if there are new/changed files to download in these thread-files.
So OE starts to download the project, suddenly does some (time/cpu-taking) parsing and suddenly is down to let's say 5000 files left to download.
To clean up I usually copy the project, delete it, paste the project and start to download it again. Doing this it downloads the whole thing again because it lost the information of already downloaded project files. (Delete -> only project files -> keep informations about already downloaded files)
That's a problem, because it might download files that I already downloaded before and downloading all again causes a lot of traffic.
Is there a way to -instead of "copy a project, delete, paste, start anew"- to just clean the project from URLs that will never appear again?
So that OE deletes these files that don't exist anymore on the server from it's project database, but keeps the files in the project download folder?
I hope you see what I try to describe :)
Thanks for any informations,
|Oleg Chernavin||12/21/2010 12:27 pm|
|Well, if the server really tells Offline Explorer that this page no longer exists (404 code in the responce header), then you may add the following line to the Project's URLs field:
At the end of the download, all files that belong to the project, but actually do not exist on the server, will be deleted.
|Nick||12/21/2010 01:21 pm|
|Dear Oleg Chernavin,
but just to be sure: will this command delete the downloaded files (in the project download folder) or just the informations in it's already downloaded files database?
(since I don't check everytime a project download finished, I just want to make sure that already downloaded files will not be deleted from harddisk)
|Oleg Chernavin||12/21/2010 01:47 pm|
|It will delete the files. No database is used during the download. All downloads go this way - Offline Explorer loads the first page and follows its links and so on. It simply compares the size and modification dates of each file with the previously downloaded ones.