I am downloading pages generated from a database. The database has a master/details format, with the detail pages including pictures.
I then extract data from these pages and put them in a database myself. The pictures are also stored.
The website I download from is updated over a number of days.
I want download only the new pages so that when I process the data I don`t end up with duplicates in my database.
I had thought ...
Additional=DeleteOldFiles
...might do the trick, but that only removes files that are no longer on the server from the local copy. But the files downloaded previously are still on the server as this is only an update.
I also though...
DeleteAfterParsing
... might work somehow, but I am working with files with no extension (generated by a database, such as...
detail.asp@cor=xx&exh=12345678
...and I am not sure exactly how to format the syntax to delete all previously downloaded files, except the pictures.
A little help!
I then extract data from these pages and put them in a database myself. The pictures are also stored.
The website I download from is updated over a number of days.
I want download only the new pages so that when I process the data I don`t end up with duplicates in my database.
I had thought ...
Additional=DeleteOldFiles
...might do the trick, but that only removes files that are no longer on the server from the local copy. But the files downloaded previously are still on the server as this is only an update.
I also though...
DeleteAfterParsing
... might work somehow, but I am working with files with no extension (generated by a database, such as...
detail.asp@cor=xx&exh=12345678
...and I am not sure exactly how to format the syntax to delete all previously downloaded files, except the pictures.
A little help!
I would suggest you not to delete files, because when you download the site the next time, Offline Explorer will try to download the deleted files again and your system will consider such files as newly loaded.
Instead, I would suggest you to keep all files, but export the newly loaded files to some other directory and import them to your database. You can use the AutoExport feature with the "Recently loaded files" option for that.
Would this work for you?
Best regards,
Oleg Chernavin
MP Staff
Instead, I would suggest you to keep all files, but export the newly loaded files to some other directory and import them to your database. You can use the AutoExport feature with the "Recently loaded files" option for that.
Would this work for you?
Best regards,
Oleg Chernavin
MP Staff