I then extract data from these pages and put them in a database myself. The pictures are also stored.
The website I download from is updated over a number of days.
I want download only the new pages so that when I process the data I don`t end up with duplicates in my database.
I had thought ...
...might do the trick, but that only removes files that are no longer on the server from the local copy. But the files downloaded previously are still on the server as this is only an update.
I also though...
... might work somehow, but I am working with files with no extension (generated by a database, such as...
...and I am not sure exactly how to format the syntax to delete all previously downloaded files, except the pictures.
A little help!
Instead, I would suggest you to keep all files, but export the newly loaded files to some other directory and import them to your database. You can use the AutoExport feature with the "Recently loaded files" option for that.
Would this work for you?