Saving List of All URLs in the Project

Author Message
Hadi 02/18/2009 11:16 pm
How can I save a list of all URLs downloaded by Offline Explorer in my project to a text file or database?
This is very useful and necessary.
And second question;
How can I save a list of all URLs downloaded in my project with the file name and address for each URL after exporting project to folder, to a text file or database?
Is it possible to save the “Map” and “Level” data of a project to a text file or database format?

Finally,
Is there a feature in Offline Explorer that exports project data (such as URLs, maps, levels, names and addresses of files in exported project and etc.) not project files (downloaded files and web pages) to text file or database or even a file format can be converted to text?
Thank you
Hadi.
Oleg Chernavin 02/19/2009 07:44 am
I think, the best way would be to use Tools - Make Google Sitemap feature. It will create an XML file with the Map contents.

Best regards,
Oleg Chernavin
MP Staff
Hadi 02/19/2009 09:38 am
> I think, the best way would be to use Tools - Make Google Sitemap feature. It will create an XML file with the Map contents.
>
> Best regards,
> Oleg Chernavin
> MP Staff

Thank you for your useful guide.
But I think the “Make Google Sitemap…” command is not a complete solution for this problem.
1. It lists only web pages, not all files in a project.
2. There is no information about the location of files after downloading, and there is not any report about changing file names after download and export.
3. And a full report about a file contains the date and time of downloading.
Offline Explorer is really the best offline browser; but I can’t find this necessary part in it, yet.
I will be very happy if you help me about this.
By best wishes
Hadi
Oleg Chernavin 02/19/2009 11:13 am
OK. Can you give me an example of a text file that would be perfect for you?

Oleg.
Hadi 02/19/2009 02:44 pm
> OK. Can you give me an example of a text file that would be perfect for you?
>
> Oleg.
Here you are a basic sample on reporting project data. This report can be in text, XML or a general database format. This is a basic sample and can be improved by common methods in reporting digital acts.
Hadi 02/19/2009 02:57 pm
Page 1 of 3 (SAMPLE)

Project Name: http://www.metaproducts.com/
Project Address: http://www.metaproducts.com/
Limit Level = 1
Filenames format in export: Joliet - CD-R 64 symbols
... and other project details
List of Files Downloaded in this project:
*************************************************************
URL: http://www.metaproducts.com/default.htm
Local Address: D:\MetaPro\default.htm
Address on Exported Package: D:\MetaPackage\www.metaproducts.com\default.htm
Changes after Downloading: (None)
Changes after Exporting: (None)
Download Date and Time: February 19, 2009, 9:56:02 PM
Links to this file: (None)
Links on this file:
http://www.metaproducts.com/mp/mpProducts_List.asp
http://www.metaproducts.com/mp/mpProducts_Features.asp?id=10
http://www.metaproducts.com/mp/mpProducts_Downloads_Current.asp
http://www.metaproducts.com/mp/metaproducts_download_express.htm
http://www.metaproducts.com/mp/mailsite.asp?URL=
http://www.metaproducts.com/mp/images/mp-logo-gray.gif
http://www.metaproducts.com/mp/images/mpRight_Arrow_Small.gif
http://www.metaproducts.com/mp/images/mpLeft_Arrow.gif
...
...
... and other links on this page
--------------------------------------------------------------------------------
Hadi 02/19/2009 02:59 pm
Page 2 of 3 (SAMPLE)
--------------------------------------------------------------------------------
URL: http://www.metaproducts.com/mp/mpProducts_Detail.asp?id=10
Local Address: D:\MetaPro\www.metaproducts.com\mp\mpProducts_Detail.asp@id=10
Address on Exported Package:
D:\MetaPackage\www.metaproducts.com\mp\mpproducts_detail.aspid10.htm
Changes after Downloading: Yes (?)to(@)
Changes after Exporting: Yes (?)to()
Download Date and Time: February 19, 2009, 9:56:25 PM
Links to this file: http://www.metaproducts.com/default.htm
Links on this file:
http://www.metaproducts.com/mp/mpCart.asp
http://www.metaproducts.com/mp/mpServices.asp
http://www.metaproducts.com/mp/images/BuyNow_Big.gif
...
... and other links on this page
--------------------------------------------------------------------------------
URL:
http://www.metaproducts.com/mp/images/pages/mpCompany_Press_Image_DE_Small.gif
Local Address: D:\MetaPro\www.metaproducts.com\mp\images\pages\mpCompany_Press_Image_DE_Small.gif
Address on Exported Package: D:\MetaPackage\www.metaproducts.com\mp\images\pages\mpcompany_press_image_de_small.gif
Changes after Downloading: (None)
Changes after Exporting: (None)
Download Date and Time: February 19, 2009, 9:56:04 PM
Links to this file: http://www.metaproducts.com/default.htm
Links on this file: (None)
Hadi 02/19/2009 03:04 pm
Page 3 of 3 (SAMPLE)
This sample was about 3 files in a project. This data should be written about each file in project.
Hadi 02/19/2009 11:11 pm
Correction on Page 1 of 3:
The correct Local Address is:

Local Address: D:\MetaPro\www.metaproducts.com\default.htm
Oleg Chernavin 02/20/2009 05:44 am
I see. In fact, listing all links for each page will make this file huge - it will be almost the size of the whole exported site.

We are working on a special product or addition to Offline Explorer that will show the site structure with all links, filtering and so on. It will allow to make detailed reports, show all errors on the server, etc.

Oleg.
Hadi 02/22/2009 03:49 am
Suggestion Page 1 of 2
In fact, I added the fields “Links to This File” and “Links on This file” to report to making an outlook of a complete report; but all fields in the sample report are not at the same degree of importance.
Creating a list of URLs downloaded is the most important field. I believe an offline browser without this availability is not acceptable. However this is existent in Offline Explorer now by command “Copy URL”; but it is limited to contents of one folder without including subfolders. My first suggestion is:
• Removing limitation of “Copy URL” command and making it follows the rules of selecting files and folders in Windows. “Select All” command will selects all files and folders in the project and Copy URL can send theme to Clipboard, Selecting a folder will selects the folder and all files and subfolders in it. Preparing a check box in Options dialog box with title “Select subfolders when selecting a folder” is a good idea for answering different needs.
Hadi 02/22/2009 04:07 am
Suggestion Page 3 of 3
Yes, reporting relations between files (listing links to the file and links at the file) makes heavy reports. And turning it over a special product or an add-on for OE is more logical.
In the abstract I suggest these points, instead of referring the whole solution of problem to a special product:
1. Removing limitation of Copy URL command for reporting URLs.
2. Preparing a basic report in OE for listing local addresses, addresses after exporting, and the date and time.
3. A full report about the project information by special program or an add-on for OE.
Very thanks for consideration.
Hadi
Hadi 02/22/2009 05:04 am
After Editing by a Moderator
What are the arguments in a message that need to be edited by a moderator? I couldn’t publish the second page of my message even after dividing it to two parts because of that. Therefore you see only the page one of the message and second part of page two titled “page 3 of 3”. I will send the first part if didn’t appear later.
Oleg Chernavin 02/22/2009 08:46 am
I see. I can add the feature to copy all URLs to clipboard. But for huge projects, copying millions URL would eat almost all memory and may take a long time.

What about adding such option only to Export? It would create a report with all files of a Project - the list would contain online URLs, filenames and exported filenames. Perhaps, also file size, MIME type, etc. (optional).

What do you think?

Oleg.
Hadi 02/22/2009 01:45 pm
It’s very good. There is no urging on improvement of “Copy URL” but the accent is on availability of a basic report on any edition of OE because the user that downloads a web site needs the information about what he downloads; If making a basic report about the project needs to creating another project on another software by same setting this means that the solution was not well.
Yes full reporting about the site structure and relations between files is not necessary always. There in no intricacy if it was possible by another product.
Hadi 02/22/2009 04:05 pm
Correction on Last Message (Hadi 2/22/2009 1:45:11 PM )

Last Sentence: There in no....

The Correct: There is no....
Oleg Chernavin 02/23/2009 10:51 am
OK. I added a simple report to the Export dialog. You may test this updated oe.exe file:

http://www.metaproducts.com/download/betas/OEP2962.ZIP

Oleg.
Hadi 02/23/2009 02:08 pm
It’s working very well!
Very thanks for your surprising answer!

By best wishes
Hadi
Oleg Chernavin 02/24/2009 06:19 am
You are welcome!

Oleg.