Links beginning with #

Author Message
Coriolis 05/21/2009 01:20 am
Is there any way to tell OEP that URLs beginning with a pound sign (#) should be sent to the server as-is, rather than treating them as anchors and truncating the URL at the pound sign? DeviantArt uses URLs of the form "http://artist.deviantart.com/gallery/#_featured--2" to access the second page of an artist''s featured work, but OEP appears to send this as "http://artist.deviantart.com/gallery/", resulting in a different page.
Oleg Chernavin 05/21/2009 05:46 am
Actually the # symbol makes browser to simply scroll inside the page after it is loaded. But it is not sent to the server in any case.

I used a sniffer application (WireShark) to see what browsers send to the server when you load such URL. MS Internet Explorer 8.0:

GET /gallery/ HTTP/1.1
Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/x-icq, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
Accept-Encoding: gzip, deflate
Host: artist.deviantart.com
Connection: Keep-Alive
Cookie: .......

FireFox 3.0:

GET /gallery/ HTTP/1.1
Host: artist.deviantart.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: ......

As you see, there is no # symbol that was sent by the browser to the server.

Best regards,
Oleg Chernavin
MP Staff
Coriolis 05/21/2009 01:44 pm
Okay, then, I don''t understand how this works. If I close all browsers and then open "http://aimeekitty.deviantart.com/gallery/#_featured--2", I get a different page than if I close everything and open a new browser window for "http://aimeekitty.deviantart.com/gallery/". How is this possible if the same URL, referrer, and cookies are being sent in each case? Thank you.
Oleg Chernavin 05/21/2009 02:11 pm
It is a script that changes the page contents. #featured is a non-working link. When you click the link, browser start the script that does the change, but the page is not reloaded.

Oleg.
Coriolis 05/21/2009 02:36 pm
Is this script not supported in OEP?
Oleg Chernavin 05/21/2009 02:40 pm
This script doesn''t work offline. I tried to understand why some time ago, but failed. Sorry.

Oleg.
Coriolis 05/21/2009 02:46 pm
Okay, thank you for clearing that up!
Oleg Chernavin 05/21/2009 03:07 pm
You are welcome! Sorry that I am unable to help.

Oleg.
bodyartist 06/02/2009 10:13 am
> You are welcome! Sorry that I am unable to help.
>
> Oleg.

I have been trying to download a http://username.deviantart.com site, to back up my galleries.

I sent a note to support, but did not get an answer over a month ago.

There has to be a way to download the site, even if the urls have to be entered manually, but for some reason, I can''t get OEP to get past the first page -- to any page with a #

If the script doesn''t work off line, the data should still be downloaded since the requests are made on-line. But, if it''s known that this doesn''t work on this site, isn''t there a way to make it so the translated URL is stored in the off-line copy?

If the browser -- MSIE and Firefox -- can view the pages, shouldn''t OEP be able to view them, and save them? If the off-line translation of the script isn''t working, the on-line script should still display/download the page, even if it won''t link in the viewable/browsable off-line copy.

What I don''t understand, is if the browser can view the pages, why the OEP program can''t crawl the pages the same way.

Oleg Chernavin 06/02/2009 10:46 am
I am sorry - I tried to do that some time ago, but I haven''t understood the script logic at all and why it doesn''t work offline. I only suppose there is a kind of AJAX linking that is not yet supported by Offline Explorer.

Oleg.