Cannot download Washington Post Op/Eds from Yahoo!

Author Message
Hallmarc 12/01/2004 12:42 pm
Hello,

My organization is trying to archive a large number of Op/Ed pieces from various sources.
For some reason, Washington Post articles never download properly.
My project source URL uses a macro and tries to download all the existing OpEds from a Yahoo! archive with level limit 1:

http://story.news.yahoo.com/fc?tmpl=fc&cid=34&lp=2&ll=b2&pg={1..95}&mod=opinion___editorials&in=world&cat=mideast_conflict_archive

The Washington Post articles on these pages resolve to URLs of the form:

http://us.rd.yahoo.com/dailynews/fc/World/mideast_conflict_archive/opinion___editorials/SIG=125ts1bo8/*http://www.washingtonpost.com/wp-dyn/articles/A47947-2004Apr27.html

If you enter this URL directly into a browser, it reloads with a URL of:
http://www.washingtonpost.com/ac2/wp-dyn?pagename=article&contentId=A47947-2004Apr27&notFound=true

Try as I might, I CANNOT get OE 3.4 to download these pages.
Oleg once gave me a solution before, but I can no longer find it.

Please advise.

Regards,
Marc
Oleg Chernavin 12/02/2004 06:58 am
Maybe it is because the URL Macro is entered incorrectly: {1..95}, while it should be {:1..95}. Does this help?

Best regards,
Oleg Chernavin
MP Staff
Hallmarc 12/02/2004 01:27 pm
No, I`m sorry I mistyped it, it is actually correct in the project. You actually answered this for me once before, but I lost the email.

Regards,
Marc

> Maybe it is because the URL Macro is entered incorrectly: {1..95}, while it should be {:1..95}. Does this help?
>
> Best regards,
> Oleg Chernavin
> MP Staff
Oleg Chernavin 12/02/2004 03:47 pm
Did I answer by E-mail to you directly or in the forum? Can you please give me the name or E-mail under which you wrote me or posted a forum message, so I could find it?

Oleg.
hallmarc 12/02/2004 04:00 pm
> Did I answer by E-mail to you directly or in the forum? Can you please give me the name or E-mail under which you wrote me or posted a forum message, so I could find it?
> > Oleg.

I thought it was answered in the forum, but I don`t see it anywhere. I believe it was sometime between Aug `02 and Aug `03. If we corresponded only through email, it was probably with hallmarc@fastmail.fm or chiamarc@sbcglobal.net. Thanks again.

Marc
Oleg Chernavin 12/03/2004 09:31 am
Is it the following post?

http://www.metaproducts.com/forum.asp?id=5004

Oleg.
Hallmarc 12/03/2004 09:55 am
Thanks! I guess it was a lot more recently than I thought (been doing this job for a while)...is there a search page for the forums that is functional?

Regards,
Marc

> Is it the following post?
>
> http://www.metaproducts.com/forum.asp?id=5004
>
> Oleg.
Oleg Chernavin 12/03/2004 09:57 am
We had some problems with the search and we had to disable this feature for a while. I hope we will fix them soon.

Oleg.
hallmarc 12/05/2004 08:57 pm
Oleg,

Thanks for referring me to the post, but sadly, the solution doesn`t seem to work any more. Would you be kind enough to investigate again, using the same problem description? Thanks.

Regards,
Marc

> We had some problems with the search and we had to disable this feature for a while. I hope we will fix them soon.
> > Oleg.
Oleg Chernavin 12/06/2004 09:57 am
I just tried the following Project:

URL:
http://story.news.yahoo.com/fc?tmpl=fc&cid=34&lp=2&ll=b2&pg={:1..95}&mod=opinion___editorials&in=world&cat=mideast_conflict_archive
Level=2
URL Filters | Server and Directory sections - Load from all...

This loaded all pages well. With Level=1, some Washington Post pages were really not loaded.

Oleg.