Downloading Yahoo Groups (Yahoo's "Neo" format)

Author Message
Ryan 07/14/2016 04:02 pm
Try as I may, I have never been able to successfully download a Yahoo group since they switched to the new "Neo" group format a few years ago. I have tried numerous times over the past few years with every setting I can think of and have never found a way to successfully download a group.

Even if I could only download the text of all the messages in the group, without the attachments or the group's photos or files that would be very helpful. But I can't find a clean, efficient (or even a really funky) way of doing it. Any of those other things that could be downloaded would be icing on the cake.

Can anyone give step by step instructions including the exact non-default settings needed to do this? There has to be a way!

Thank you!
Ryan 07/26/2016 12:38 am
Or not. :(
Oleg Chernavin 07/26/2016 07:23 pm
I think, it should be possible to download this way - create a new Project with the starting address, like:

https://groups.yahoo.com/neo/groups/groud_name/conversations/messages

Open its Properties dialog, select "Open pages in the Internal browser and save". Allow to scroll pages before saving, say, up to 10 times.

Set Level to 1 and allow downloading from the starting directory in the URL Filters - Directory section.

This should be enough. Use the latest 7.2 version for these settings.

Best regards,
Oleg Chernavin
MP Staff
Ryan 07/27/2016 03:12 pm
Thank you for your reply.

I think it may be necessary for you to sign into your yahoo account before being able to access the messages. So I did that.

I also did exactly what you suggested, but unfortunately it did not work. It ran, and the "messages" directory appeared in the map as it always does when I try it, but no messages were retrieved.

Is it possible for someone at MetaProducts to actually try doing this to see if they can get it to work? It's simple to set up a yahoo account and try it.

Thanks again for the help.
Oleg Chernavin 07/27/2016 09:11 pm
I just tried that and it worked. Messages were saved. Here are my Project settings - for Offline Explorer Pro 7.2 version, do not use them for any earlier version, please.

You may select the below starting from the [Object] line, copy to clipboard and then use Ctrl+V in Offline Explorer to paste there:

[Object]
OEVersion=Pro 7.2.4518
Type=0
IID=62805
Caption=https://groups.yahoo.com/neo/groups/lowdosenaltrexone/conversations/messages
URL=https://groups.yahoo.com/neo/groups/lowdosenaltrexone/conversations/messages
Lev=1
Weekday=257
FTText.Exts=htmlhtmaspaspxjspstmstmlidcshtmlhtxtxttextxspxmlrxmlcfmwmlphpphp3
FTImages.Exts=gifjpgjpegtiftiffxbmfifbmppngipxjp2j2cj2kwbmplwfwebp
FTVideo.Exts=mpgavianimpegmovflvfliflcvivrmramrvasfasxwmvm1vm2vvobsmilmp4m4v
FTAudio.Exts=wavriffmp3midmp2m3uravocwmaapeoggm4aaif
FTArchive.Exts=7zziparcgzzarjlhalayleirarcabtarpakacejarpdftgzexeiso
FTUDef.Exts=jsaxdcssssivbsdtdxslswfclassent
FTText.B=ooxooo
FTImages.B=ooxooo
FTVideo.B=xoxooo
FTAudio.B=ooxooo
FTArchive.B=ooxooo
FTUDef.B=ooxooo
FTOther.B=ooxooo
FTSizes=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,3,3,0,3,0,0,0,0,0,0,0,0
NotIgnoreLogout=False
RSrvsBx=1
RPathBx=1
RProt=255
LastStart=124:229:84:109:101:202:228:64:
LastEnd=0:0:0:0:0:0:0:0:
PrjStart=6:214:172:141:218:252:227:64:
LastStarted=28.07.2016 4:04:13
S200=434
S400=13
SAbr=4
SPar=184
SSav=434
SLast=200
SSiz=8920224
SMdf=301
SHTML=172
LFiles=446
LSize=70162694
Descr=e1xydGYxXGFuc2lcZGVmZjB7XGZvbnR0Ymx7XGYwXGZuaWwgTVMgU2FucyBTZXJpZjt9fQ0KXHZpZXdraW5kNFx1YzFccGFyZFxsYW5nMTA0OVxmMFxmczE2IA0KXHBhciB9DQo=
ImgDim=0,0,0,0
ConvertRSS=True
SavedPagesCnt=20
DownloadWithBrowser=True
SavePageScroll=10
Ryan 07/28/2016 12:27 am
Thank you. Please tell me where exactly in OE to paste this. This looks like it goes into an .ini file maybe? Or should it be pasted somewhere else in the OE program window? I don't know where to paste it. Thanks again.
Ryan 07/28/2016 12:55 am
OK... thank you. I found out you can paste that data into the projects window and it created a new project.

However, it still did not work for me. I got the same results. A few files are downloaded, but no messages were downloaded.

Questions:

1. Do you have to be signed in to Yahoo?

2. Any idea what the problem could be?

3. How did you choose the "lowdosenaltrexone" Yahoo group as an example? That is one of my subscribed yahoo groups, but how did you know this?
Ryan 07/28/2016 01:16 am
Here is a screenshot of the map pane after I run the project. No messages are downloaded.

http://tinyurl.com/zsuathf
Oleg Chernavin 07/28/2016 06:55 pm
Ryan,

To make sure we are using the same version, can you please install the most recent one:

http://metaproducts.com/download/betas/opsetup.exe

Then open the Project Properties dialog, make sure the "Open pages in the Internal Browser and save" is selected and try to redownload this Project.

Oleg.
Ryan 07/28/2016 08:07 pm
WOO HOO! It works!!! :D

I was originally using the v7.2.0.4514 trial version that was still posted on the site as of a few days ago and maybe still is.

When I switched to this beta, it worked on the first try. I'm not sure if it is because something changed in the beta from the that release version OR there may have been something wrong with the yahoo sign in process I was using before. For whatever reason this time it worked! I suspect that it wasn't OE's fault but something I was doing wrong signing into yahoo groups. I think you must be signed in for this to work properly and I was having trouble with that on earlier tests.

Thank you Oleg!!! I think I finally can do this thanks to you!

I'm still wondering how you knew I was subscribed to that yahoo group though. Have I mentioned that group before maybe? Please tell me if you don't mind because my curiosity is raging.
Oleg Chernavin 07/28/2016 08:29 pm
That's great that it works finally!

Regarding the group - I don't use Yahoo groups myself. You asked me to take a look at this group download a year ago:

https://www.metaproducts.com/forum/Post.aspx?id=9203

I registered there and added this group to my list to test the download on it.

Oleg.
Ryan 07/30/2016 07:47 pm
Thank you Oleg. Yes, I remember that. I thought that may have been when I mentioned that group.

Thanks again for all your help!
Oleg Chernavin 07/30/2016 07:55 pm
Yes, I used this forum search to find that message.

Oleg.