Support for migration of a Yahoo! group

(Mitch Davis) #1

Hello. My apologies, this is almost certainly miscategorised, but there just doesn’t seem to be a “dumb questions” category, and your FAQ seems to be fluff not stuff people have actually asked.

I’m a user of a popular open source program. That program has a Yahoo! group. A recent thread in that group mentioned the brokenness of Yahoo! Groups, and the overwhelming consensus from the membership (and from the main op) is that a change to something-anything-not-Yahoo!-groups is well overdue.

Two questions:

  1. Is it possible for an ordinary subscriber to use Discourse purely through email, as one would use a conventional mailing list such as mailman?

  2. Is it possible to migrate content from Yahoo! Groups to Discourse?  (I am thinking of something like the Google Groups migration API:
        Get Started  |  Groups Migration API  |  Google Developers )

I’ve read the “is forum migration worth it” topic. The Yahoo! content doesn’t need to disappear, and nothing special needs to be done to keep links working. I guess it’s more of a membership acceptance thing. More people are likely to agree to a move if they know the old content is already there.

Thanks for your help.



Edit: you ‘can’ respond to topics by email. The rest of my post is fluffy philosophy stuff and junk opinions (so skip to the next response).

Edit 2 Well apparently my email came through with ‘mixed success’. I smell exception handling in someone’s future.

I’ll chime in (someone else may want to add or correct me if I am wrong).

Beyond various notifications that lead a user back to a discourse site from email. One would have to log in on (visit) to write or respond to a conversation.Exception there is an API, but that is mostly for pulling stuff for display ATM but beyond the API limitations (which will become more robust over time), emailing conversations wouldn’t work here because it adds its own nuanced complexities you have to pass all sorts of special things that are client specific. who are you? what are you passing? where do I put the stuff your are send me?

And god forbid someone comes up with an awesome email spam situation so you have to account for frequency and request thresholds through some sort of listener module.

More than that, I believe the philosophy is to aggregate users/usage to participate on your particular forum ‘visitors’ which increases your presence/importance so your forum pulls even more users creating richer experiences. If most people are talking through email then parts of that is missing. Even if conversations are growing, the visitor numbers becomes fractional. Essentially a ghost forum.

Someone else might know more, but this is what I am seeing so far.

Someone else would have to answer your second question.

(Benjamin Kampmann) #3

That isn’t entirely correct.

Once enabled, discourse does have a reply-by email feature, as well as posting via email and via the User Setting Email all posts Users can receive everything per mail. So overall, you can emulate an Mailinglist behaviour through discourse by now as for e.g. also discussed here.

It isn’t very feature-rich though. There is no way to change the noise per topic via email (one of the great features of Discourse) for example or support CC/BCC’ed emails.

If you just want another ML manager, I think Discourse will be missing a few features that at least some Pro’s aren’t willing to let go of. If on the other hand, you are also up for a cultural change, then discourse is definitely worth a try as its conversation focus and UI is by far better than any ML manager I’ve encountered ever.

The question I’d have though, would be whether Yahoo Groups has a decent export function. Because often people want to keep the history as well and I dunno about any one-click yahoo-to-discourse-importer yet. (There was a discussion about important google groups once though…)

(Tobias Eigen) #9

@CapnKernel @lightyear There used to be a great script to export yahoo groups email to mbox files, which could then be imported to various other things like mhonarc mailing list archives or forums. I used it a number of times. Looks like it stopped working in 2013 though and I don’t know why.

Maybe you have a good archive in your own email of message history for your yahoo group and can use that. The nice thing is that once the email is in an IMAP folder you can open up two mailboxes in an email client and then drag mail from one account’s folders to another… and then set up discourse to access that mailbox and grab the messages.

As has been discussed here already, the email-in functionality in discourse is not yet very robust. I tried importing a bunch of messages and was not very successful. I’m eager to try again though and see if we can identify these issues so that importing mailing list archives becomes feasible.

One tool that really licked this problem successfully a long time ago is the unfortunately named FUD Forum. There is an accompanying script called maillist.php that worked very well - you could just pipe in the mbox files contents with the following command and each message would be processed individually, putting the messages in the right forum, creating user accounts as needed, attaching files etc. Really neat.

cat /home/user/old.mbox | formail -s php -f /path/to/maillist.php 1

On the other hand, SMF refused to implement yahoo group killer functionality back when I was trying to convince them of it… if I recall the developers shared the view espoused above that forums and mailing lists are just too different.

Email from Yahoo group marked as spam
(Benjamin Kampmann) #12

I wouldn’t generally say you can’t import MLs into Forums. It is just as both are text-based “format parsing” is quite an issue, especially since they have very different conventions (just think of inline-replies vs “discourse quotes” … gaaah).

So imported data - for the time being - would mostly not look very good in most forums.

Email-in as a functionality won’t work in your favour either, because it ignores most meta-information like the send-date (all will be created at the moment of “import”) and does require all users to be signed up with these exact email-addresses. If you make the later happen you could technically at least get all the content on-board.

Though my current approach for these kind of efforts is to simply create a google-search-able HTML-Archive of old things and then have a fresh start with the new tool you want to be using.

(James Milligan) #13

I run a Yahoo Groups forum and like @CapnKernel we’re looking at alternatives. I’ve been following Discourse for 6 months or so, and was under the impression that posting new topics from email wasn’t implemented yet, which has been the major issue stopping me from moving to it. From your reply (@lightyear) that doesn’t appear to be the case? Is there any more detail on this at all?

(Tobias Eigen) #14

@lake54 there’s a bunch of info on meta on this - just search for email. Here’s one started by @lightyear

Below is a screenshot of the category settings page on my discourse where I have set up a volunteers category and specified a email address discourse looks for to fetch email for the category. I have it set up as an alias for the discourse pop mailbox on google apps.

This functionality works as long as the sender email address is already in discourse, although I have had experiences with it failing silently… probably because I was trying it out with attachments that were too large for discourse to handle.

(James Milligan) #15

Ah, last time I probably looked into Discourse was before that thread which explains it. Been rather busy since to be honest (final year @ university). That looks absolutely brilliant though, props to @lightyear for the work on that, once exams are out of the way I’ll try and get a test site set up, see what our members think.

Re attachments, what size is ‘large’ in your case? We occasionally get the odd attachment, but they’re usually either normal size images, or the odd PDF (manuals etc). Forum is for camera operators, so often need to get info back to them rather quickly.

(Alex Armstrong) #16

Has anyone migrated – or tried to migrate but failed – a Yahoo! Groups group? Is it even possible?

I don’t see a specific import script, but there’s one for mbox.

The group is very old and I want to shut it down and archive it. I’m mostly curious about how to archive it, in case the fancy strikes me to try importing it into Discourse.

(Jeff Atwood) #17

Definitely get it into mbox format. Mail support (incoming and outgoing) and mbox format import is quite mature now.

(Jon Bartlett) #18

Sorry for resurrecting this old post but couldn’t find anything else relating to Yahoo Groups.

I have been looking for a solution that would migrate Yahoo Group data into Discourse but couldn’t find anything that was working in 2017. So I started work on an exporter and importer which would serve that purpose. In hindsight the exporter probably should have gone into mbox format but I have ended up with a dedicated exporter which takes the results of the Yahoo Groups API call and stores in a MongoDB collection. This can be found here GitHub - jonbartlett/yahoo-groups-export: Export Yahoo Groups forum to file or database

On the import side I have just imported 14 years of data from a private Yahoo Group into Discourse using the importer contained into this pull request FEATURE: Yahoo Groups forum importer by jonbartlett · Pull Request #5293 · discourse/discourse · GitHub