I don’t think you should be using sudo when npm installing stuff. I’ve had some bad experiences doing so.
I think I tried it without sudo first without success and then just threw in the sudo in case it helped, which it didn’t. I don’t even have any .svgs among the files I’m importing. , so I’m not going to try and troubleshoot this. I resolved this by re-installed node/npm following the instructions from the official repo.
It would be nice to figure out how to suppress the error message about svgogifsicle, because there’s other errors that I’m having a hard time spotting.
(I did try bundle exec ruby script/import_scripts/ning.rb --no-svgo, but clearly that option needs to go somewhere else.)
Here are two errors that may be related.
The first:
20 / 257 ( 7.8%) Upload not valid :( /vagrant/ning/discussions/54-HinchliffeAMICAL2.pdf
#<ActiveModel::Errors:0xc8c02b0 @base=#<Upload id: nil, user_id: -1, original_filename: "54-HinchliffeAMICAL2.pdf", filesize: 5,198,132, width: nil, height: nil, url: "", created_at: nil, updated_at: nil, sha1: "be0daf8ed4d4cbd980c43e3710f620500f8b8778", origin: nil, retain_hours: nil>, @messages={:filesize=>["Sorry, the file you are trying to upload is too big (maximum size is 3072KB)."]}>
The is a large file error. There were several large files causing problems, but they disappeared when I increased max image size kb before running the importing script. But this one doesn’t, which is strange. The file is only 5 MB and I’ve set max image size kb to 100000.
The second:
189 / 257 ( 73.5%) Attachment file doesn't exist: /vagrant/ning/FileNotFound
This is a missing file. It’s either the previous large file or something else. If the latter, I’d like to track down the filename, but I can’t figure out how. Any suggestion of how to find which topic is “189 / 257” in the Ning data?
I’m a doofus. There are two settings. One for images (max image size kb) and one for other file types (max attachment size kb). Increasing the size of both settings fixes all the issues with large files.
I still haven’t figured out how to track the missing file. Any ideas?
you guys are amazing, it has been a year already that i’ve been trying to import mine with the help of someone here but to no avail. i’ve left it hanging. i’ve looked at buddypress and even drupal just to stay away from ning.
anyhow, ill just keep track of your progress for me to understand it better.
on another note, @alehandrof would be willing to help me import my stuff once your successful for a little fee? that’s the least i could do to say thanks.
@Rodelio_Lagahit I found the help in this forum – including from members of the core dev team – to be remarkable. But you do need to have some decent sysadmin skills to tackle this stuff in the first place.
I read through lots of forums and docs in order to figure out everything I needed to do.
If there was one thing that I wish I figured out earlier, it’s this: I needed to setup a local Vagrant instance and do the import and configuration there. Then I exported the database and re-imported it in my production Docker.
If you aren’t able to get things going on your own, check with Discourse and [Communiteq (formerly DiscourseHosting)](https://www.communiteq.com). I seem to recall that both can help you with importing/installing for a one-time fee.
4 Likes
We only do migrations for Enterprise customers or Business customers who pre-pay for a year of service.
For assisted migrations, we highly recommend using the #marketplace category. There’s a growing number of very capable freelance developers in our community. Fair compensation for migration work is usually in the $150-$500 range, depending on the complexity of the migration.
3 Likes
This is noted. I’ve tried that before months ago. Anyhow, worth revisiting.
can you share your tutorial on how to get this done? im not really a coder but would be willing to get my hands dirty for this to work. @alehandrof @zogstrip
A user just alerted me that some (all?) of the links to files form our old Ning forum aren’t working. They appear linked but the links are empty:
These were fine at some point, but apparently have stopped working somewhere along the line.
Any ideas?
You mean the attachments were imported into Discourse but went missing?
The example you show mentions a spreadsheet, so maybe it wasn’t an allowed file type. Before starting the import, there’s a setting to configure:
EXTRA_AUTHORIZED_EXTENSIONS = ["bmp", "ico", "txt", "pdf"]
The importer output would mention attachments that it couldn’t import.
3 Likes
My current understanding is that the files were imported into Discourse, but went missing somewhere along the way.
I had asked about extensions way back when and had included all the ones I needed:
EXTRA_AUTHORIZED_EXTENSIONS = ["bin", "bmp", "doc", "docx", "gif", "htm", "ico", "jpeg", "jpg", "json", "pdf", "png", "ppsx", "ppt", "pptx", "txt", "xls", "xlsx", "zip"]
It’s not just spreadsheets. I just checked more thoroughly and every file except for images seems to be missing. 
For good measure, I ran rake posts:rebake as that helped with some images woes I had in the past. But no joy.
The files are included in /var/www/discourse/public/uploads/default/original/1X.
I was able to cross-reference a topic in Discourse with Ning.
This code:
<li>
<a>AMICAL 2014 Members Council Meeting - Minutes</a> (14 May 2014)</li>
Should be linking to a file called:
- Ning:
AMICAL2014MembersCouncilMinutes_distmembers.pdf - Discourse:
0baa9dd232fe29c1b7cb53376b0d2feda905e091.pdf
Does this help me at all?
Hmm, it’s hard to debug someone else’s import work. I read your other topic where you mention that all your upload url’s used “http://localhost:3000”. Here you say you ran rake posts:rebake, which would probably not go well in a production environment where they are all using localhost:3000. After moving from a dev environment (localhost:3000) to a production server with a real domain, you should fix all the links:
RAILS_ENV=production bundle exec ruby script/discourse remap 'localhost:3000' 'forum.mydomain.com'
So my guess is that rebaking all posts with broken links caused them to be lost.
3 Likes
I don’t entirely follow, but I’ll take your word for it. I am puzzled as to why rebaking corrected images, but not other kinds of links.
Is there anything useful I can do in my current situation? (The site is live and has been active with new content since March.)
Hey, @neil. I’ve tried what seems like dozens of times (It might only be 6) to use the Ning Network Archiver to download the data from a forum, but it keeps giving errrors like “couldn’t download x messages” and the resulting JSON files are corrupt. (Not just the stuff that the script fixes, but parts of records being truncated such that parts of values are truncated and fields are missing.)
That was on a Windows machine and a Windows VM running under Linux. Did you download the data? If you did, did you use a Mac? Any other hints about just getting the JSON files downloaded?
FWIW, I did this on a Mac. (And, as you can read above, it wasn’t all puppy dogs and unicorns.)
Sigh. So much for that silver bullet. Did you download Air in August? My next hail Mary will be to see if I can download an older version of Air in hopes that there is some incompatibility between newer versions of Air and the Ning Network Archiver.
My last attempt, this time on a mac, says “26400/26729 files successfully downloaded. 329 files could not be downloaded.” If experience is any indicator, this means that the JSON file is corrupted (perhaps 329 times, but I haven’t gotten that far yet).
I’ve never been the one to get the data from Ning, but your experience sounds typical. I’ve received corrupted json files and had to fix them in the importers as best I could. I’ve also seen files that simply stop and can’t be repaired. Typically people need to contact Ning support to receive files that work and have all the data. I’m not sure if that’s possible anymore. If you can reach Ning support, also make sure they send you all replies in each topic (since they’re truncated at 100 or 200), and get them to send you all the attachments too.
5 Likes
I did the migration in March 2016. My Adobe Air is 22.0.0.153 (how to find it), but I don’t know what it was at the time.
FWIW, I had that issue as well, but my community site was much smaller. I had 3 files that could not be downloaded either. This was not, as far as I could tell, an indicator that the JSON files were corrupted. Well, that’s partly true. The JSON files that Ning creates are [bleep] [bleeping]ing weird. So most things that read JSON files will have nothing to do with them. But they import fine because Neil’s accounted for their weirdness.
I never did find out exactly what those files were but because we had all critical files backed up elsewhere, and they were so few, we decided to not try to figure this out and go ahead with the migration.
I must have broken the links to files later, but given that the site was live at this point, I just stuck with it. I’m pretty sure this is an unrelated issue to the export.
3 Likes