# An Importer for Google Groups

**URL:** <https://meta.discourse.org/t/an-importer-for-google-groups/45624>\
**Category:** Product Support\
**Created:** [13 Giugno 2016, 11:20pm UTC](https://meta.discourse.org/t/an-importer-for-google-groups/45624 "2016-06-13T23:20:33Z")\
**Posts on this page:** 1\
**Showing post:** 8

<div class="post-metadata">

**Author:** ![steinwaywhw](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/steinwaywhw/32/121566_2.png) [@steinwaywhw](https://meta.discourse.org/u/steinwaywhw)\
**Post date:** [14 Giugno 2016, 6:21pm UTC](https://meta.discourse.org/t/an-importer-for-google-groups/45624/8 "2016-06-14T18:21:48Z")

</div>

Alright, I figured it out and I hope this could help others.

I’m using [GitHub - icy/google-group-crawler: [Deprecated] Get (almost) original messages from google group archives. Your data is yours. · GitHub](https://github.com/icy/google-group-crawler) with the latest version (b42f28d4c36eddce1df039566f3615b0753a2dc6) of mbox import script.

## Crawler Part

1. `export _GROUP=your_group_name`
2. `./crawler.sh -sh > wget.sh`
3. `./wget.sh` _note that `chmod +x wget.sh` might be needed_

## Import Part

1. Need to install sqlite3  
a. inside docker, find Gemfile at `/var/www/discourse` and add dependency for sqlite3  
b. `apt-get install -y sqlite3 libsqlite3-dev`  
c. `bundle install --no-deployment` as `root`
2. Need to modify `mbox.rb` in `script/import_scripts`  
a. find `MBOX_DIR` and update it accordingly.  
b. find `#{MBOX_DIR}/messages/*` in `all_messages` and modify the subdir to be `mbox` as this is what the crawler names it.  
c. comment all calls `all_records_exist`, namely, line 210, line 252, and line 317. These lines causing troubles connecting to sqlite3 database. Not sure if you will see the same problem. You can play with these.
3. `su discourse`
4. `RAILS_ENV=production ruby ./mbox.rb`

Refresh your Discourse webpage in the browser, and good to go.

Actually, I hope someone with experience could tell me what’s going on with that “all\_records\_exist?” error. That function is from the `base.rb` file, and the exception basically says the connection object is `null`.

## Update (July 2016)

Hi, I just discussed with the author of google\_group\_crawler and he confirmed that now the email addresses seen by the crawler is not full address, but only a portion of it. Probably for security reasons from Google side. Therefore, I extended the importer script as shown here on [glot.io](http://glot.io): [Google Group Importer for Discourse - Ruby Snippet - glot.io](https://glot.io/snippets/efyndm7qs5), which will load a separate users list csv file (can be exported from Google group as a group owner/manager), and run a cross scan to recover emails. Hope it helps.

---

_[View the full topic](https://meta.discourse.org/t/an-importer-for-google-groups/45624)._