# Database Cleanup while writing a new Import Script

**URL:** https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134
**Category:** Development
**Created:** [August 7, 2018, 4:59pm UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134 "2018-08-07T16:59:17Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![praveendhinwa](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/praveendhinwa/32/108964_2.png) [@praveendhinwa](https://meta.discourse.org/u/praveendhinwa)
#### Post date: [August 7, 2018, 4:59pm UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/1 "2018-08-07T16:59:17Z")

</div>

Hello Friends,

I am trying to write an import script from OSQA to discourse. I have local Ubuntu installation of discourse. I wanted a way to know whether there is some cleanup script available which can delete all the data from all the tables of the discourse? Currently, when I run the script, the next time the entries already inserted are skipped.

If such a cleanup script was available, then I can experiment locally frequently, and it will help me essentially in writing such a script. If such a script is not available, then I guess my course is to just create stored function in PSQL.

Please guide me regarding this. I would be very thankful for this.

---

<div class="post-metadata">

### Author: ![gerhard](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/gerhard/32/119479_2.png) [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [August 7, 2018, 6:02pm UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/2 "2018-08-07T18:02:46Z")

</div>

I’m executing the following commands whenever I want to rebuild my development instance.

```plaintext
cd ~/Repositories/discourse

rm -R public/upload/*
rm -R tmp/*
rm -R log/*

redis-cli flushall
bundle exec rake db:drop db:create db:migrate

```

---

<div class="post-metadata">

### Author: ![praveendhinwa](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/praveendhinwa/32/108964_2.png) [@praveendhinwa](https://meta.discourse.org/u/praveendhinwa)
#### Post date: [August 7, 2018, 6:39pm UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/3 "2018-08-07T18:39:31Z")

</div>

Thanks a lot 🙂 It worked for me.

---

<div class="post-metadata">

### Author: ![RGJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/rgj/32/523185_2.png) [@RGJ](https://meta.discourse.org/u/RGJ)
#### Post date: [August 7, 2018, 7:11pm UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/4 "2018-08-07T19:11:40Z")

</div>

I always take a backup of an empty Discourse install with some tweaks made (admin user, allow\_restore enabled, permalink settings, some adjusted limits to allow for a good import) and restore it when I want to start over again. Flushing redis is a good idea as well.

---

<div class="post-metadata">

### Author: ![gerhard](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/gerhard/32/119479_2.png) [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [August 7, 2018, 7:40pm UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/5 "2018-08-07T19:40:37Z")

</div>

That’s what I usually do as well. 😉  
I have scripts for that too.

**backup**

```bash
#!/bin/bash

BACKUP_NAME=${1:-development}

mkdir -p /data/backups/discourse
pg_dump -d discourse_development -F c -b -v -f /data/backups/discourse/${BACKUP_NAME}.backup

```

**restore**

```bash
#!/bin/bash

BACKUP_NAME=${1:-development}

cd /data/backups/discourse

cd ~/Repositories/discourse
rm -R public/upload/*
rm -R public/backups/default/*
rm -R tmp/*
rm -R log/*

# close all connections because I usually forget to disconnect something...
psql -d postgres -c "SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'discourse_development' AND pid <> pg_backend_pid();"

redis-cli flushall

bundle install
bundle exec rake db:drop db:create

pg_restore -d discourse_development -v /data/backups/discourse/${BACKUP_NAME}.backup
bundle exec rake db:migrate

```

---

<div class="post-metadata">

### Author: ![freemangl](https://avatars.discourse-cdn.com/v4/letter/f/8c91f0/32.png) [@freemangl](https://meta.discourse.org/u/freemangl)
#### Post date: [October 23, 2018, 5:04am UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/6 "2018-10-23T05:04:18Z")

</div>

thanks!  
I got these error in docker:

> ```
> root@infdis-app:/var/www/discourse# bundle exec rake db:drop db:create db:migrate DISABLE_DATABASE_ENVIRONMENT_CHECK=1
> ===========================================
> DEPRECATION: The cocaine gem is deprecated. Please upgrade to terrapin. See https://github.com/thoughtbot/terrapin/ for further instructions.
> ===========================================
> PG::InsufficientPrivilege: ERROR: must be owner of database discourse
> : DROP DATABASE IF EXISTS "discourse"
> Couldn't drop database 'discourse'
> rake aborted!
> 
> ```

but after I su - discourse, I got another error:

> ```
> discourse@infdis-app:/var/www/discourse$ bundle exec rake db:drop db:create db:migrate DISABLE_DATABASE_ENVIRONMENT_CHECK=1
> ===========================================
> DEPRECATION: The cocaine gem is deprecated. Please upgrade to terrapin. See https://github.com/thoughtbot/terrapin/ for further instructions.
> ===========================================
> rake aborted!
> Errno::EACCES: Permission denied @ rb_sysopen - /var/www/discourse/tmp/ember-rails/ember.js
> 
> ```

---

<div class="post-metadata">

### Author: ![gerhard](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/gerhard/32/119479_2.png) [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [October 23, 2018, 11:25am UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/7 "2018-10-23T11:25:24Z")

</div>

You shouldn’t use my scripts from above inside a Docker container. They are meant for a **Ubuntu development environment**.

---

<div class="post-metadata">

### Author: ![freemangl](https://avatars.discourse-cdn.com/v4/letter/f/8c91f0/32.png) [@freemangl](https://meta.discourse.org/u/freemangl)
#### Post date: [October 23, 2018, 11:27am UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/8 "2018-10-23T11:27:24Z")

</div>

It’s a bit hard for me to build a **development environment**.  
Is there any way I can go pass these error in docker?

---

<div class="post-metadata">

### Author: ![muji786](https://avatars.discourse-cdn.com/v4/letter/m/e19adc/32.png) [@muji786](https://meta.discourse.org/u/muji786)
#### Post date: [December 10, 2018, 9:50pm UTC](https://meta.discourse.org/t/database-cleanup-while-writing-a-new-import-script/94134/9 "2018-12-10T21:50:33Z")

</div>

Praveen were you able to create the migration script from osqa to discouse? If so would you share it with the wider community? Thanks
