Database Cleanup while writing a new Import Script

Dev
1

Hello Friends,

I am trying to write an import script from OSQA to discourse. I have local Ubuntu installation of discourse. I wanted a way to know whether there is some cleanup script available which can delete all the data from all the tables of the discourse? Currently, when I run the script, the next time the entries already inserted are skipped.

If such a cleanup script was available, then I can experiment locally frequently, and it will help me essentially in writing such a script. If such a script is not available, then I guess my course is to just create stored function in PSQL.

Please guide me regarding this. I would be very thankful for this.

2

I’m executing the following commands whenever I want to rebuild my development instance.

cd ~/Repositories/discourse

rm -R public/upload/*
rm -R tmp/*
rm -R log/*

redis-cli flushall
bundle exec rake db:drop db:create db:migrate
4 Likes
3

Thanks a lot :slight_smile: It worked for me.

4

I always take a backup of an empty Discourse install with some tweaks made (admin user, allow_restore enabled, permalink settings, some adjusted limits to allow for a good import) and restore it when I want to start over again. Flushing redis is a good idea as well.

5

That’s what I usually do as well. :wink:
I have scripts for that too.

backup

#!/bin/bash

BACKUP_NAME=${1:-development}

mkdir -p /data/backups/discourse
pg_dump -d discourse_development -F c -b -v -f /data/backups/discourse/${BACKUP_NAME}.backup

restore

#!/bin/bash

BACKUP_NAME=${1:-development}

cd /data/backups/discourse

cd ~/Repositories/discourse
rm -R public/upload/*
rm -R public/backups/default/*
rm -R tmp/*
rm -R log/*

# close all connections because I usually forget to disconnect something...
psql -d postgres -c "SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'discourse_development' AND pid <> pg_backend_pid();"

redis-cli flushall

bundle install
bundle exec rake db:drop db:create

pg_restore -d discourse_development -v /data/backups/discourse/${BACKUP_NAME}.backup
bundle exec rake db:migrate
2 Likes
6

thanks!
I got these error in docker:

root@infdis-app:/var/www/discourse# bundle exec rake db:drop db:create db:migrate DISABLE_DATABASE_ENVIRONMENT_CHECK=1
===========================================
 DEPRECATION: The cocaine gem is deprecated. Please upgrade to terrapin. See https://github.com/thoughtbot/terrapin/ for further instructions.
===========================================
PG::InsufficientPrivilege: ERROR:  must be owner of database discourse
: DROP DATABASE IF EXISTS "discourse"
Couldn't drop database 'discourse'
rake aborted!

but after I su - discourse, I got another error:

discourse@infdis-app:/var/www/discourse$ bundle exec rake db:drop db:create db:migrate DISABLE_DATABASE_ENVIRONMENT_CHECK=1
===========================================
 DEPRECATION: The cocaine gem is deprecated. Please upgrade to terrapin. See https://github.com/thoughtbot/terrapin/ for further instructions.
===========================================
rake aborted!
Errno::EACCES: Permission denied @ rb_sysopen - /var/www/discourse/tmp/ember-rails/ember.js
7

You shouldn’t use my scripts from above inside a Docker container. They are meant for a Ubuntu development environment.

1 Like
8

It’s a bit hard for me to build a development environment .
Is there any way I can go pass these error in docker?

9

Praveen were you able to create the migration script from osqa to discouse? If so would you share it with the wider community? Thanks