How sam's development environment is configured

I wanted to do a post (that I try to keep up to date) with notes about my current dev environment.

(note, this post is a work in progress, as stuff changes I will update it)

My building blocks

  • Windows 10 creators update
  • Wsltty which I prefer built in bash for windows console
  • VMWare workstation with latest Ubuntu installed
  • Mobaxterm cause its nice to have an X server
  • Vim plus my vim setup installed inside the VM
  • rbenv plus a bunch of rubies for my dev
  • 7+ taskbar tweaker cause Windows makes it annoying to ungroup icons
  • i3wm - I use it almost exclusively on my 3 monitor setup

Hacks

A desktop shortcut to my linux VM

I use this shortcut to access my VM - target:

%LOCALAPPDATA%\wsltty\bin\mintty.exe --wsl -o Padding=5 -o Locale=C -o Charset=UTF-8 /bin/wslbridge -C~ -t /bin/bash -c "DISPLAY=0.0.0.0:0.0 ssh -X ubuntu"

It adds a bit of :sparkler: in that I have nice padding on my terminal and connect my mobax windows manager to the session so I can run stuff like git gui if I want.

A shortcut to puma

I use a custom shortcut to launch puma:

%LOCALAPPDATA%\wsltty\bin\mintty.exe --wsl -o Padding=5 -o Locale=C -o Charset=UTF-8 /bin/wslbridge -C~ -t /bin/bash -c "ssh ubuntu '(cd /home/sam/Source/discourse && echo `pwd` && . ~/.zshrc && cd ~/Source/discourse && bin/puma --kill-existing)'

This shortcut is pinned to my taskbar and allows me to easily launch puma when needed and easily find it.

Using Chrome directly in Windows to access dev

I prefer running Chrome in Windows to running it in X Windows, I prefer the rendering Windows provides and it just works a bit faster in this setup.

I have it configured so ubuntu always resolves to my VM ip, I handle that by providing VMWare a static lease for the VM per this article.

It basically means I edit:

C:\ProgramData\VMware\vmnetdhcp.conf

And add something like this to the conf

host ubuntu {
    hardware ethernet 00:0C:29:16:2A:D6;
    fixed-address 192.168.84.132;
}

Once that is done and my c:\Windows\System32\Drivers\etc\hosts file is edited to reflect, I can access my dev env using ubuntu instead of shuffling IPs. The IP is stable on reboot so this just works once setup.

Seamless SSH from WSL to VM

I generated an RSA key in my WSL instance and use ssh-copy-id to stage the public key to the Linux VM.

This means I do not enter passwords when I ssh into the VM using the above shortcuts, this is critical to this workflow.

Better errors support

Better errors is the awesome gem that makes debugging in dev so much easier. Unfortunately, out of the box, it is super strict about who is allowed access.

I use a :hammer: to allow my Windows machine access to http://ubuntu:9292/__better_errors by adding this to my .zshrc:

export TRUSTED_IP:"0.0.0.0/0"

Feel free to use less of a sledgehammer, but completely losing out on better errors would be a huge shame.

Vim autospec

I use the Vim autospec configuration documented here: discourse/notify_file_change at master 路 discourse/discourse 路 GitHub

This means that as soon as I save a spec, autospec runs it using the power of :tophat:

20 Likes

Thanks, @sam. Does anyone run a native Ubuntu dev environment? (Or maybe it鈥檚 so easy that it鈥檚 not worth documenting.)

@techAPJ does and @tgxworld but it is far less fiddly.

2 Likes

I pretty much still follow this guide:

:slight_smile:

For headless installation of Discourse dev-env on Ubuntu, I recommend using our official Docker dev image: discourse/bin/docker at master 路 discourse/discourse 路 GitHub

5 Likes

Oh! That鈥檚 gold. I鈥檇 been rolling my own. Now I鈥檓 glad I asked. :slight_smile: That might deserve its own topic.

5 Likes

I notice that your dev environment has the content of try. Is that something mere mortals can get, for starting out with a pre-populated test space?

2 Likes

I would love to have a test database out there that anyone could download, unfortunately try would need a bit of scrubbing (emails / ips) if you feel like taking on such a project let me know :wink:

8 Likes

If @elijah does not want to take on the project, I鈥檇 be happy to once I finish my current project.

5 Likes

Yeah, I think I could do that.

5 Likes

Any progress on this?

I鈥檝e got code that reads the database dump and creates a new dump.sql with almost entirely new content, and generates a whole set new placeholder images.

But the restore didn鈥檛 like my dump.sql and I need to track down where I screwed it up.

4 Likes

The first error in the log (/var/lib/dockers/containers/idhere/idhere.json) is:

{"log":"2017-06-18 02:16:36.036 UTC [2492] discourse@discourse ERROR:  relation \"api_keys\" already exists\r\n","stream":"stdout","time":"2017-06-18T02:16:37.497495095Z"}
{"log":"2017-06-18 02:16:36.036 UTC [2492] discourse@discourse STATEMENT:  CREATE TABLE api_keys (\r\n","stream":"stdout","time":"2017-06-18T02:16:37.510775323Z"}

In the restore progress window in the browser everything leading up to that looks like:

[2017-06-18 02:16:33] 'eli' has started the restore!
[2017-06-18 02:16:33] Marking restore as running...
[2017-06-18 02:16:33] Making sure /var/www/discourse/tmp/restores/default/2017-06-18-021633 exists...
[2017-06-18 02:16:33] Copying archive to tmp directory...
[2017-06-18 02:16:33] Unzipping archive, this may take a while...
[2017-06-18 02:16:35] Extracting metadata file...
[2017-06-18 02:16:35] Validating metadata...
[2017-06-18 02:16:35]   Current version: 20170605014820
[2017-06-18 02:16:35]   Restored version: 20170307181800
[2017-06-18 02:16:35] Extracting dump file...
[2017-06-18 02:16:35] Restoring dump file... (can be quite long)
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] ERROR:  relation "api_keys" already exists

Which is making me think, not a syntax error. That鈥檚 the first CREATE TABLE statement in the file. I understand that during restores there鈥檚 supposed to be some CREATE SCHEMA / RENAME SCHEMA magic that goes on (eg Manually restoring a Discourse backup for development ), but I don鈥檛 see that in the Docker log, the less verbose on screen restore log, or inside the dump.sql file.

I thought that might be the problem, so I manually put in

DROP SCHEMA IF EXISTS restore CASCADE;
CREATE SCHEMA restore;

Without luck.

Is this process sound? It鈥檚 how I set things up:

mkdir /var/discourse /var/hold
git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
./discourse-setup
# errored out because of mail config,
vi container/app.yml

tar cf - . | gzip > /var/hold/fresh-install.tgz

And then restore them between runs:

cd /var/discourse
./launcher stop app
rm -rf ./* /var/lib/dockers/containers/*
tar xzf /var/hold/fresh-install.tgz
./launcher rebuild app
mkdir -p  /var/discourse/shared/standalone/backups/default
cp /var/hold/backup.tgz /var/discourse/shared/standalone/backups/default/try-begriffin-2017-03-09-145156-v20170307181800.tar.gz

Then I have to go into the web browser and create my admin account, enable restores, and try to restore the backup.

But that should reset everything past initial docker install, right?

This looks backwards to me, are you restoring older over newer?

1 Like

Yes. Is taking a backup from an old system and putting it on a newer one not supported?

No that is not supported.

1 Like

Uh no, that鈥檚 migrating forward. It鈥檚 taking a backup from current and restoring it onto old that鈥檚 bad.

5 Likes

Is this the recommended setup for windows?

Also, are you able to debug in the VM? Set breakpoints and such?

Recommended is a funny word :slight_smile: This is the setup I still use and still love. I have a Win 10 desktop with 3 4k monitors and while coding I dedicate 2 to my VM.

I find Vim + i3wm to be a killer excellent combo. Debugging wise I am a puts debugger. Combined with bin/rake autospec which works on plugins and has Vim integration where I auto focus on the running spec, I have a lethal and highly effective combo.

I would not say my setup is for everyone though, but 鈥 if you go down this path a lot of gold awaits at the end of the rainbow. :rainbow:

8 Likes