How sam's development environment is configured


(Sam Saffron) #1

I wanted to do a post (that I try to keep up to date) with notes about my current dev environment.

(note, this post is a work in progress, as stuff changes I will update it)

My building blocks

  • Windows 10 creators update
  • Wsltty which I prefer built in bash for windows console
  • VMWare workstation with latest Ubuntu installed
  • Mobaxterm cause its nice to have an X server
  • Vim plus my vim setup installed inside the VM
  • rbenv plus a bunch of rubies for my dev
  • 7+ taskbar tweaker cause Windows makes it annoying to ungroup icons
  • i3wm - I use it almost exclusively on my 3 monitor setup

Hacks

A desktop shortcut to my linux VM

I use this shortcut to access my VM - target:

%LOCALAPPDATA%\wsltty\bin\mintty.exe --wsl -o Padding=5 -o Locale=C -o Charset=UTF-8 /bin/wslbridge -C~ -t /bin/bash -c "DISPLAY=0.0.0.0:0.0 ssh -X ubuntu"

It adds a bit of :sparkler: in that I have nice padding on my terminal and connect my mobax windows manager to the session so I can run stuff like git gui if I want.

A shortcut to puma

I use a custom shortcut to launch puma:

%LOCALAPPDATA%\wsltty\bin\mintty.exe --wsl -o Padding=5 -o Locale=C -o Charset=UTF-8 /bin/wslbridge -C~ -t /bin/bash -c "ssh ubuntu '(cd /home/sam/Source/discourse && echo `pwd` && . ~/.zshrc && cd ~/Source/discourse && bin/puma --kill-existing)'

This shortcut is pinned to my taskbar and allows me to easily launch puma when needed and easily find it.

Using Chrome directly in Windows to access dev

I prefer running Chrome in Windows to running it in X Windows, I prefer the rendering Windows provides and it just works a bit faster in this setup.

I have it configured so ubuntu always resolves to my VM ip, I handle that by providing VMWare a static lease for the VM per this article.

It basically means I edit:

C:\ProgramData\VMware\vmnetdhcp.conf

And add something like this to the conf

host ubuntu {
    hardware ethernet 00:0C:29:16:2A:D6;
    fixed-address 192.168.84.132;
}

Once that is done and my c:\Windows\System32\Drivers\etc\hosts file is edited to reflect, I can access my dev env using ubuntu instead of shuffling IPs. The IP is stable on reboot so this just works once setup.

Seamless SSH from WSL to VM

I generated an RSA key in my WSL instance and use ssh-copy-id to stage the public key to the Linux VM.

This means I do not enter passwords when I ssh into the VM using the above shortcuts, this is critical to this workflow.

Better errors support

Better errors is the awesome gem that makes debugging in dev so much easier. Unfortunately, out of the box, it is super strict about who is allowed access.

I use a :hammer: to allow my Windows machine access to http://ubuntu:9292/__better_errors by adding this to my .zshrc:

export TRUSTED_IP:"0.0.0.0/0"

Feel free to use less of a sledgehammer, but completely losing out on better errors would be a huge shame.

Vim autospec

I use the Vim autospec configuration documented here: discourse/notify_file_change at master · discourse/discourse · GitHub

This means that as soon as I save a spec, autospec runs it using the power of :tophat:


Is VMware Workstation Player enough for Discourse-only development, or the more expensive VMware Workstation Pro provides some real advantages?
(Jay Pfaffman) #2

Thanks, @sam. Does anyone run a native Ubuntu dev environment? (Or maybe it’s so easy that it’s not worth documenting.)


(Sam Saffron) #3

@techAPJ does and @tgxworld but it is far less fiddly.


(Arpit Jalan) #4

I pretty much still follow this guide:

:slight_smile:

For headless installation of Discourse dev-env on Ubuntu, I recommend using our official Docker dev image: discourse/bin/docker at master · discourse/discourse · GitHub


(Jay Pfaffman) #5

Oh! That’s gold. I’d been rolling my own. Now I’m glad I asked. :slight_smile: That might deserve its own topic.


(Eli the Bearded) #6

I notice that your dev environment has the content of try. Is that something mere mortals can get, for starting out with a pre-populated test space?


(Sam Saffron) #7

I would love to have a test database out there that anyone could download, unfortunately try would need a bit of scrubbing (emails / ips) if you feel like taking on such a project let me know :wink:


Is there any fake data to be used to test a plugin?
(Joshua Rosenfeld) #8

If @elijah does not want to take on the project, I’d be happy to once I finish my current project.


(Eli the Bearded) #9

Yeah, I think I could do that.


(Jay Pfaffman) #10

Any progress on this?


(Eli the Bearded) #11

I’ve got code that reads the database dump and creates a new dump.sql with almost entirely new content, and generates a whole set new placeholder images.

But the restore didn’t like my dump.sql and I need to track down where I screwed it up.


(Eli the Bearded) #12

The first error in the log (/var/lib/dockers/containers/idhere/idhere.json) is:

{"log":"2017-06-18 02:16:36.036 UTC [2492] discourse@discourse ERROR:  relation \"api_keys\" already exists\r\n","stream":"stdout","time":"2017-06-18T02:16:37.497495095Z"}
{"log":"2017-06-18 02:16:36.036 UTC [2492] discourse@discourse STATEMENT:  CREATE TABLE api_keys (\r\n","stream":"stdout","time":"2017-06-18T02:16:37.510775323Z"}

In the restore progress window in the browser everything leading up to that looks like:

[2017-06-18 02:16:33] 'eli' has started the restore!
[2017-06-18 02:16:33] Marking restore as running...
[2017-06-18 02:16:33] Making sure /var/www/discourse/tmp/restores/default/2017-06-18-021633 exists...
[2017-06-18 02:16:33] Copying archive to tmp directory...
[2017-06-18 02:16:33] Unzipping archive, this may take a while...
[2017-06-18 02:16:35] Extracting metadata file...
[2017-06-18 02:16:35] Validating metadata...
[2017-06-18 02:16:35]   Current version: 20170605014820
[2017-06-18 02:16:35]   Restored version: 20170307181800
[2017-06-18 02:16:35] Extracting dump file...
[2017-06-18 02:16:35] Restoring dump file... (can be quite long)
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] SET
[2017-06-18 02:16:36] ERROR:  relation "api_keys" already exists

Which is making me think, not a syntax error. That’s the first CREATE TABLE statement in the file. I understand that during restores there’s supposed to be some CREATE SCHEMA / RENAME SCHEMA magic that goes on (eg Manually restoring a Discourse backup for development ), but I don’t see that in the Docker log, the less verbose on screen restore log, or inside the dump.sql file.

I thought that might be the problem, so I manually put in

DROP SCHEMA IF EXISTS restore CASCADE;
CREATE SCHEMA restore;

Without luck.

Is this process sound? It’s how I set things up:

mkdir /var/discourse /var/hold
git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
./discourse-setup
# errored out because of mail config,
vi container/app.yml

tar cf - . | gzip > /var/hold/fresh-install.tgz

And then restore them between runs:

cd /var/discourse
./launcher stop app
rm -rf ./* /var/lib/dockers/containers/*
tar xzf /var/hold/fresh-install.tgz
./launcher rebuild app
mkdir -p  /var/discourse/shared/standalone/backups/default
cp /var/hold/backup.tgz /var/discourse/shared/standalone/backups/default/try-begriffin-2017-03-09-145156-v20170307181800.tar.gz

Then I have to go into the web browser and create my admin account, enable restores, and try to restore the backup.

But that should reset everything past initial docker install, right?


(Jeff Atwood) #13

This looks backwards to me, are you restoring older over newer?


(Eli the Bearded) #14

Yes. Is taking a backup from an old system and putting it on a newer one not supported?


(Jeff Atwood) #15

No that is not supported.


(Kane York) #16

Uh no, that’s migrating forward. It’s taking a backup from current and restoring it onto old that’s bad.