Adding command line tools support for user api keys

#1

I can give you my use case to help a bit with next steps! I’ve added a discourse endpoint to the “helpme” client --> HelpMe | HelpMe - a command line tool for helping you out. and Journal of Open Source Software: HelpMe Command Line Helper Utility so that users can ask for help from the command line with any of the following:

helpme discourse
helpme discourse neurostars.org
helpme discourse neurostars.org braincategory

And specifically, they can post a topic, record an asciinema (terminal recording), capture the environment, and of course include text! Actually you can see a recording and full details in this pull request --> Add Discourse Helper by vsoch · Pull Request #33 · vsoch/helpme · GitHub. So - what this comes down to is the user needing an api token to create a topic. They would export it once to their environment:

export HELPME_DISCOURSE_TOKEN=xxxxx

and then it would save to their personal configuration in $HOME. There is also an “admin” configuration that could provide a token to a user, but 1. it’s more likely the user has installed it on his/her own, and 2. The user could discover the token via looking into the running code. So ideally, I want:

  1. Given I am a user of a discourse board, and user tokens are enabled, I have a web page I can go to and it will generate a token for me.
  2. I can then plug it into the client to make a post!

The issue with using some “All Users” or admin token is that we can’t ship that with the software or even risk that it gets in the hands of anyone other than an admin. So the ideal is either already living with discourse, or on a separate server that has it too. Given the second, I’m not sure how we would connect the interface to a particular discourse board (and somewhat delegate the giving out of tokens) so I think the first is most logical. I’ve never made a plugin before, nor have I coded in ruby, so I would need much pointers on these things.

1 Like
User API keys specification
#2

I see, I am open to a PR here that makes your use case somewhat easier:

helpme discourse somesite.com
You have no authentication token to somesite.com to create one visit:
http://somesite.com/api_key/new?application_name=helpme&public_key=ABCD&nonce=NONCE&client_id=client_id&auth_redirect...

Then user goes to web browser pastes in URL and at the end of the process INSTEAD of being redirected off site with the encrypted auth token we would simply render the encrypted token on screen.

It means this controller needs to accept another param:

And we need another view defined to show the encrypted API key that helpme would decrypt.

2 Likes
#3

Ah, fantastic! Let me start to look over the development docs and I’ll see what I can do. The example usage is spot on, even better than the old school “log in and generate” that I had in mind!

2 Likes
#4

Do you have any tips/ examples/ templates that could help me? I have no idea what I’m doing, the most I can tell is that I might add @headless = params[:headless] but then I don’t actually know what that’s doing… :confused: I think I would want to:

  • add the parameter headless, with option to be empty (default) if not provided by the user (not headless mode)
  • based on the API key, use the API secret to encrypt it? (Where is that?)
  • if headless is true, print to the screen
#5

heyo! My awesome collaborator is helping me out, so hopefully we can put our heads together and have a PR, maybe sometime over the weekend? I’ll ping you here if we run into trouble! His main language is Ruby, and he’s awesome, so I think for now we’re good! (and having fun too!)

---- edited!

hey @sam if we have a custom discourse repo, what is the quickest way to get a development server running? I’m looking at the docker-compose (and associated Docker images) from the main discourse image, and it’s quite a hairball. I don’t see any clear or easy way to combine the discourse base, the minidb, and minidb extras, and all the other dependencies that are stacked into there. What would you suggest?

4 Likes
#6

Easiest way to get started is

5 Likes
#7

hey @sam! This is fine if I just want to run the released discourse, but if I want to run a (development) version, from my Github repository, well I’ve been parsing through the dockerfiles this morning and it’s a nightmare - everything is a custom tool or command that has so many layers to reverse engineer I’m pulling my hair out. Ideally, we need an image (Dockerfile) that takes a discourse repository as an argument, and then generates the equivalent discourse:latest but with the custom respository. I can start with this docker-compose.yml

version: '2'
services:
  postgresql:
    image: 'bitnami/postgresql:9.6'
    volumes:
      - 'postgresql_data:/bitnami'
  redis:
    image: 'bitnami/redis:latest'
    environment:
      - ALLOW_EMPTY_PASSWORD=yes
    volumes:
      - 'redis_data:/bitnami'
  discourse:
    image: 'bitnami/discourse:latest'
    labels:
      kompose.service.type: nodeport
    ports:
      - '80:3000'
    depends_on:
      - postgresql
      - redis
    volumes:
      - 'discourse_data:/bitnami'
    environment:
      - POSTGRESQL_HOST=postgresql
      - POSTGRESQL_ROOT_USER=postgres
      - POSTGRESQL_CLIENT_CREATE_DATABASE_NAME=bitnami_application
      - POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME=bn_discourse
      - POSTGRESQL_CLIENT_CREATE_DATABASE_PASSWORD=bitnami1
      - DISCOURSE_POSTGRESQL_NAME=bitnami_application
      - DISCOURSE_POSTGRESQL_USERNAME=bn_discourse
      - DISCOURSE_POSTGRESQL_PASSWORD=bitnami1
  sidekiq:
    image: 'bitnami/discourse:latest'
    depends_on:
      - discourse
    volumes:
      - 'sidekiq_data:/bitnami'
    command: 'nami start --foreground discourse-sidekiq'
    environment:
      - DISCOURSE_POSTGRESQL_NAME=bitnami_application
      - DISCOURSE_POSTGRESQL_USERNAME=bn_discourse
      - DISCOURSE_POSTGRESQL_PASSWORD=bitnami1
      - DISCOURSE_HOST=discourse
      - DISCOURSE_PORT=3000
volumes:
  postgresql_data:
    driver: local
  redis_data:
    driver: local
  discourse_data:
    driver: local
  sidekiq_data:
    driver: local

And I’d ideally just want to replace the name of the image with one that I’ve built. That’s all I’ve been done (for 4 hours now this morning) and I’m not making any progress because it’s so complicated. Is there no simple Dockerfile to do this? It might seem easy to you because you wrote all this / have developed it too for years, so the most I can do is tell you how challenging this is for (someone with pretty good container and development experience) to unravel. I’m very frustrated that this is so complicated. How can we do better?

As an example, when I develop some Django application, and let’s say the repository is downloaded to /code, I can just map a volume at $PWD:/code and then a restart to the container (via docker-compose) will update the app with my changes. Here in order to get to the source code addition I have to go all the way back to the base container:

And download the entire set of files that go into there, but then the container that uses it (discourse_dev) needs a whole set up of yaml configurations (also from the repository) h

which seem to be set up by various ruby scripts and there is no clear list of instructions for “how to deploy this with your own discourse.” If I try a different strategy to reproduce the discourse:latest (that has two base images of minideb and minideb-extras I again am hit with a custom install script from a packaged version, downloaded from a server that isn’t Github

# Install required system packages and dependencies
RUN install_packages advancecomp ghostscript gifsicle hostname imagemagick jhead jpegoptim libbsd0 libc6 libcomerr2 libcurl3 libedit2 libffi6 libgcc1 libgcrypt20 libgmp-dev libgmp10 libgnutls30 libgpg-error0 libgssapi-krb5-2 libhogweed4 libicu57 libidn11 libidn2-0 libjpeg-progs libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap-2.4-2 liblzma5 libncurses5 libnettle6 libnghttp2-14 libp11-kit0 libpq5 libpsl5 libreadline7 librtmp1 libsasl2-2 libssh2-1 libssl1.0.2 libssl1.1 libstdc++6 libtasn1-6 libtinfo5 libunistring0 libxml2 libxml2-dev libxslt1-dev libxslt1.1 optipng pngcrush pngquant zlib1g zlib1g-dev
RUN bitnami-pkg install ruby-2.4.5-0 --checksum 6b8fe1a5db54cc642125e96caf239329d27b2871cd0830a4158c312668a2afc7
RUN bitnami-pkg unpack postgresql-client-9.6.11-0 --checksum f1825273d8f7a0c88fecf6e3f91e9bc940ce1f8b69add7dfa14aca51d15209aa
RUN bitnami-pkg install git-2.19.2-0 --checksum 24817a90223cfc91ce38cdc459dc6647dcf36754bf0300433ddb00ec276757ff
RUN bitnami-pkg unpack discourse-sidekiq-2.1.4-0 --checksum 52ca8aabdf383a6b3958f7e85015e3c438bf6a5b431c510ded9642ac3532247a
RUN bitnami-pkg unpack discourse-2.1.3-0 --checksum 714bb0e0a0f47a01fb5a61c1cdf71af5c6779abbfd5045f3d603e4a4c7539f9f

yeah… :frowning:

#8

You do not want to run your own version of Discourse from your own github repo. I promise. For many of the reasons that you have pointed out. You want to develop a plugin. Maybe you should see Beginners Guide to Install Discourse for Development using Docker and Beginner's Guide to Creating Discourse Plugins - Part 1

See also Can Discourse ship frequent Docker images that do not need to be bootstrapped?. Bitnami is out of date and not supported here.

3 Likes
#9

haha yes, that describes all my frustrations quite well.

So - an update to the controller script that @sam pointed out here:

You are saying this should be done by way of a plugin? And I can’t update the code there, and then test it locally? is there no logical way? As a developer, and given the discourse/discourse is the core codebase, I would suggest that there needs to be an avenue to do this.

And given yes to the above - how can the software stack be reproducible?

#10

The only case that it makes sense to change the codebase rather than make changes in a plugin is if your change will be accepted in core. Otherwise, you’ll have to merge Discourse changes into your own, everyone who has done so was very sorry and spent lots of time and money undoing those changes.

You want to make changes in a plugin.

#11

Correct - that is my use case. I am testing code that will be a PR to core, and I’d like to have a development server to test before doing a PR. I could PR blindly but that is not something I have ever done, it seems like bad practice.

1 Like
#12

Then follow any of the development #howto documents.

1 Like
#13

Please read the topic closely, that system is out there for hacking on you locally checked out source tree, strongly recommend not doing this docker compose bitnami hack

#14

okay, I didn’t realize the bitnami code was a hack! :fearful:

@sam I have a few more questions on the API call. How does the client_key fit in to this? Do you mean that helpme is registered as an application somewhere, and then given a key? And how is this distributed to the user?

I’m trying to map this to my understanding of OAuth2. With OAuth2 the provider would register the application (with a client id and secret) and the user doesn’t need to create a key. For example, to get a Github OAuth2 for Singularity Hub I:

  • register the application in Github
  • am given a client secret and key
  • add the secret and key to Singularity Hub
  • Singularity hub uses the key/secret to create tokens (token and refresh) on the user’s behalf

In our case, we are only dealing with one provider - discourse - and so there is no “Github” to register the application to. But maybe something like this?

  • register the helpme client on a discourse board (this is like the board giving approval for helpme to be used)
  • the discourse board turns on the endpoint to accept generation of tokens (and generated a client_key? where does it go / how is it used?)
  • the user goes through the authentication flow (accepting in browser) to get a token
  • the token is valid for some scope to post questions, etc.

I’m not sure where a public and private key fit in here, because it’s not typical to ask every user to do that, and actually would probably be too much and deter people from using it. The first validation is by way of the user accepting in the web interface, and the following come from the API token used after that. Could we talk about the generation and then use of the various keys/ secrets, and how you see this working?

#15

Hi,

I’m working on this a bit as well.

What’s I’ve got so far is:

  • Discourse admin configures “Allowed user api auth redirects” to their own host ( i.e. https://your-discourse.org/user-api-key ).
  • An additional show action in the user_api_key controller shows the payload ( which the user will use to copy-paste into an environment variable)

Sounds like there’s a complication with the requirement for an RSA key pair. Since this isn’t a bundled mobile app, users would have to generate their own keys to get the token. Would it be possible to make this key pair optional if we are just redirecting back to the discourse app instead of a 3rd party? Likelihood of a man in the middle seems low, but I’m definitely unfamiliar with the requirements of this.

1 Like
#16

Keep in mind nothing is bundled, you hand the server your public key, it then encrypts your token using it. I know it may feel a bit overkill but I do not want to amend the protocol here. So when you “show” the api key to the user actually show the encrypted api key. Your CLI can then decrypt it cause it holds the private key.

1 Like
#17

hey everyone! A quick question. We have a development discourse set up, and I’m testing a call to request a (terminal given) token. So my question - what is a nonce? It looks like a unique id of some kind, but I’m wondering how / when / where it’s generated (so I can do it on behalf of my user, etc.) Thanks!

#18

A nonce can be anything, a random string. You use is to protect against replay attack.

When we give you your encrypted payload you check the nonce matches the one you expect, if it does not well, someone is replaying an old encrypted token. Clearly for your use this level of security is … a bit … overkill…

2 Likes
#19

Cool thanks! I’ll generate a uuid.uuid4() string. I think @fitz and I are probably close - he’s in a different time zone and we will touch bases soon, but hopefully we will have something working and for review! And if we have more questions, thank you again in advance for being so quick to help.

2 Likes
#20

Ok, looping back to this…

I’ve put something together here : Make auth_redirect param options on user_api_keys · cfitz/discourse@f5c631d · GitHub

Makes the auth_redirect param optional, so that the payload will just be returned after the UserApiKey is created.

Does this look alright?
Should I add something to allow admins to disable this feature? Should I add some additional text to be displayed in the new.html.erb?

thanks!