Using the Discourse API to post with uploaded files?


(Peter N Lewis) #1

Continuing the discussion from Fill in post for user?:

Is it possible to use the Discourse API (ideally with curl) to upload files (and get the Discourse URL for the uploaded file) (best), or to post a message that includes one or more files (ok, but less ideal)?


(Kane York) #2

The way that the client does it is to first upload the file, then include the URL in the post.


(Peter N Lewis) #3

Right, but is there a (documented?) way for me to upload the file via the API?


(Mittineague) #4

Did you try rawurlencoding the “body” as shown in the topic you linked from?

Could be tricky not doing via a script. eg.

This is a test post 

<img src="/uploads/default/24704/911e2267120ae22f.png" width="579" height="64">

This%20is%20a%20test%20post%20%0D%0A%0D%0A%3Cimg%20src%3D%22%2Fuploads%2Fdefault%2F24704%2F911e2267120ae22f.png%22%20width%3D%22579%22%20height%3D%2264%22%3E

* of course it should be one line without the forum’s wordwrapping


(Peter N Lewis) #5

I’ve done raw URL encoding before, but that is generally limited to the length of a URL, which is normally limited to around 4k, which would be insufficient for a large image or document file.

My current thinking is that I would be able to upload the file(s) using the API, and then get the paths that Discourse has given them, and then create a post using the Compose a new pre-filled topic via URL technique to reference those already uploaded images.

But I need to have at least some way to upload the files or the whole purpose is defeated.

I could upload them somewhere else and store them somewhere else and reference them, but that would mean I’d lose the automatic “clean up unreferenced files” facility that Discourse offers.


(Régis Hanol) #6

You can definitely use the API to upload a file by POSTing your file to /uploads. The only caveat, is that if you want the url back, you will have to listen to the “/uploads/*” channels via the Message Bus.


Uploading files (async) and get url back?
(Peter N Lewis) #7

Hmm, I seem to be going deeper and deeper down a rabbit hole (not helped by having no clue what the Message Bus is or how to use it from curl).

So, since you say I can upload a file using the API, I presume there is some sort of curl command that I could feed the site API key to, with a POST command and upload a file. OK, that’s good (better if I knew how to do it exactly, but a start).

But if that is not going to immediately tell me what the uploaded file URL is (in the response for example), how do I find out using Message Bus (and if it is some sort of queue of messages, how would I know that the uploaded file matched the one I just uploaded?)


(Daniel Brief) #8

FWIW, here’s how we upload an image (before getting stuck trying to use the Message Bus):

POST to mydiscoursesite.com/uploads.json?
with POST data of:

"file": <a file>
"type": "avatar"  (since we're using this for an avatar, not sure the right value for you)
"user_id" : <user's id>

This should return “Success: OK” and then apparently you need to make a request to /message-bus/930ea9c4b86243dca8e375084f954848/poll (I think any 32-char hex string is OK?) with the right data so that you can get the id of the uploaded image. I haven’t yet figured out how to tell /poll which user I want info about. In the browser it looks like it knows from the user’s session, but it seems silly to make the server log in as a specific user. Also, IIUC, Message Bus might return without info if the upload hasn’t finished, so you may have to make the request again.

If you figure more of this out please reply.


(Marcin Rataj) #9

Perhaps Discourse could return path to uploaded file along with “Success: OK” response?
For example, via additional HTTP header:
X-Upload-Path: /uploads/default/24704/911e2267120ae22f.png

It feels like an easy change to implement and it would eliminate the need of looking at the message bus when uploading via API call.

@zogstrip do you think it is feasible?


(Peter N Lewis) #10

It would be nice to include the uploaded path, but I think the issue is that it is all deferred so the link is not known at upload time.

That said, I’m not sure its really a good idea for it to be deferred since even in the UI I find the experience of uploading a file somewhat unpleasant, as you go through the upload process and then at the end nothing happens, and then sometime later an uploaded file magically is added to your message. The experience would be better (IMHO) if the upload was not deferred and the UI spun until it was finished - its not like you can really continue on with your message while your waiting for the system to insert a random link.


(Peter N Lewis) #11

OK, nearly there, but one remaining thing stumps me.

You can upload a file with:

curl -X POST -F type=composer -F client_id=x0f3b591bb4848dd899b6e6ee0feaff9 -F 'files[]=@FileName.kmmacros' 'http://forum.keyboardmaestro.com/uploads.json?api_key=APIKEY&api_username=peternlewis'

The client id you can make up some UUID, the file name is as specified, and the APIKEY is your APIKEY form the forum, and the api_username is your username (its not requires, but it might be required if you are using the site APIKEY, I’m not sure).

The web site returns:

{"success":"OK"}

Then you can get the URL with:

curl -X POST -d '/uploads/composer=61' 'http://forum.keyboardmaestro.com/message-bus/x0f3b591bb4848dd899b6e6ee0feaff9/poll?api_key=APIKEY&api_username=peternlewis'

The “61” is the message id you have seen and goes up one for each message that you receive. But I’ve got no idea how to figure out the correct message ID to use, or how to specify “just give me the latest message” or whatever.

The web site returns:

[{"global_id":85522,"message_id":62,"channel":"/uploads/composer","data":{"id":1276,"user_id":1,"original_filename":"FileName.kmmacros","filesize":6371,"width":null,"height":null,"url":"/uploads/default/original/2X/6/63f94e337e2e3e336ba59a118086d442f07e6d47.kmmacros","created_at":"2015-06-12 06:16:04 UTC","updated_at":"2015-06-12 06:16:04 UTC","sha1":"63f94e337e2e3e336ba59a118086d442f07e6d47","origin":null,"retain_hours":null}}]

Aha, but if you use a unique client_id each time, then you can just use =0 for the message id and you’ll get the answer!

So that should word - I have no idea how far off I am from using the system properly, but that at least works I believe. Off to write more code to see if I can put it all together.


How to get uploaded image url using the Discourse API?
(Daniel Brief) #12

I can’t check right now but I think that if you pass -1 then it will return the next value


(Peter N Lewis) #13

In my tests, if I passed 1, it would give me the oldest matching message bus entry. So if I had uploaded multiple entries with the same client_id, I would not get the last one, I would get the first one (though that ID was something like 53 at the time, not 1, so I’m not quite sure what it is counting, maybe all messages since restart? I had restarted relatively recently).


(Daniel Brief) #14

Did you mean 1 or -1 ? I meant -1


(Peter N Lewis) #15

Ahh, -1. Maybe. I didn’t try that. That would be helpful.

Ahh, ok, interesting, -1 returns the next value, as you say. But note: it does not return the last entry, it returns an entirely differently formatted result with the last value:

curl -X POST -d '/uploads/composer=-1' 'http://forum.keyboardmaestro.com/message-bus/1234b591bb4848dd899b6e6ee0feaff9/poll?api_key=APIKEY&api_username=peternlewis'

returns:

[{"global_id":-1,"message_id":-1,"channel":"/__status","data":{"/uploads/composer":75}}]

75 is the last message ID on the /uploads/composer channel.

But unfortunately it is the last message ID on the channel for any client_id. Not for this specific client ID (it may be for this user, I’m not sure). If I upload (via the web for example) another file, it will show 76 even though the client_id on the web upload is different.

So it seems the best bet is: choose a new UUID, use that as the client_id, upload the file, and the use the poll with the same client_id and 0 message_id.


(Kane York) #16

No, with a 0 “current position” (message id) it will try to catch you up on everything that’s happened. The only reason it’s not doing that is because the messages are scoped to a particular client ID. You want to do this:

  • Pick a UUID
  • Send a -1 poll to get the current position (“join the message bus”)
  • Upload the file, giving the client ID
  • Send polls on the message bus with the same client ID and current position from earlier until the upload completes (until 1 message is delivered)

In other words, you get to implement the message bus protocol from Bash. Hey, at least it’s not websockets :wink:

And if you’re going this far, may as well toss in cookie auth (instead of api_key) as well. strike that - you’re doing this on behalf of a user


(Régis Hanol) #17

Yeah I reckon it’s not a spectacular user experience. I’ll fix our API upload story next week.


(Peter N Lewis) #18

Will this change the API at all? If the API for the upload could (relatively synchronously) return the URL for the upload, that would be a great improvement in simplifying what I am trying to do.

In any event, I’m going to write a middleware server under my control with my own API for my application to talk to, and then it will talk API to the Discourse server, so I should be able to track any API changes easily enough, and my own API will likely behave exactly like that, responding with the URL for the uploaded file. It would be nice to have a better impedance match to the Discourse API though!


(Peter N Lewis) #19

If you use a different client ID for each upload, then there will only ever be one message for the client ID, so wont that simplify the process? Why bother with the “Send a -1 poll to get the current position (“join the message bus”)” (which requires yet another round trip to the server) when just using 0 will work fine to find the one and only poll response?


(Régis Hanol) #20

The only difference there will be is that when using the API with an API key, uploading something will be synchronous instead of asynchronous.


Cannot upload a file through the API with the synchronous parameter