API の URL 経由アップロードで結果が不安定

みなさん、こんにちは。

この件について、ご知見をお聞かせください。
API を通じてアップロードを実行しようとしていますが、常に以下のエラーが発生します。

{"errors"=>["申し訳ありませんが、アップロードするファイルを提供する必要があります。"]}

これは、Canvas から生成された公開 URL をアップロードしようとした際に発生します。例えば:

https://instructure-uploads-apse2.s3.ap-southeast-2.amazonaws.com/account_24310000000000001/attachments/777/IT%20Helpdesk%20Support%20Duty%20Statement%205%20Nov%202015.pdf?response-content-disposition=inline%3B%20filename%3D%22IT%20Helpdesk%20Support%20Duty%20Statement%205%20Nov%202015.pdf%22%3B%20filename%2A%3DUTF-8%27%27IT%2520Helpdesk%2520Support%2520Duty%2520Statement%25205%2520Nov%25202015.pdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZAVS3WMRR54DT4Q%2F20180328%2Fap-southeast-2%2Fs3%2Faws4_request&X-Amz-Date=20180328T032334Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=d4407ec41af0ffbf0bff523dba1322a18b726939242383876ed92ab1798977ee

一方、別の URL を使用すると、例えば:

http://unec.edu.az/application/uploads/2014/12/pdf-sample.pdf

正常に動作するようです。

ブラウザでその URL にアクセスし、PDF ファイルを表示することは確実に可能です。

何が間違っているのでしょうか?

よろしくお願いいたします。
マイケル

It looks like that url isn’t browsable and has some permission locking on it. Your browser can access it because you are logged in to aws, but your Discourse instance cannot access it because it has not been authorized.

The URL has an expiry set on it, that is probably why you can’t see it now. It is generated on the fly when using the Canvas API

It is difficult to show you but here is another URL that has just been generated. I can access it from incognito browser too.

https://instructure-uploads-apse2.s3.ap-southeast-2.amazonaws.com/account_24310000000000001/attachments/777/IT%20Helpdesk%20Support%20Duty%20Statement%205%20Nov%202015.pdf?response-content-disposition=inline%3B%20filename%3D%22IT%20Helpdesk%20Support%20Duty%20Statement%205%20Nov%202015.pdf%22%3B%20filename%2A%3DUTF-8%27%27IT%2520Helpdesk%2520Support%2520Duty%2520Statement%25205%2520Nov%25202015.pdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZAVS3WMRR54DT4Q%2F20180328%2Fap-southeast-2%2Fs3%2Faws4_request&X-Amz-Date=20180328T201440Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=a476c1046e6b356c3bee618247957aea3c1dce3f0f442391130515a030fb8016

I guess I am just trying to find out why these kinds of files are not able to be uploaded via Discourse API and others are.

Something is still up with that url. Chrome is just being fancy and loading it right away for you (handling the headers or something).

Once you are able to get it to work with curl -O <url> it should work in Discourse.

When using curl -O <url> It kept giving errors: Warning: Failed to create the file
But I can get curl -0 <url> -o filename.pdf to work.
Can discourse use other curl command lines? Or can I somehow get it to perform the same download as curl -0 <url> -o filename

Thanks

I’m unable to get curl -0 <url> -o filename.pdf to work. Can you verify that the filename.pdf is the actual pdf document and not just some xml error.

Also -0 just changes it to “Use HTTP 1.0 (H)” where as -O saves it as a file instead of trying to render the pdf in the console.

The Discourse app doesn’t actually use curl to download the file, but it is just a good way to test anonymous downloads of files and when the discourse app uses ruby to download the file it does do something similar to curl.

In order to determine what the issue is here can you provide some more info on how your s3 bucket file permissions are setup? How do you generate all the extra url parameters to allow “anonymous” access to a private file?

Another option might be to use curl or whatever your using to access the discourse api to download the file or use the aws-sdk to download the file as an authenticated user and then upload the actual file to the discourse api instead of just a url.

Also once you get this all working you might want to turn on this setting in Discourse “prevent anons from downloading files”.

Thanks for the tip.
I ended up downloading the file to local storage and then uploading to Discourse.
A bit hacky but works well.

Thanks