Plugin idea: Image optimization with WebP

feature
1

Inspired by this:

When pictures are uploaded or locally downloaded, Discourse generates a WebP version of that image which it’ll use if it detects a compatible browser.

  • Should probably only use WebP for thumbnails. Expanded images should always be the source file.
  • Maybe the best approach would be a lightweight “standalone service” the way Discourse’s avatar generator works.
2 Likes
2

I think converting animated GIFs to MP4s is way more practically useful than this in the near term.

The average animated GIF would be a lot smaller than 66% of the size when converted to MP4… a whoooole lot smaller.

Untitled.jpg1243×530 159 KB 

4,751,045 AlertSpicyBlueandgoldmackaw.gif
  292,891 AlertSpicyBlueandgoldmackaw.mp4

That’s 16.2 times smaller…

8 Likes
3

Hi @codinghorror and @erlend_sh ,
I am a GSoC '16 aspirant. I found the idea of converting GIFs to MP4s very interesting and am interested in helping out implementing the same.

I have referred to the links mentioned on the GSoC ideas’ page on the main code repo and have done a fair bit of google’ing looking for ideas related to gif to mp4 conversion. Please do let me know how I should proceed now. If a mentor is already assigned to this idea, I could get in touch with them to better understand what I am expected to do.

Thank you,
Shreedhar

4

Of the GSoC ideas outlined, this one is probably the most loosely defined. You’ll probably want to submit a proposal for one other idea, as this one is still a bit “up in the air”.

There’s no mentor assigned yet (we don’t even know how many slots we got) so in the meantime the best use of our time here would be to scope out this task.

There are a couple different ways to do this. I wonder if perhaps a stand-alone microservice would be the way to go, like we did in order to optimize avatar rendering.

  1. User submits a .gif image. If “Gif converter” service is enabled then:
  2. Server downloads the gif
  3. Gif is converted to mp4
  4. Original gif link is replaced with local .mp4 file.
2 Likes
5

As a continuation of this small plugin spec:

  1. Discourse persist original_filename which is useful as an indicator of GIF.
  2. It’s better to show a gif badge like Twitter since it’s possible to pause the generated video. It may be confusing if there is no such badge.
  3. While replacing original file may be hard to retrieve the original gif, a linked list like model could be used (origin of video can point to relative path of original gif)
  4. Do some modification preventing original gif being removed by scheduled task
2 Likes
6

In order to get familiarised with the discourse codebase was trying to add conversion of gif to video format and storing it.

Firstly here is what I understood from the code and correct me if I am wrong.

  1. When a file is uploaded, it triggers UploadsController#create
  2. UploadsController makes some optimizations and then call Upload.create_for
  3. Upload model creates a record and sends an object back to UploadsController.
  4. UploadsController sends back the data as json to Ember app.
  5. Ember app checks for link to be an image and wraps it around img tag.

What I thought of doing:-

  1. I chose to use ffmpeg to convert gif to mp4 video and then insert the mp4 video to the uploads record.
  2. I have added a script to convert gif to mp4 video here in upload model.
  3. But there was an error message saying that format was not supported.
  4. I figured out that it was because of this validator check and if I add an exception for mp4 extension everything works, but people are then able to upload mp4 videos.
  5. I could have added the script in OptimizedImage model, but it looks like it is used mainly for storing thumbnails of the images.
    Note: -With the above implementation we will be losing the original gif image.

Could somebody tell me if I have understood the flow correctly and suggest me changes in my implementation?

7

Hello,

This plugin is part of my proposal for this year’s Outreachy / GSoC. I consider that this plugin might improve user’s experience significantly when browsing on a forum with lots of GIFs. Users will no longer have to wait for big GIFs to download. Moreover, the video can be streamed whilst it is downloaded.

Community managers will also see an improvement in their bandwidth usage and used disk space, although they may need slightly more processing power. NOTE: The used disk space will be improved only if the original GIFs will be no longer stored.

The goal is to create a lightweight service that can convert GIFs to MP4s. It would be ideal if it would support multiple backends for converting files from GIF to MP4 (e.g. ffmpeg, Cloudinary)

Conversion flow:

  • User uploads or links a GIF. Depending on site settings, only the user uploads, linked files or both will be converted. In some cases, the plugin can get smart. For example services like Imgur serve both GIFs and MP4s. Consider: https://i.imgur.com/kZyMZOb.gif. Replacing “.gif” with “.mp4” will “convert” the GIF to an MP4 immediately.
  • The GIFs are queued for conversion. A processing queue is required because multiple ffmpeg instances may be a resource hog and make Discourse unresponsive. The conversion service has to carefully manage resources. I believe this is the most difficult part of this project.
  • After the conversion changes, the GIF is replaced with the MP4 video.
4 Likes
8

Great effort, I think GIFs deserve to be MP4s and the benefits are huge!

1 Like