Making post images more accessible with Discourse AI

[nat (Natalie T)]

In 2024 (link), we introduced AI image captions in Discourse AI. It was useful especially for accessibility and relied on people remembering to click “Caption with AI” when writing a post.

We’ve now reworked image captioning so captions can be generated in the background for post images, stored separately from the post raw (the actual content that the author wrote stays as-is), and show up in where they’re useful: for screen readers, search, and image lightboxes.

With this version, image captions can now be automatically generated in the background after a post is created. Here’s a quick look

How it works

AI image captions are now attached to post images without inserting generated text into the post itself. This keeps captions available for accessibility and search, without adding extra visible text into the body of every post.

When captions are available, Discourse will:

• add the AI caption to post images for screen readers
• show the caption in the image lightbox
• include the caption text in search indexing
• let users edit captions from the composer preview when editing a post
• support locale-specific captions for localized content

Some screenshots in case videos are not your thing:

Visible in lightbox

Editable in composer

Searchable

Setup Where?

This feature lives under the new Image Caption AI feature area in Discourse AI, moved from the previous AI Helper AI feature area into its dedicated area.

To use it, you’ll need:

• discourse_ai_enabled
• ai_post_image_captions_enabled
• a working ai_image_caption_agent

The selected image caption agent needs image uploads enabled, and must use a vision-capable LLM. If the feature is enabled but the agent is not ready, Discourse will show an admin problem so it’s easier to spot configuration issues.

Existing posts

There is also an optional backfill job for existing post images.

Backfill is off by default, so communities can choose whether they want to generate captions for older posts, and at what rate. This can be configured with:

• ai_post_image_captions_backfill_hourly_rate
• ai_post_image_captions_backfill_max_age_days

Try it out

For communities hosted by Discourse our models are already image capable. If you’d like to make post images more useful for people using screen readers, enable the feature and backfill (see Existing posts above)

As mentioned above, for self-hosted communities, do make sure your image caption agent is using a vision-capable model.

[loginerror (Maciej Kuźmicz)]

Will those captions be crawled by search engines?

[nat (Natalie T)]

Great question -

Somewhat.

The captions are rendered into the post HTML as aria-description on the image, so search engine crawlers can see them when crawling the topic page. They are also included in Discourse’s own search index.

They are not inserted as visible post text or alt text, so whether a search engine indexes or weights the aria-description text is up to the crawler.