AI Translation: What happened to "Translatable Categories" and how are translation costs calculated?

[Sara_Carmona_y_Lladó]

Hi team,

I’m trying to understand the current behavior of the AI Translation feature and whether there has been a configuration change or migration in recent versions.

When we initially configured the AI Translation feature, the settings page contained a field called “Translatable categories” with the description:

Only content in the selected categories will be translated. Subcategories must be added separately.

We used that configuration to define a limited translation scope.

However, when reviewing the current configuration, we now see a field called “Excluded categories” instead.

This raises a few questions:

Configuration changes

Has the AI Translation feature changed from a “Translatable categories” model to an “Excluded categories” model?

If so:

• Was there an automatic migration?
• How were the existing category selections converted?
• Is there any way to determine whether the current configuration was migrated automatically or modified manually?

Translation scope

The Translation Progress page currently displays:

Backfill settings are configured to translate all posts after March 2018.

How should this message be interpreted?

Does it refer to:

• all posts in the forum,
• all posts except excluded categories,
• or another translation scope?

Cost calculation

We would also like to better understand how costs are generated for the AI Translation feature.

Specifically:

• When a post is translated during backfill, is the translated content stored and reused?
• Or is a new translation request generated each time a user accesses translated content?
• What contributes to the cost:
  • backfill operations,
  • newly created content,
  • users viewing translated content,
  • or a combination of these?

Reporting

Is there a way to identify:

• which categories are currently being translated,
• how many posts have been translated per category,
• and which categories are generating the largest translation costs?

We have screenshots showing both the historical “Translatable categories” configuration and the current “Excluded categories” configuration and can provide them if needed.

Thanks for your help.

[Moin]

@Sara_Carmona_y_Lladó

Configuration changes

Has the AI Translation feature changed from a “Translatable categories” model to an “Excluded categories” model?

If so:

• Was there an automatic migration?
• How were the existing category selections converted?
• Is there any way to determine whether the current configuration was migrated automatically or modified manually?

Yes there was a migration

@pmusaraj

Following that change, the ability to limit topic translation to specific categories is still present. But unlike the initial implementation, now Discourse defaults to translating all content and an ai_translation_excluded_categories site setting is available to do just that, exclude categories from translation.

Existing communities should have been migrated automatically to this system. Let us know if you run into any issues.

The migration took a list of all categories on the forum and removed the ones configured in the translateable categories setting. The remaining categories were saved in the new excluded categories setting, preserving the existing translation behavior.

You should be able to see manual changes on the setting in your staff action logs. You can filter those for changes on a specific setting (The url is something like https://forum.example.com/admin/logs/staff_action_logs?filters=%7B%22subject%22%3A%22ai_translation_excluded_categories%22%2C%22action_name%22%3A%22change_site_setting%22%7D)

Post translations are stored in the database, so the same content is translated only once.

You can use data explorer to get more data on the translations on your forum.
This query returns all categories on your forum except for the ones configured to be excluded from ai translation:

SELECT c.id as category_id, c.name
FROM categories c
WHERE c.id NOT IN (
  SELECT unnest(string_to_array(value, '|')::integer[])
  FROM site_settings
  WHERE name = 'ai_translation_excluded_categories'
)
ORDER BY c.id

[Moin]

You can also use data explorer for your other questions

Is that about what you had in mind?

SELECT
    c.id AS category_id,
    COUNT(DISTINCT pl.post_id) AS translated_posts,
    COUNT(DISTINCT pl.locale) AS languages,
    COUNT(*) AS total_translations
FROM post_localizations pl
JOIN posts p ON p.id = pl.post_id
JOIN topics t ON t.id = p.topic_id
JOIN categories c ON c.id = t.category_id
GROUP BY c.id, c.name
ORDER BY total_translations DESC