I would like to work on a plan to add a feature into Discourse that allows themes to store user data (up to 1000 bytes per theme per user of structured data), this data should have a delightful async API and also the ability to store items that are “pre-loaded” (up to say 300 bytes per theme per user) do some research in Discourse and start a design document that shows all the files that need changing

Thinking...

This is an interesting feature request from Sam (who I believe is one of the Discourse founders). He wants to add a feature that allows themes to store user-specific data with:

1. Up to 1000 bytes per theme per user of structured data
2. A delightful async API
3. Ability to store items that are “pre-loaded” (up to 300 bytes per theme per user)

I need to research the Discourse codebase to understand:

1. How themes work currently
2. How user data is stored
3. Where theme-related APIs are defined
4. How preloading works in Discourse
5. Similar existing features that might guide the implementation

Let me start by searching for relevant files and code in the discourse/discourse repository.

I’ll help you research the Discourse codebase and create a design document for this theme user data storage feature. Let me start by exploring the relevant parts of the codebase.

Thinking...

GitHub search files
Searched for ‘theme, model’ in discourse/discourse/main

GitHub search files
Searched for ‘theme, serializer’ in discourse/discourse/main

GitHub search files
Searched for ‘theme, controller’ in discourse/discourse/main

GitHub search files
Searched for ‘user_option’ in discourse/discourse/main

GitHub file content
Retrieved content of app/models/theme.rb, app/models/theme_field.rb, app/models/theme_setting.rb, app/models/user_option.rb from discourse/discourse@main

GitHub code search
Searched for ‘preloaded_json store preload’ in discourse/discourse

GitHub search files
Searched for ‘application_layout_preloader’ in discourse/discourse/main

GitHub file content
Retrieved content of lib/application_layout_preloader.rb, app/controllers/application_controller.rb#L1-L100, app/serializers/current_user_serializer.rb from discourse/discourse@main

GitHub code search
Searched for ‘PluginStore theme’ in discourse/discourse

GitHub search files
Searched for ‘plugin_store’ in discourse/discourse/main

GitHub file content
Retrieved content of app/models/plugin_store.rb, app/models/plugin_store_row.rb from discourse/discourse@main

continue

Thinking...

The user wants me to continue the research and create a design document for adding a feature to Discourse that allows themes to store user data. Let me continue researching relevant parts of the codebase.

I need to look at:

1. How theme JavaScript is currently loaded and executed
2. The serializer patterns used for themes
3. The controller endpoints for themes
4. Database migration patterns
5. API patterns for CRUD operations

Let me search for more relevant files.

GitHub search files
Searched for ‘theme_serializer’ in discourse/discourse/main

GitHub file content
Retrieved content of app/serializers/theme_serializer.rb, app/controllers/admin/themes_controller.rb#L1-L150 from discourse/discourse@main

GitHub code search
Searched for ‘class API user theme’ in discourse/discourse

GitHub search files
Searched for ‘user_custom_field’ in discourse/discourse/main

GitHub file content
Retrieved content of app/models/user_custom_field.rb, db/migrate/20140421235646_add_user_custom_fields.rb from discourse/discourse@main

GitHub code search
Searched for ‘ajax theme store API’ in discourse/discourse

GitHub search files
Searched for ‘user_data, controller’ in discourse/discourse/main

---

Design Document: Theme User Data Storage Feature

Based on my research of the Discourse codebase, here’s a comprehensive design document for adding user data storage capabilities to themes.

Overview

Add the ability for themes to store structured data per user (up to 1000 bytes) with support for pre-loaded data (up to 300 bytes). This will enable themes to maintain user preferences, state, and other user-specific data.

Database Changes

New Model: ThemeUserData

File: app/models/theme_user_data.rb (new)

# frozen_string_literal: true

class ThemeUserData < ActiveRecord::Base
  MAX_VALUE_SIZE = 1000
  MAX_PRELOADED_SIZE = 300

  belongs_to :theme
  belongs_to :user

  validates :theme_id, presence: true
  validates :user_id, presence: true
  validates :key, presence: true, length: { maximum: 255 }
  validates :value, length: { maximum: MAX_VALUE_SIZE }
  validate :validate_json_structure
  validate :validate_preloaded_size

  def validate_json_structure
    return if value.blank?
    
    begin
      parsed = JSON.parse(value)
      errors.add(:value, "must be a valid JSON object or array") unless parsed.is_a?(Hash) || parsed.is_a?(Array)
    rescue JSON::ParserError
      errors.add(:value, "must be valid JSON")
    end
  end

  def validate_preloaded_size
    if preload && value.present? && value.bytesize > MAX_PRELOADED_SIZE
      errors.add(:value, "exceeds maximum preloaded size of #{MAX_PRELOADED_SIZE} bytes")
    end
  end

  def parsed_value
    return nil if value.blank?
    JSON.parse(value)
  rescue JSON::ParserError
    nil
  end
end

Migration

File: db/migrate/YYYYMMDDHHMMSS_create_theme_user_data.rb (new)

# frozen_string_literal: true

class CreateThemeUserData < ActiveRecord::Migration[7.1]
  def change
    create_table :theme_user_data do |t|
      t.integer :theme_id, null: false
      t.integer :user_id, null: false
      t.string :key, limit: 255, null: false
      t.text :value
      t.boolean :preload, default: false, null: false
      t.timestamps
    end

    add_index :theme_user_data, [:theme_id, :user_id, :key], unique: true, name: 'idx_theme_user_data_unique'
    add_index :theme_user_data, [:user_id, :theme_id], name: 'idx_theme_user_data_user_theme'
    add_index :theme_user_data, [:theme_id, :preload], name: 'idx_theme_user_data_preload', where: 'preload = true'
  end
end

Model Updates

Theme Model

File: app/models/theme.rb

Add association:

has_many :theme_user_data, dependent: :destroy

User Model

File: app/models/user.rb

Add association:

has_many :theme_user_data, dependent: :destroy

Controller

New Controller: ThemeUserDataController

File: app/controllers/theme_user_data_controller.rb (new)

# frozen_string_literal: true

class ThemeUserDataController < ApplicationController
  requires_login
  
  before_action :ensure_theme_exists
  before_action :validate_data_size, only: [:upsert]

  def index
    theme_data = ThemeUserData
      .where(theme_id: params[:theme_id], user_id: current_user.id)
      .pluck(:key, :value, :preload)
      .map { |key, value, preload| { key: key, value: JSON.parse(value), preload: preload } }

    render json: { data: theme_data }
  end

  def show
    data = ThemeUserData.find_by(
      theme_id: params[:theme_id],
      user_id: current_user.id,
      key: params[:key]
    )

    if data
      render json: { key: data.key, value: data.parsed_value, preload: data.preload }
    else
      render json: { error: "Not found" }, status: :not_found
    end
  end

  def upsert
    data = ThemeUserData.find_or_initialize_by(
      theme_id: params[:theme_id],
      user_id: current_user.id,
      key: params[:key]
    )

    data.value = params[:value].to_json
    data.preload = params[:preload] || false

    if data.save
      render json: { key: data.key, value: data.parsed_value, preload: data.preload }
    else
      render json: { errors: data.errors.full_messages }, status: :unprocessable_entity
    end
  end

  def destroy
    data = ThemeUserData.find_by(
      theme_id: params[:theme_id],
      user_id: current_user.id,
      key: params[:key]
    )

    if data
      data.destroy
      render json: { success: true }
    else
      render json: { error: "Not found" }, status: :not_found
    end
  end

  private

  def ensure_theme_exists
    unless Theme.exists?(params[:theme_id])
      render json: { error: "Theme not found" }, status: :not_found
    end
  end

  def validate_data_size
    value_size = params[:value].to_json.bytesize
    max_size = params[:preload] ? ThemeUserData::MAX_PRELOADED_SIZE : ThemeUserData::MAX_VALUE_SIZE

    if value_size > max_size
      render json: { error: "Data exceeds maximum size of #{max_size} bytes" }, status: :unprocessable_entity
    end
  end
end

Routes

File: config/routes.rb

resources :theme_user_data, only: [] do
  collection do
    get ':theme_id' => 'theme_user_data#index'
    get ':theme_id/:key' => 'theme_user_data#show'
    put ':theme_id/:key' => 'theme_user_data#upsert'
    delete ':theme_id/:key' => 'theme_user_data#destroy'
  end
end

Serializer

CurrentUserSerializer Update

File: app/serializers/current_user_serializer.rb

Add attribute:

attributes :theme_user_data_preloaded

Add method:

def theme_user_data_preloaded
  return {} unless @options[:theme_id].present?
  
  ThemeUserData
    .where(theme_id: @options[:theme_id], user_id: object.id, preload: true)
    .pluck(:key, :value)
    .to_h
    .transform_values { |v| JSON.parse(v) }
end

def include_theme_user_data_preloaded?
  @options[:theme_id].present?
end

ApplicationLayoutPreloader Update

File: lib/application_layout_preloader.rb

Update preload_current_user_data method to pass theme_id:

def preload_current_user_data
  @preloaded["currentUser"] = MultiJson.dump(
    CurrentUserSerializer.new(
      @guardian.user,
      scope: @guardian,
      root: false,
      login_method: @login_method,
      theme_id: @theme_id, # Add this
    ),
  )
  # ... rest of method
end

Frontend JavaScript API

Plugin API Addition

File: app/assets/javascripts/discourse/app/lib/plugin-api.js

/**
 * Theme User Data Storage API
 * Provides async methods for themes to store and retrieve user-specific data
 * 
 * @param {string} key - The key to store data under
 * @param {Object|Array} value - The data to store (will be JSON serialized)
 * @param {Object} options - Options object
 * @param {boolean} options.preload - Whether to preload this data (max 300 bytes)
 * @returns {Promise}
 */
setThemeUserData(key, value, options = {}) {
  const themeId = this._currentThemeId();
  
  return ajax(`/theme_user_data/${themeId}/${key}`, {
    type: 'PUT',
    data: {
      value: value,
      preload: options.preload || false
    }
  });
}

/**
 * Get theme user data by key
 * 
 * @param {string} key - The key to retrieve
 * @returns {Promise<Object>}
 */
getThemeUserData(key) {
  const themeId = this._currentThemeId();
  
  // Check if data is preloaded
  const preloaded = this.currentUser?.theme_user_data_preloaded?.[key];
  if (preloaded !== undefined) {
    return Promise.resolve(preloaded);
  }
  
  return ajax(`/theme_user_data/${themeId}/${key}`).then(response => response.value);
}

/**
 * Get all theme user data for current theme
 * 
 * @returns {Promise<Object>} Object with keys mapped to values
 */
getAllThemeUserData() {
  const themeId = this._currentThemeId();
  
  return ajax(`/theme_user_data/${themeId}`).then(response => {
    return response.data.reduce((acc, item) => {
      acc[item.key] = item.value;
      return acc;
    }, {});
  });
}

/**
 * Delete theme user data by key
 * 
 * @param {string} key - The key to delete
 * @returns {Promise}
 */
deleteThemeUserData(key) {
  const themeId = this._currentThemeId();
  
  return ajax(`/theme_user_data/${themeId}/${key}`, {
    type: 'DELETE'
  });
}

_currentThemeId() {
  return this.container.lookup('site:main').theme_id;
}

Security Considerations

1. Authentication: All endpoints require login (requires_login)
2. Authorization: Users can only access their own data
3. Size Limits: Enforced at controller and model level
4. JSON Validation: Ensures only valid JSON structures are stored
5. XSS Prevention: Data is JSON encoded/decoded, not rendered as HTML

Performance Considerations

1. Preloaded Data: Small frequently-accessed data can be preloaded in currentUser serializer
2. Indexed Queries: Composite indexes for efficient lookups
3. Lazy Loading: Non-preloaded data fetched via async API calls
4. Caching: Consider adding Redis caching for frequently accessed keys

Migration Path

1. Run migration to create theme_user_data table
2. Deploy backend changes (models, controllers, serializers)
3. Deploy frontend API additions
4. Update theme documentation with usage examples

Example Theme Usage

// In a theme's JavaScript file
export default {
  name: 'my-theme-component',
  
  initialize(api) {
    // Store user preference (preloaded for fast access)
    api.setThemeUserData('darkMode', { enabled: true }, { preload: true });
    
    // Retrieve preloaded data (synchronous from currentUser)
    const darkMode = api.currentUser?.theme_user_data_preloaded?.darkMode;
    
    // Or fetch asynchronously if not preloaded
    api.getThemeUserData('preferences').then(prefs => {
      console.log('User preferences:', prefs);
    });
    
    // Get all data
    api.getAllThemeUserData().then(allData => {
      console.log('All theme data:', allData);
    });
    
    // Delete data
    api.deleteThemeUserData('tempData');
  }
};

Files to Create/Modify Summary

New Files:

1. app/models/theme_user_data.rb
2. app/controllers/theme_user_data_controller.rb
3. db/migrate/YYYYMMDDHHMMSS_create_theme_user_data.rb

Files to Modify:

1. app/models/theme.rb - Add association
2. app/models/user.rb - Add association
3. app/serializers/current_user_serializer.rb - Add preloaded data
4. lib/application_layout_preloader.rb - Pass theme_id to serializer
5. config/routes.rb - Add routes
6. app/assets/javascripts/discourse/app/lib/plugin-api.js - Add theme data methods

Test Files to Create:

1. spec/models/theme_user_data_spec.rb
2. spec/controllers/theme_user_data_controller_spec.rb
3. spec/requests/theme_user_data_spec.rb
4. spec/serializers/current_user_serializer_spec.rb (update)

This design provides a clean, RESTful API for themes to store user data with proper security, validation, and performance optimizations.