Discourse AI - Embeddings

documentation admins
1

This topic covers the configuration of the Embeddings module of the Discourse AI plugin.

Feature set

The Embeddings modules can automatically generate embeddings of every new post in your Discourse instance. Those are used by our semantic features, like semantic suggested topics and semantic topic search.

When configured, this module will add a “Related Topics” section to the bottom of all topic pages, where topics who are similar to the current page will be linked, helping users find related discussions to what they are currently reading on:

image
image1023×475 49.9 KB

It also adds a Semantic Search option on the full page search:

image
image983×354 63.9 KB

Providers

This module can use two providers:

  • Open Source: a collection of open source models from SBERT. Recommended and default.

  • OpenAI: Uses OpenAI API to generate embeddings. You will need a working OpenAI API key for this.

Installation

After installing the plugin and enabling this module, you will need to use the following commands to backfill the embeddings for the existing content of your forum:

rake ai:embeddings:create_table
rake ai:embeddings:backfill
rake ai:embeddings:index #optional

Settings

  • ai_embeddings_enabled: Enables or disables the module

  • ai_embeddings_discourse_service_api_endpoint: URL where the API is running for the module. If you are using CDCK hosting this is automatically handled for you. If you are self-hosting check the self-hosting guide.

  • ai_embeddings_discourse_service_api_key: API key for the API configured above. If you are using CDCK hosting this is automatically handled for you. If you are self-hosting check the self-hosting guide.

  • ai_embeddings_models: Every model enabled here be used to generate embeddings from user posts.

  • ai_embeddings_semantic_suggested_model: Model that will be used to semantic suggested topics. Model picked here must be enabled in the ai_embeddings_models setting too.

  • ai_embeddings_generate_for_pms: If PMs should also have embeddings generated automatically when created.

  • ai_embeddings_semantic_related_topics_enabled: Shows related topics at the bottom of the topic page. This will add an extra block between the topic last post and the suggested topics with topics who are semantic related to the current topic.

  • ai_embeddings_pg_connection_string: Database connection string for the PostgreSQL instance that will store embeddings. If you are using CDCK hosting this is automatically handled for you. If you are self-hosting check the self-hosting guide.

  • ai_openai_api_key: OpenAI API key. Necessary if you want to use OpenAI to generate embeddings. If so, don’t forget to pick text-embedding-ada-002 in ai_embeddings_models and in ai_embeddings_semantic_suggested_model.

10 Likes
3

Hey again.

After setting up the ankane/pgvector:latest container to store the embeddings, I am getting these errors:

Job exception: PQsocket() can’t get socket descriptor

rack-mini-profiler-3.1.0/lib/patches/db/pg.rb:110:in `exec'  
rack-mini-profiler-3.1.0/lib/patches/db/pg.rb:110:in `async_exec'  
mini_sql-1.4.0/lib/mini_sql/postgres/connection.rb:209:in `run'  
mini_sql-1.4.0/lib/mini_sql/postgres/connection.rb:181:in `exec'  
/var/www/discourse/plugins/discourse-ai/lib/modules/embeddings/topic.rb:74:in `persist_embedding'  
/var/www/discourse/plugins/discourse-ai/lib/modules/embeddings/topic.rb:15:in `block in generate_and_store_embeddings_for'  
/var/www/discourse/plugins/discourse-ai/lib/modules/embeddings/topic.rb:13:in `each'  
/var/www/discourse/plugins/discourse-ai/lib/modules/embeddings/topic.rb:13:in `generate_and_store_embeddings_for'  
/var/www/discourse/plugins/discourse-ai/lib/modules/embeddings/jobs/regular/generate_embeddings.rb:14:in `execute'  
/var/www/discourse/app/jobs/base.rb:249:in `block (2 levels) in perform'  
rails_multisite-4.0.1/lib/rails_multisite/connection_management.rb:80:in `with_connection'
/var/www/discourse/app/jobs/base.rb:236:in `block in perform'  
/var/www/discourse/app/jobs/base.rb:232:in `each'  
/var/www/discourse/app/jobs/base.rb:232:in `perform'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:202:in `execute_job'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'  
sidekiq-6.5.8/lib/sidekiq/middleware/chain.rb:177:in `block in invoke'  
/var/www/discourse/lib/sidekiq/pausable.rb:134:in `call'  
sidekiq-6.5.8/lib/sidekiq/middleware/chain.rb:179:in `block in invoke'  
sidekiq-6.5.8/lib/sidekiq/middleware/chain.rb:182:in `invoke'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:169:in `block in process'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'  
sidekiq-6.5.8/lib/sidekiq/job_retry.rb:113:in `local'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'  
sidekiq-6.5.8/lib/sidekiq.rb:44:in `block in <module:Sidekiq>'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:263:in `stats'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'  
sidekiq-6.5.8/lib/sidekiq/job_logger.rb:13:in `call'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'  
sidekiq-6.5.8/lib/sidekiq/job_retry.rb:80:in `global'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:124:in `block in dispatch'  
sidekiq-6.5.8/lib/sidekiq/job_logger.rb:39:in `prepare'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:123:in `dispatch'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:168:in `process'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:78:in `process_one'  
sidekiq-6.5.8/lib/sidekiq/processor.rb:68:in `run'  
sidekiq-6.5.8/lib/sidekiq/component.rb:8:in `watchdog'  
sidekiq-6.5.8/lib/sidekiq/component.rb:17:in `block in safe_thread'

Creating the tables and backfilling worked fine.

Google says something about connection pool size - how can I increase it?

4

After switching over to a RDS Postgres instance, everything seems smooth now.

Used code:

import { Aspects, NestedStack, NestedStackProps } from "aws-cdk-lib";
import { InstanceType, SecurityGroup, SubnetConfiguration, Vpc } from "aws-cdk-lib/aws-ec2";
import {
    DatabaseInstance, DatabaseInstanceEngine, Credentials
} from "aws-cdk-lib/aws-rds";
import { Construct } from 'constructs';
import { PostgresEngineVersion } from "aws-cdk-lib/aws-rds";
import { ParameterGroup } from "aws-cdk-lib/aws-rds";
import { StorageType } from "aws-cdk-lib/aws-rds";
interface RdsPostgreSqlNestedStackProps extends NestedStackProps {
    readonly vpc: Vpc;
    readonly securityGroup: SecurityGroup;
    readonly subnet: SubnetConfiguration;
}

export class RdsPostgreSqlNestedStack extends NestedStack {
    databaseInstance: DatabaseInstance;

    constructor(scope: Construct, id: string, props: RdsPostgreSqlNestedStackProps) {
        super(scope, id, props);

        const parameterGroup = new ParameterGroup(this, 'ParameterGroup', {
            description: "Parameter group for " + process.env.CDK_DEPLOY_DISCOURSE_STACK_ID,
            engine: DatabaseInstanceEngine.postgres({ version: PostgresEngineVersion.VER_15_2 }),
            parameters: {
              shared_preload_libraries: 'pg_stat_statements'
            },
          });

        this.databaseInstance = new DatabaseInstance(this, "PostgresInstance", {
            engine: DatabaseInstanceEngine.postgres({ version: PostgresEngineVersion.VER_15_2 }),
            credentials: Credentials.fromGeneratedSecret('embeddings'),
            databaseName: 'embeddings',
            parameterGroup: parameterGroup,
            vpc: props.vpc,
            securityGroups: [props.securityGroup],
            vpcSubnets: {
                subnetGroupName: props.subnet.name
            },
            storageEncrypted: true,
            storageType: StorageType.GP3,
            multiAz: false,
            allocatedStorage: 20,
            allowMajorVersionUpgrade: false,
            autoMinorVersionUpgrade: true,
            deletionProtection: false,
            publiclyAccessible: false,
            
        });
    }
}
1 Like