Discourse AI - Sentiment

[Falco]

This topic covers the configuration of the Sentiment module of the Discourse AI plugin.

Feature set

The Sentiment modules can automatically classify the sentiment and emotion score of every new post in your Discourse instance.

Settings

• ai_sentiment_enabled: Enables or disables the module

• ai_sentiment_inference_service_api_endpoint: URL where the API is running for the module. If you are using CDCK hosting this is automatically handled for you. If you are self-hosting check the self-hosting guide.

• ai_sentiment_inference_service_api_key: API key for the API configured above. If you are using CDCK hosting this is automatically handled for you. If you are self-hosting check the self-hosting guide.

• ai_sentiment_models: We off a model for sentiment and one for emotion. Enable the modules you want to use here. You can enable both.

[nodomain]

The sentiment service also errors:

INFO:     10.0.2.145:5142 - "POST /api/v1/classify HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 77, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/dist-packages/starlette_exporter/middleware.py", line 289, in __call__
    await self.app(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 106, in __call__
    response = await self.dispatch_func(request, call_next)
  File "./app.py", line 56, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 80, in call_next
    raise app_exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 69, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "./app.py", line 72, in classify
    results = models[parsed_params.model](params["content"])
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/text_classification.py", line 138, in __call__
    result = super().__call__(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/base.py", line 1067, in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/base.py", line 1074, in run_single
    model_outputs = self.forward(model_inputs, **forward_params)
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/base.py", line 983, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/text_classification.py", line 165, in _forward
    return self.model(**model_inputs)
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/transformers/models/roberta/modeling_roberta.py", line 1206, in forward
    outputs = self.roberta(
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/transformers/models/roberta/modeling_roberta.py", line 814, in forward
    buffered_token_type_ids_expanded = buffered_token_type_ids.expand(batch_size, seq_length)
RuntimeError: The expanded size of the tensor (1234) must match the existing size (514) at non-singleton dimension 1.  Target sizes: [1, 1234].  Tensor sizes: [1, 514]

Thanks for some tips. I am really interested in this

[RGJ (Richard - Communiteq)]

So what does this module actually do next to making an entry in the classification_results table?

[Falco]

@RGJ

So what does this module actually do next to making an entry in the classification_results table?

At the moment that’s all it does. We are working on summarizing and exposing the results of those classifications as information in the UI as show in

@Falco

User overall sentiment on the user profile page - Future UI mockup

Overall community sentiment over time - Future UI mockup

[mattdm (Matthew Miller)]

I love the UI mockup!

Does anyone have any Data Explorer queries to use in the meantime? My SQL is kinda on the ¿donde esta la biblioteca? level…

[Falco]

@AlexDev made some last week they can share here.

[AlexDev (Alex Devlaeminck)]

There’s a few DE queries for the Discourse AI plugin I shared here:

Community Sentiment and Toxicity queries

For those wondering how to view the classification results database for the Community Sentiment and Toxicity modules, this can be done using the Data Explorer plugin, and the classification_results table. This is useful for seeing how the AI plugin is functioning on your site and classifying posts. AI Sentiment SELECT target_id as post_id, model_used, classification->'negative' as negative, classification->'neutral' as neutral, classification->'positive' as positive from classification_result…

[MarieMc]

Hello @Falco do you have an update on when this widget will become available on in the admin dashboard?

[Falco]

This is scheduled for our next sprint, so I’d guess no later than October.

[Samantha_O]

Is this still expected no later than October? asking for a friend

[Falco]

Yup, we did a first pass in the equation we going to use to classify between neutral and the sentiments, and we liked what we saw, so we are moving into implementation next.