Enable Sentiment Analysis

documentation admins
,
2

The sentiment service also errors:

INFO:     10.0.2.145:5142 - "POST /api/v1/classify HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 77, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/dist-packages/starlette_exporter/middleware.py", line 289, in __call__
    await self.app(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 106, in __call__
    response = await self.dispatch_func(request, call_next)
  File "./app.py", line 56, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 80, in call_next
    raise app_exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 69, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "./app.py", line 72, in classify
    results = models[parsed_params.model](params["content"])
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/text_classification.py", line 138, in __call__
    result = super().__call__(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/base.py", line 1067, in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/base.py", line 1074, in run_single
    model_outputs = self.forward(model_inputs, **forward_params)
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/base.py", line 983, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
  File "/usr/local/lib/python3.9/dist-packages/transformers/pipelines/text_classification.py", line 165, in _forward
    return self.model(**model_inputs)
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/transformers/models/roberta/modeling_roberta.py", line 1206, in forward
    outputs = self.roberta(
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/transformers/models/roberta/modeling_roberta.py", line 814, in forward
    buffered_token_type_ids_expanded = buffered_token_type_ids.expand(batch_size, seq_length)
RuntimeError: The expanded size of the tensor (1234) must match the existing size (514) at non-singleton dimension 1.  Target sizes: [1, 1234].  Tensor sizes: [1, 514]

Thanks for some tips. I am really interested in this :slight_smile:

1 Like
3

So what does this module actually do next to making an entry in the classification_results table?

1 Like
4

At the moment that’s all it does. We are working on summarizing and exposing the results of those classifications as information in the UI as show in

8 Likes
5

I love the UI mockup!

Does anyone have any Data Explorer queries to use in the meantime? My SQL is kinda on the ¿donde esta la biblioteca? level…

6

@AlexDev made some last week they can share here.

3 Likes
7

There’s a few DE queries for the Discourse AI plugin I shared here:

5 Likes
9

Hello @Falco do you have an update on when this widget will become available on in the admin dashboard?

1 Like
10

This is scheduled for our next sprint, so I’d guess no later than October.

1 Like
11

Is this still expected no later than October? asking for a friend

4 Likes
12

Yup, we did a first pass in the equation we going to use to classify between neutral and the sentiments, and we liked what we saw, so we are moving into implementation next.

6 Likes
13

Hyped for Halloween and sentiment’s release! :rocket:

14

Will there be any “backfilling” capabilities for this module (to calculate sentiment for pre-existing content)?

2 Likes
15

Yes, this should be reasonably easy to add (at least a rake task for it)

4 Likes
16

Any timeframe on when the sentiment dashboard of sorts will be available? To Hifihedgehog’s point - it’s almost all Hallow’s Eve and with that comes the end of October. Can we expect our trick or treating gift from Discourse to be this? asking for a friend… again

:jack_o_lantern:

7 Likes
17

We are on it, @Roman_Rizzi is working on it this week, hard to put a date on it though, we hope to have something in next few weeks and will reply here as soon as it is ready.

In the mean time, are you using data explorer queries? they provide all the information its just that UI is not as fancy.

6 Likes
18

Are there any plans to add support for other languages?

2 Likes
19

Yes, both by adding multilingual simple ML models and by using multilingual LLMs to classify the data instead of dedicated models.

2 Likes
20

We’ve released the first pass on exposing this info in the dashboard:

image
image1269×627 25.7 KB

It will rollout to our customer with the module enabled in the next deploy. Emotion still need some work, but we are getting there!

6 Likes
21

Looks quite good on my forum! Is there a way to fill in more of a backlog after installing the plugin (self-hosted)? I’d be interested in running the analysis on all the posts in the last quarter or so in the background.

1 Like
22

We will write a proper backfill rake task for this in the coming weeks!

7 Likes