Discourse AI - NSFW

Falco · April 24, 2023, 7:40pm

This topic covers the configuration of the NSFW (Not Safe For Work) module of the Discourse AI plugin.

Feature set

The NSFW modules can automatically classify the NSFW score of every new image upload in posts and chat messages in your Discourse instance. This system helps identify and manage content that may be inappropriate or explicit for a professional or public environment.
Optionally, you can also enable automatic flagging of content that crosses a threshold.

Settings

ai_nsfw_detection_enabled: Enables or disables the module
ai_nsfw_inference_service_api_endpoint: URL where the API is running for the module. If you are using CDCK hosting this is automatically handled for you. If you are self-hosting check the self-hosting guide.
ai_nsfw_inference_service_api_key: API key for the API configured above. If you are using CDCK hosting this is automatically handled for you. If you are self-hosting check the self-hosting guide.
ai_nsfw_models: The model we’ll use for image classification. We offer two different ones with their thresholds:
- opennsfw2 returns a single score between 0.0 and 1.0. Set the threshold for an upload to be considered NSFW through the ai_nsfw_flag_threshold_general setting.
- nsfw_detector returns scores for four categories: drawings, hentai, porn, and sexy. Set the threshold for each one through the respective ai_nsfw_flag_threshold_${category} setting. If the general one is lower than any of these, we’ll consider the content NSFW.
ai_nsfw_flag_automatically: Automatically flag posts/chat messages that are above the configured thresholds. When this setting is disabled, we only store the classification results in the database.

nodomain · April 29, 2023, 10:19pm

Thanks.

I found this issue when using the nsfw service, after uploading an image. I use S3 for image storage, if this is relevant.

INFO:     10.0.2.145:23412 - "POST /api/v1/classify HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 77, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/dist-packages/starlette_exporter/middleware.py", line 289, in __call__
    await self.app(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 106, in __call__
    response = await self.dispatch_func(request, call_next)
  File "./app.py", line 49, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 80, in call_next
    raise app_exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 69, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "./app.py", line 79, in classify
    results = models[parsed_params.model](tfile)
  File "/usr/local/lib/python3.9/dist-packages/opennsfw2/_inference.py", line 31, in predict_image
    pil_image = Image.open(image_path)
  File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 3283, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file '/tmp/tmpbqq82655'
ERROR:uvicorn.error:Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 77, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/dist-packages/starlette_exporter/middleware.py", line 289, in __call__
    await self.app(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 106, in __call__
    response = await self.dispatch_func(request, call_next)
  File "./app.py", line 49, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 80, in call_next
    raise app_exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 69, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "./app.py", line 79, in classify
    results = models[parsed_params.model](tfile)
  File "/usr/local/lib/python3.9/dist-packages/opennsfw2/_inference.py", line 31, in predict_image
    pil_image = Image.open(image_path)
  File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 3283, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file '/tmp/tmpbqq82655'

In Sidekiq it looks like this:

What am I doing wrong?

nodomain · April 30, 2023, 8:36am

I think this maybe related to S3 really. When uploading a picture, the error logs shows these errors:
URL to S3 redacted…

So it seems to be that the plugin tries to load the image too early or it does not find it due to other reasons.

Falco · April 30, 2023, 7:48pm

I only tested it with sites using a CDN for S3 or using local storage. If using Object Storage I advise using a CDN to front it.

nodomain · April 30, 2023, 9:36pm

That’s exactly what we do - AWS Cloudfront. The stack is based on GitHub - aws-samples/aws-cdk-for-discourse: AWS CDK for Discourse. I’ll try to debug and post my results.

Topic		Replies	Views
Discourse AI Post Classifier - Automation rule announcements automation , ai	12	1577	November 1, 2023
Discourse AI - Toxicity admins ai , ai-toxicity	4	2210	November 23, 2023
Discourse AI - Self-Hosted Guide sysadmin ai	59	6642	March 9, 2024
3.3.0.beta1: Discourse Discover opt in, Hot topics page, "It's illegal" flag reason, and more! announcements release-notes	1	1185	March 15, 2024
Discourse AI plugin official , ai	13	19301	March 26, 2024

Discourse AI - NSFW

Feature set

Settings

Related Topics