Discourse AI - NSFW

Discourse · 2023 年4 月 24 日 19:40

本主题涵盖了 Discourse AI 插件的 NSFW（不适宜工作场所）功能的配置。

所需用户级别：管理员

NSFW 模块可以自动为您的 Discourse 实例中帖子和聊天消息的每次新图片上传分类 NSFW 分数。此系统有助于识别和管理可能不适合专业或公共环境的内容。

您还可以启用对超出阈值的内容进行自动标记。

设置

ai_nsfw_detection_enabled：启用或禁用该模块

ai_nsfw_inference_service_api_endpoint：模块 API 运行的 URL。如果您使用 CDCK 托管，则会自动处理。如果您自行托管，请查看自行托管指南。

ai_nsfw_inference_service_api_key：上面配置的 API 的 API 密钥。如果您使用 CDCK 托管，则会自动处理。如果您自行托管，请查看自行托管指南。

ai_nsfw_models：我们将用于图像分类的模型。我们提供两种不同的模型及其阈值：

opennsfw2 返回一个介于 0.0 和 1.0 之间的分数。通过 ai_nsfw_flag_threshold_general 设置，将上传被视为 NSFW 的阈值。

nsfw_detector 返回四个类别的分数：drawings（绘画）、hentai（动漫）、porn（色情）和 sexy（性感）。通过各自的 ai_nsfw_flag_threshold_${category} 设置每个类别的阈值。如果通用阈值低于其中任何一个，我们将认为该内容为 NSFW。

ai_nsfw_flag_automatically：自动标记高于配置阈值的帖子/聊天消息。禁用此设置时，我们仅将分类结果存储在数据库中。

附加资源

Discourse AI

Install plugins on a self-hosted site

nodomain · 2023 年4 月 29 日 22:19

谢谢。

我在使用 NSFW 服务时发现了这个问题，上传图片后出现。我使用 S3 进行图片存储，如果这很重要的话。

INFO:     10.0.2.145:23412 - "POST /api/v1/classify HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 77, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/dist-packages/starlette_exporter/middleware.py", line 289, in __call__
    await self.app(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 106, in __call__
    response = await self.dispatch_func(request, call_next)
  File "./app.py", line 49, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 80, in call_next
    raise app_exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 69, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "./app.py", line 79, in classify
    results = models[parsed_params.model](tfile)
  File "/usr/local/lib/python3.9/dist-packages/opennsfw2/_inference.py", line 31, in predict_image
    pil_image = Image.open(image_path)
  File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 3283, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file '/tmp/tmpbqq82655'
ERROR:uvicorn.error:Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 77, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.9/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/dist-packages/starlette_exporter/middleware.py", line 289, in __call__
    await self.app(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 106, in __call__
    response = await self.dispatch_func(request, call_next)
  File "./app.py", line 49, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 80, in call_next
    raise app_exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 69, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "./app.py", line 79, in classify
    results = models[parsed_params.model](tfile)
  File "/usr/local/lib/python3.9/dist-packages/opennsfw2/_inference.py", line 31, in predict_image
    pil_image = Image.open(image_path)
  File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 3283, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file '/tmp/tmpbqq82655'

在 Sidekiq 中看起来是这样的：

我哪里做错了？

nodomain · 2023 年4 月 30 日 08:36

我认为这可能与 S3 有关。上传图片时，错误日志显示了这些错误：
S3 的 URL 已隐藏…

所以看起来该插件尝试过早地加载图片，或者由于其他原因找不到它。

Falco · 2023 年4 月 30 日 19:48

我只用过使用 S3 CDN 或使用本地存储的网站进行测试。如果使用对象存储，我建议使用 CDN 来进行前端访问。

nodomain · 2023 年4 月 30 日 21:36

我们正是这样做的——AWS Cloudfront。该堆栈基于 https://github.com/aws-samples/aws-cdk-for-discourse。我将尝试调试并发布我的结果。

Noxitb · 2024 年5 月 20 日 03:41

我从哪里获取API密钥？

NateDhaliwal · 2024 年6 月 6 日 09:41

应该在管理员面板下，对吗？

Shauny · 2024 年7 月 15 日 12:23

对我来说，这似乎过于敏感了，它标记的内容太多了。仅仅是人的照片就会被标记为 NSFW。

我的阈值设置为 60。我不清楚是应该增加还是减少这个数字，以及它的限制是什么？如果我将其设置为 100，这是否相当于禁用了它？关于它的文档似乎不多。

另外，为什么不像毒性那样，没有办法为特定角色完全禁用它？我确信我的工作人员不会发布任何粗鲁的内容，但他们却因为 AI 标记他们的内容而不断地与之争论。这在我的论坛上已经成了一个笑话。

正在使用 opennsfw2。

Falco · 2024 年8 月 1 日 16:02

不幸的是，那些旧的 ML 模型缺乏灵活性，虽然它们对运行自己的实例的要求不高，但我们发现这种严格性对于使用 Discourse 的各种社区来说是一个太大的障碍。

目前我们的计划是弃用所有依赖于这些旧模型的特性，并将所有内容迁移到现代 LLM，这样您就可以调整提示以更好地满足您的需求。

Shauny · 2024 年10 月 6 日 16:08

这个有任何改变吗？NSFW检测器仍然非常不准确，并且在大多数包含人物的图片上都会触发。

Falco · 2024 年10 月 6 日 17:26

是的，我们在 Discourse AI 中 NSFW 检测的后续计划中已经沟通了我们的计划，但简而言之，您可以使用 Discourse AI Post Classifier - 自动化规则来获得更高的准确性和灵活性。

我们通过 Vision LLMs 对 NSFW 用例进行了评估，并取得了优异的结果，因此我们目前正在为企业客户推出此功能，并将在未来几周内发布相关指南。

话题		回复	浏览量
Setting up NSFW detection in your community Site Management moderation , automation , how-to , ai	0	716	2024 年10 月 10 日
NSFW image blurring in chat Support chat , ai	5	513	2024 年9 月 26 日
Discourse AI Plugin official , included-in-core , ai	89	37572	2025 年10 月 14 日
Introducing Discourse AI Blog	26	3626	2023 年5 月 4 日
Have AI check for inappropriate post or at least words and flag the post Support ai , ai-toxicity	3	403	2023 年7 月 7 日

Discourse AI - NSFW

设置

附加资源

相关话题