I am pretty impressed with Claude 3 🎉

Community Building
,

AI-generated summary

The discussion revolves around the effectiveness of various AI models for generating topic summaries on a Hungarian-language forum. Don shares their experience, starting with GPT-4 and GPT-3.5, which were either too expensive or defaulted to English summaries. They found that Claude 3 Haiku was effective for generating Hungarian summaries at a low cost, making it ideal for initial use. After testing, they upgraded to Claude 3 Opus, which impressed them with its ability to analyze images, categorize posts, and generate statistical insights from topics—marking a significant improvement. Jagster notes that GPT-4o often defaults to Finnish or English, depending on content, and lacks consistent language control. sam acknowledges the issue with English-only summaries from Gemini Flash and agrees it should be fixed. Don concludes that starting with Haiku allows for cost-effective testing, with the option to scale up to Sonnet or Opus as usage grows, making Claude 3 a top choice for multilingual, high-quality summarization.

1

Hello :waving_hand:

I want to share a little hint for those who want to set up topic summary AI feature. I tried a lot of model and I was looking for the model which is cheap for massive usage and can recognise the language of our site which is Hungarian. First I tried GPT4 which is of course great but expensive, GPT3.5 is most of the time created the summaries in English.

Then I tried the Gemini 1.5 line-up, the Pro version is also great but that seems too expensive too for us. The Flash was what I really want! But Flash in my tests created the summaries only in English. :frowning:

Then I tried the Claude 3 line-up which is awesome in my opinion. The smallest model, Haiku can creates the most of the time, I mean ~95% great topic summaries in Hungarian language which is the main goal on our forum. I tried the Sonnet too which impressed me more. But I picked the Haiku for now because of the price and we want to get some usage statistics before goes up to Sonnet. It’s cheap and clever enough. I suggest it everyone to try.

9 Likes
2

With OpenAI we need ability to tell language. General one is not liable.

But. GPT-4o uses in my case Finnish almost all the time. If there is images or piece of text in English it creates summary in English. And because it is OpenAI even that isn’t consistent :man_facepalming:

1 Like
3

I definitely want to get this fixed, I am sure we can do something

6 Likes
4

Update: I recalculated the things and noted that if I enabled first Haiku, members used it as a new thing. They generated in a few days a few hundred summarization ~800. Than we can go higher because those topics has been summarized already and these will reduce the usage. So now we are using Opus after a few days Sonnet usage and Opus is a whole new dimension. :flushed_face: Opus can recognize images and can makes statistics about the topic posts. I mean e.g. there 5000 posts in topic and Opus can categorized the replies and list the most replied things and makes suggestions based from posts. I think Opus is the best I’ve ever tried, yet. :sweat_smile:

So in my opinion if you start with the cheapest which is now Haiku then you can go upper to Sonnet and go on because the usage will reduced. Which is cool because you have the option to check every model. :slightly_smiling_face: But it’s really depends on the forum usage of course…

3 Likes