Daily Summary Feedback

Totally off topic but I am seeing a pattern here. More anglorelated ones are naming AI as a man. And from different language and culture families coming ones see AI as a woman.

Well… if you guys change that name it may start behave and is not so pissed all the time :wink:

I let him have another go. He’s decided to just not include the links this time. :person_shrugging:

1 Like

I do have to say: it’s really no longer a summary at this point… I don’t think it’s doing the concept of highlights very well atm.

5 Likes

I’m girding myself for another crack at the prompt.

I’m thinking, “Bert, make this more comprehensive and include your own thoughts to expand on topics you particularly enjoyed reading”

2 Likes

Recommendation

Have a fourth automation rule you use for testing, have it only target yourself in a PM

Makes feedback loop far shorter, you can run through 5-6 iterations in a few minutes

The issue I’m having is that it produces a decent result and then degrades over time based on the same prompt. I’m not sure how to compensate for that.

1 Like

Decent result consistently when testing? I am not sure but maybe reducing temp or top p may help here, I thought I had it close to zero

The last two seem to be much closer to what’s intended, so I’m not sure what could be changed to make it stick to the script more consistently. I can try a fourth automation and rapid testing, but if 5 come out good and the next 5 not so good then I’m not 100% sure what levers I can pull to tighten it up. I’m not that familiar with writing AI prompts so that’s going to play a part. I’m also not sure how he decides what’s ’interesting’ and whether it’s just a bit random.

I tried my hand at an engagement report:

topic Views Topic Likes Watching Tracking Manually Muted
Daily Summary (9pm UTC) (9) 331 4 1 5 11
Daily Summary (5am UTC) (10) 395 5 1 5 13
Daily Summary (1pm UTC) (10) 391 2 1 6 12
Weekly Summary (2) 108 5 2 4
1 Like

I don’t know at all how GPT (directly or thru API) works. System prompts are really hard for me, and using english creative way doesn’t make it any easier. Because of those and a couple other things I haven’t too realistic expectations what GPT can do. So I am like GPT in this matter: no knowlidge, reading easy access copy&paste articles where I don’t know at all if facts are even remotely trustful and blanks I’m filling using halkucination. But I have a lot, and I mean really lot tests and tries burning way too much money.

Is that enough about disclaimer?

I claim GPT has longer memory than it should. Or I put it that way: it depends of, I don’t know, like how much users OpenAI has. Or how close using cap one is. But defenetly fresh gpt-4 remembers way longer than it should. Gpt-3.5 line not that much and I claim that 3.5 models has lobotimised receltly favour of 4.

And you guys will take step fpteard and say I should leave grownups alone and have theirs conversation because you aren’t using GPT by OpenAI :rofl:

But everytime I make iterations changing prompt a bit and starting new conversations I got same’ish results: first close or totally what I wanted, and then it started fail me.

Then I started to do a bit different things and I managed get answers that weren’t possible unless GPT have longer memory. Sure, there is strong possibility I actually feed false memories to it, or I wasn’t as smart as I thought (again, english is difficult to me) and it was just lucky guess or coincident, aka. what GPT does.

So what you guys call degration is actually kind of frustration. Stupid human is asking same question again and again and smart world dominating intelligence try give a bit different answe try after try. So, in the end it has zero interesting topics from here left, because it offered those all thru iteration rounds.

Your Bert is just really pissed off because nothing it offers to you isn’t good enough and finally it just say screw you guys, I’m going home — well, it is pollite guy so he doesn’t use exactly those wording, but the tone is same.

TLDR; iterations can waste possibilities because there is not endless amount of new and interesting topics (no matter if devs say it is not possible; those guys can’t read humans, so I’m not totally sure if they undestand AI any better :rofl: )

Idk, I start to feel rather annoyed of this bot. I generally really appreciate that there is more product experimentation coupled with user feedback. But this feature is so experimental right now, it feels somewhat non-consensual to be consistently mentioned by this bot. And needing to mute the system user to not take part in this doesn’t seem the right way to drop out here.

I’d much prefer if this could be implemented in a less intrusive way as for now.

1 Like

I think you should be able to mute the topics rather than the @Discourse user if you wanted to opt-out, but I do think the idea of it being opt-in was more appealing.

1 Like

Continuing the discussion from Daily Summary (5am UTC):

So I saw this topic: Daily Summary Feedback, which appears to tell me about the link pings in the category.

I noticed I got three link pings in a row from this topic:
image
I wonder why I got link pinged three times by the AI-summary.

If you check out any of the OPs of the summary topics, there’s some explainer in there as well. :+1:

2 Likes

This was discussed before

2 Likes

These are helpful summaries I like them.

Does anyone actually read it for informational purposes? My opinion is that it’s too specific. I would be more interested in higher level summaries. Like a weekly summary that takes all the daily ones and synthesizes all the content at a higher level.

Like instead of “this person informed that person about a bug” maybe like “11 new bugs reported this week bug

Or something like “AI was the most discussed topic this week with a new plugin being released [url]”

2 Likes

I think we should dial this back to 1 per day. The three per day to try to account for different time zones doesn’t feel like it is worth the tradeoffs at this time.

If we really want something like that, it feels like we need an AI bot persona instead, and people can ask it what happened since they last visited and to, “oh, please bring me a coffee with the news too, will ya?”

5 Likes

Actually… I like that idea of butler. And if playing more this mindgame — even better if it would work similar way than discobot, or if we could have link-a-like to put in menus.

I need my report at 8AM, @pmusaraj loves his report in his morning.

Before we throw the baby out with the bathwater, what are the actual open issues?

  • People need to actively mute, vs starting off muted
  • Rubbish links in topics
  • Rubbish notifications

I can think of solutions for all of these, especially if we are willing to hide these reports from anonymous for now.


An example comprehensive solution that makes this opt in:

  • Make 3 secure categories for the 3 reports
  • Allow people to freely join 1 of the 3 groups the reports are secured under
  • Add a new markdown extensions ?silent=true at end of URL means, no notification, no linked topic. (which is generally useful anyway)

Problem solved?

4 Likes

I don’t have very strong feelings about this, but I am trying to imagine where this ends up, and I’m having trouble imagining something very elegant right now.

I certainly am not advocating for making things more complex right now.

Do you have something in mind yet for where this ends up?

1 Like