Load Data Explorer queries from library

It’s great to have these all in one place, rather than spread across meta!

I see you’ve organised them into your fork of the discourse-data-explorer repository. Did you have any plans to improve the plugin to access the pre-made queries directly?

I think it would be really cool to have a “load from library” button in the plugin, which allows browsing these queries. Adding new queries to the library would then just need a PR. Would that be pr-welcome @team?

Yeah a nice way to share queries would be nice, I worry about vetting.

I think the best way is to ship data explorer with a large amount of built in queries, then we do PRs whenever we want to add queries.

Can you do mockups of the UI for that? I totally support adding something here.

I’m imagining a new button on the main plugin interface “Load from Library”

Which then opens a popup with a scrollable and filterable table of available queries

In terms of the data itself, I quite like how @SidV has stored them in their own folder, each in their own .sql file here

We would need to add some metadata to the top of the sql files. We already have [params], so maybe have something like

-- [title] Solved Answers
-- [description] Lists users with most solved topics
-- [source] https://meta.discourse.org/t/63981/4?u=sidv

-- [params]
-- string :interval = 1 year
SELECT  ua.acting_user_id, 
        count(case t.user_id when ua.acting_user_id then 1 else null end) as is_op, 
        count(case t.user_id when ua.acting_user_id then null else 1 end) as not_op,
        count(*) as total
FROM user_actions ua
LEFT JOIN topics t ON target_topic_id = t.id
WHERE action_type=15
AND ua.created_at >= CURRENT_DATE - INTERVAL :interval
GROUP BY ua.acting_user_id

I find the whole selection of queries from a combobox deeply flawed, and it breaks down big time when you have a lot of queries.

What we should simply do it display a full list of linkable queries on the front page, we can add an icon next to builtin to show that they are built in and not editable.

The whole “clone from library” workflow feels like busywork to me.

I agree about the combobox.

I was trying to keep within the existing structure of the UI, but rethinking it from scratch will probably give a better result. It does significantly increase the amount of work that this is though :wink:

Мне очень нравится эта инициатива сделать лучшие запросы более доступными. Но не было бы безопаснее и проще создать здесь, на Meta, категорию #queries, аналогичную Customization > Plugin, где для каждого запроса будет отдельная тема? Там можно публиковать безопасные и проверенные запросы, а неработающие переносить в категорию #broken-queries.

Преимущество в том, что мы можем обсуждать, задавать вопросы и совершенствовать запросы прямо здесь, на Meta. Это также гораздо проще реализовать, хотя, конечно, потребуется определённая работа по поддержанию порядка. :seedling:

Possibly, trouble is we would not have safety. How would we vet that a query or edit to a query is “safe”.

When you say “safe”, what do you mean?

I thought that the data-explorer plugin is inherently safe - it only has read-only access to the database, and there are timeouts in place to prevent performance issues.

Or are you simply talking about a query being working/not working?

Working/not working I guess. There is is also the question of if it works as described.

But yeah, I guess a category on meta can work as the source of truth, would make it very easy to add queries and discuss them.

We would have to be super careful with naming conventions and kicking out dupes.

Data Explorer теперь включает множество встроенных запросов, и мы можем добавлять новые через PR в репозиторий data-explorer, так что этот вопрос решён :smiley: