How to change the response default_limit in data explorer?

dev
1

Hi,

What is the best way to set the default_limit to something like 12345678 ?
Suppose that I need and I want more than 1000 rows.

2

I canā€™t figure out what default_limit youā€™re talking about. I donā€™t see that in site_settings.yml. What are you trying to do? And are you trying to do it in a plugin or theme component (since you posted in dev)?

1 Like
3

The data-explorer pluginā€™s default_limitā€¦

Capture dā€™eĢcran, le 2022-01-07 aĢ€ 18.07.22
Capture dā€™eĢcran, le 2022-01-07 aĢ€ 18.07.221862Ɨ1306 211 KB

4
1 Like
5

Ok, the only way is to set another value like this:
I was hoping there was a better way in Discourseā€™s settings UI.

./launcher enter app
cd /var/www/discourse/plugins/discourse-data-explorer/
sed -i 's/QUERY_RESULT_DEFAULT_LIMIT = 1000/QUERY_RESULT_DEFAULT_LIMIT = 12345678/g' /var/www/discourse/plugins/discourse-data-explorer/plugin.rb
sed -i 's/QUERY_RESULT_MAX_LIMIT = 1000/QUERY_RESULT_MAX_LIMIT = 12345678/g' /var/www/discourse/plugins/discourse-data-explorer/plugin.rb
6

Certainly not the only way.

As per my link, you can fork the plugin, change the constant and clone your fork.

You could always PR a plugin setting to that plugin ā€¦

1 Like
7

Not the only way. Itā€™s possible to make changes like that with launcher. You can log at the other templates for examples of how to make changes to files.

1 Like
8

If this isnā€™t the only way, can you provide an example thanks.

9

They recently change the code to a new hidden settingsā€¦

10 
cd /var/discourse
./launcher enter app
sed -i 's/QUERY_RESULT_DEFAULT_LIMIT = 1000/QUERY_RESULT_DEFAULT_LIMIT = 10000000/g' /var/www/discourse/plugins/discourse-data-explorer/plugin.rb
sed -i 's/QUERY_RESULT_MAX_LIMIT = 1000/QUERY_RESULT_MAX_LIMIT = 10000000/g' /var/www/discourse/plugins/discourse-data-explorer/plugin.rb
sed -i 's/default: 1000/default: 10000000/g' plugins/discourse-data-explorer/config/settings.yml
sed -i 's/max: 10000/max: 10000000/g' plugins/discourse-data-explorer/config/settings.yml
rails c
SiteSetting.data_explorer_query_result_limit=1000000
exit
exit
reboot
11

How can one change such a ā€œhidden settingā€ in the most forward way?

I hope it does not require editing code to change such a basic limit of the plugin.

12

You can change a hidden site setting using the rails console (if you have server access).

cd /var/discourse
./launcher enter app
rails c
SiteSetting.data_explorer_query_result_limit = 10000

Though the upper limit is 10,000, and I believe this is only the number of results that are displayed ā€˜on-screenā€™ - you can already export 10,000 results even with the setting at the default 1000.

I think adding a page offset to your query is useful if you want to grab batches to export and join together in a spreadsheet (etc). eg:

-- [params]
-- integer :page = 0
-- integer :limit = 10000

SELECT 
    category_id,
    created_at::date,
    id AS topic_id,
    views
FROM topics
WHERE deleted_at IS NULL
ORDER BY created_at DESC
OFFSET :page * :limit
LIMIT :limit
2 Likes
13

I only got 1000 results when querying a query via the API and considering the significant overhead per request (factor 100 to the raw request to the database) I rather increase that limit than do multiple queries. :slight_smile:

Is that rail console approach persistent over software upgrades and restarts?

14

Changing a site setting via the rails console is the same as doing it through the UI (just for the hidden ones there is no UI :slight_smile:), so the change will stick and not be affected by rebuilds or restarts, etc. :+1:

My knowledge on running them through the API is not as good as the regular way, but I think thereā€™s a method of getting the full 10,000 without changing the setting in the rails console in this topic - Run Data Explorer queries with the Discourse API

3 Likes
15

Thank you, that limit=ALL parameter solves it for me.

2 Likes