# Slow Data Explorer queries

**URL:** https://meta.discourse.org/t/slow-data-explorer-queries/290217
**Category:** Data & reporting
**Tags:** performance
**Created:** [January 1, 2024, 8:49pm UTC](https://meta.discourse.org/t/slow-data-explorer-queries/290217 "2024-01-01T20:49:35Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![n1bff](https://avatars.discourse-cdn.com/v4/letter/n/e95f7d/32.png) [@n1bff](https://meta.discourse.org/u/n1bff)
#### Post date: [January 1, 2024, 8:49pm UTC](https://meta.discourse.org/t/slow-data-explorer-queries/290217/1 "2024-01-01T20:49:35Z")

</div>

I created some [Data Explorer](https://meta.discourse.org/t/32566?silent=true) queries.

If I execute them in the admin backend, the GUI shows me a very quick execution time for the DB query itself, e.g. “Query completed in 7.9 ms.”

The network log of my browser shows that e.g. [https://example.com/admin/plugins/explorer/queries/1/run](https://example.com/admin/plugins/explorer/queries/1/run) was accessed and that it took a 1-2 _magnitudes longer_ to get the result, e.g. 150ms.

When I `curl` such a query it takes even longer, about _twice_ the time compared to running the query in the admin panel.

How can I have [Data Explorer](https://meta.discourse.org/t/32566?silent=true) queries that I can retrieve _fast_ without that much overhead?

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [January 1, 2024, 8:57pm UTC](https://meta.discourse.org/t/slow-data-explorer-queries/290217/2 "2024-01-01T20:57:53Z")

</div>

From the information you provided, I’d say that those are simple queries (fast to execute in the RDBMS), that return many rows (slow to de/serialize in the backend and transfer).

If the current performance is blocking your use case, there are many different solutions you can explore:

- Lowering row counts per page, so each page is faster.

- Building a plugin that exposes the same information in a new API endpoint and is smarter about caching and data format.

- Querying the underlying database directly.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [January 1, 2024, 9:18pm UTC](https://meta.discourse.org/t/slow-data-explorer-queries/290217/3 "2024-01-01T21:18:26Z")

</div>

> [@n1bff](#):
>
> DB query itself, e.g. “Query completed in 7.9 ms.”

We have magical lookups for certain column names (user\_id, post\_id, etc) alias them to something else like post\_id1 etc..

---

<div class="post-metadata">

### Author: ![n1bff](https://avatars.discourse-cdn.com/v4/letter/n/e95f7d/32.png) [@n1bff](https://meta.discourse.org/u/n1bff)
#### Post date: [January 1, 2024, 9:36pm UTC](https://meta.discourse.org/t/slow-data-explorer-queries/290217/4 "2024-01-01T21:36:01Z")

</div>

Thanks guys!

> [@Falco](#):
>
> - Lowering row counts per page, so each page is faster.

It is the same if I have just one single row as result of a query. E.g. a query that returns a single integer, ran in the [Data Explorer](https://meta.discourse.org/t/32566?silent=true) panel in the admin backend it takes 9 ms, ran from the outside against the [Data Explorer](https://meta.discourse.org/t/32566?silent=true) API via curl it takes 200 ms.

> [@Falco](#):
>
> - Building a plugin that exposes the same information in a new API endpoint and is smarter about caching and data format.
> - Querying the underlying database directly.

I would very much like to avoid that, I use [Data Explorer](https://meta.discourse.org/t/32566?silent=true) because it was suggested on these forums as a nice way to get information from the DB and a well working API for that.

> [@sam](#):
>
> We have magical lookups for certain column names (user\_id, post\_id, etc) alias them to something else like post\_id1 etc…

My query uses them for JOINs but not in the result set. The SELECT only has one column and that is a newly calculated one with a arbitrary name.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [January 1, 2024, 9:54pm UTC](https://meta.discourse.org/t/slow-data-explorer-queries/290217/5 "2024-01-01T21:54:53Z")

</div>

> [@n1bff](#):
>
> My query uses them for JOINs but not in the result set. The SELECT only has one column and that is a newly calculated one with a arbitrary name.

What does NGINX say the server time is for the request, could this be a network issue?

---

<div class="post-metadata">

### Author: ![n1bff](https://avatars.discourse-cdn.com/v4/letter/n/e95f7d/32.png) [@n1bff](https://meta.discourse.org/u/n1bff)
#### Post date: [January 3, 2024, 6:49pm UTC](https://meta.discourse.org/t/slow-data-explorer-queries/290217/6 "2024-01-03T18:49:11Z")

</div>

Assuming that those are the floating point numbers 3 places right of the HTTP status code (e.g. `... 200 642 "-" **0.107 0.108**"system" ...`, highlighted with stars by me) they are about 10 times the duration of the query time reported in the backend and pretty much exactly half the time it takes to `curl` from my client.
