# Algolia reindex problems

**URL:** https://meta.discourse.org/t/algolia-reindex-problems/296462
**Category:** Bug
**Tags:** algolia-search
**Created:** [February 23, 2024, 10:36am UTC](https://meta.discourse.org/t/algolia-reindex-problems/296462 "2024-02-23T10:36:28Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![RGJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/rgj/32/523185_2.png) [@RGJ](https://meta.discourse.org/u/RGJ)
#### Post date: [February 23, 2024, 10:36am UTC](https://meta.discourse.org/t/algolia-reindex-problems/296462/1 "2024-02-23T10:36:28Z")

</div>

While performing `algolia:reindex` using the rake task, it crashed with the following error

```plaintext
Clearing users from Algolia
Pushing users to Algolia
...
Successfully pushed 1849 users to Algolia
Clearing tags from Algolia
Pushing tags to Algolia
..
Successfully pushed 53 tags to Algolia
Clearing posts from Algolia
Pushing posts to Algolia
rake aborted!
Algolia::AlgoliaHttpError: Record at the position 662 objectID=690 is too big size=20920/20000 bytes. Please have a look at https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/in-depth/index-and-records-size-and-usage-limitations/#record-size-limits (Algolia::AlgoliaHttpError)

```

As far as I can see the plugin does not contain any functionality to split longer posts into separate chunks. I was able to work around the problem by not including long posts by adding

`objects.reject! { |object| object.to_json.bytesize > 20000 }`

right before the `@index.save_objects` call but it also means these posts are not being indexed at all.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [February 25, 2024, 9:34pm UTC](https://meta.discourse.org/t/algolia-reindex-problems/296462/2 "2024-02-25T21:34:08Z")

</div>

Any ideas what the API expects? Are we supposed to truncate or send it through in multiple chunks?

---

<div class="post-metadata">

### Author: ![RGJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/rgj/32/523185_2.png) [@RGJ](https://meta.discourse.org/u/RGJ)
#### Post date: [February 25, 2024, 10:42pm UTC](https://meta.discourse.org/t/algolia-reindex-problems/296462/3 "2024-02-25T22:42:17Z")

</div>

Per [Index long pages - Algolia](https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/how-to/indexing-long-documents/) it needs to be chunked, and `distinct` needs to be set to `true` when searching.

The maximum size is [dependent on the plan](https://support.algolia.com/hc/en-us/articles/4406981897617-Is-there-a-size-limit-for-my-index-records) but I cannot find a way to query it. Given the fact that there is also an “average record size across all records” limit on certain plans, it might be good to chunk at ~~10 KB~~ 10000 bytes.
