# Do not strip diacritics for search in Vietnamese

**URL:** https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050
**Category:** Bug
**Created:** [31.Август.2018 00:52:45 UTC](https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050 "2018-08-31T00:52:45Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Pham\_Quyet\_Nghi](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pham_quyet_nghi/32/100298_2.png) [@Pham\_Quyet\_Nghi](https://meta.discourse.org/u/Pham_Quyet_Nghi)
#### Post date: [31.Август.2018 00:52:45 UTC](https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050/1 "2018-08-31T00:52:45Z")

</div>

The results shown in the search do not display the Vietnamese sign

My post example has the title and the content as follows

**title**

> “không hiển thị dấu Tiếng Việt trong kết quả tìm kiếm”

**content**

> Khi tôi đăng câu hỏi lên diễn đàn với ngôn ngữ Tiếng Việt lúc tôi tìm kiếm thì không hiển thị đầy đủ ngôn ngữ

Enter search keywords “`không hiển thị dấu Tiếng Việt trong kết quả tìm kiếm`”

the result will be

> Khi toi dang cau hoi len dien dan voi ngon ngu Tieng Viet luc toi tim kiem thi khong hien thi đay đu ngon ngu

how to want to display the correct accented text?

Thanks.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [31.Август.2018 00:58:16 UTC](https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050/2 "2018-08-31T00:58:16Z")

</div>

Yes this is pretty tricky, I can see how it is a problem for Vietnamese communities, the excerpts must look very confusing.

In Vietnamese is there ever a reason to type without diacritics? eg type khong and mean không? I imagine it is a super duper hard **no** cause the language is tone based, so the is the equivalent of me typing “dog” and meaning “milk”.

I think the best way forward here is to make diacritic stripping optional and turn this off in Vietnamese communities.

A bigger and more complex change is to amend it so excerpts are generated based off cooked and not the “normalized” cooked text.

FYI English speakers:

dấu: A sign  
dàu: Head  
dãu: Pudding

So yeah this is a pretty giant issue for Vietnamese.

---

<div class="post-metadata">

### Author: ![Pham\_Quyet\_Nghi](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pham_quyet_nghi/32/100298_2.png) [@Pham\_Quyet\_Nghi](https://meta.discourse.org/u/Pham_Quyet_Nghi)
#### Post date: [31.Август.2018 01:11:47 UTC](https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050/4 "2018-08-31T01:11:47Z")

</div>

I see a page displaying bookmarks in search results  
Example, this page [See details](https://canthuenhadat.com.vn/search?q=thu%C3%AA%20m%E1%BA%B7t%20b%E1%BA%B1ng%20l%C3%A0m%20c%E1%BB%ADa%20h%C3%A0ng)

 ![hien%20th%E1%BB%8B%20k%E1%BA%BFt%20qu%E1%BA%A3%20c%C3%B3%20d%E1%BA%A5u%20trong%20t%C3%ACm%20ki%E1%BA%BFm](https://global.discourse-cdn.com/meta/original/3X/6/a/6a7554732afebe73bc19ff7d7404bf62a0901c7d.jpeg)

**this configuration where?** Thanks

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [31.Август.2018 01:13:05 UTC](https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050/5 "2018-08-31T01:13:05Z")

</div>

Not yet, I will add an option today, hold tight.

The reason it looks “good” on old search results is cause they have not been indexed yet using the new algorithm. If you edit any of the posts there the bug will pop up.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [31.Август.2018 01:49:35 UTC](https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050/6 "2018-08-31T01:49:35Z")

</div>

This is fixed per:

[https://github.com/discourse/discourse/commit/9b7cab589ac15a034d1c0e700230c1b3f63f8ba0](https://github.com/discourse/discourse/commit/9b7cab589ac15a034d1c0e700230c1b3f63f8ba0)

After you rebuild with this commit, make sure you run:

```plaintext
./launcher enter app
rake search:reindex

```

You can try rebuilding in 10-20 minutes when the commit hits tests passed.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [01.Сентябрь.2018 08:00:11 UTC](https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050/7 "2018-09-01T08:00:11Z")

</div>

This topic was automatically closed after 30 hours. New replies are no longer allowed.
