### Author:  [@Canapin](https://meta.discourse.org/u/Canapin)
#### Post date: [19 juni 2018 om 10:52 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/1 "2018-06-19T10:52:24Z")
Hi.
In French, we have a lot of accented characters, and also **a lot** of people that don’t bother typing them when writing.
Because Discourse seems to be strict regarding accents, this leads to partial results only when a user enters a string which can contain accents, from the search feature or the category filter when creating a topic :
And
And so on…
In the topic creation:
But:
I think this is a major flaw regarding the search efficiency in some languages.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [20 juni 2018 om 01:03 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/2 "2018-06-20T01:03:37Z")
I support a change that strips off all accent chars prior to tokenizing it feels reasonably safe.
Going to add this to our 2.1 roadmap.
---
### Author:  [@danekhollas](https://meta.discourse.org/u/danekhollas)
#### Post date: [20 juni 2018 om 01:45 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/3 "2018-06-20T01:45:26Z")
OMG, thanks in advance! This is huge issue for Czech alphabet as well.
I am slightly confused though because I am pretty sure I saw a related topic here on META which appeared to claim that this should be handled by a proper setting of PostgreSQL DB.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [20 juni 2018 om 01:50 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/4 "2018-06-20T01:50:44Z")
In theory it should once we configure unaccent right per: [PostgreSQL: Documentation: 9.6: unaccent](https://www.postgresql.org/docs/9.6/static/unaccent.html)
But I feel like the simplest and most robust thing to do here is just to never send these accents up to the index in the first place, do the work in ruby to remove them. Then we do not reply on PG text search being configured just so.
That said if you are feeling brave I recommend you try some of those options above to correct this.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [26 juni 2018 om 00:04 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/5 "2018-06-26T00:04:25Z")
@j.jaffeux what is your opinion on this regarding select-kit ?
We would need something like this:
>
---
### Author:  [@j.jaffeux](https://meta.discourse.org/u/j.jaffeux)
#### Post date: [26 juni 2018 om 07:04 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/6 "2018-06-26T07:04:30Z")
Should be easy to support and a good idea. normalize is not supported by IE but maybe we can make it work with [unorm/src/unorm.js at master · walling/unorm · GitHub](https://github.com/walling/unorm/blob/master/src/unorm.js)
---
### Author:  [@j.jaffeux](https://meta.discourse.org/u/j.jaffeux)
#### Post date: [26 juni 2018 om 10:21 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/7 "2018-06-26T10:21:47Z")
I just merged a commit supporting this:
[https://github.com/discourse/discourse/commit/eb9b99e5193ab82e551351d07b973ad9714ff378](https://github.com/discourse/discourse/commit/eb9b99e5193ab82e551351d07b973ad9714ff378)
For now it won’t work with browser which don’t have native support for normalize as shown here: [String.prototype.normalize() - JavaScript | MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/normalize)
@Canapin if you have time to test this it should solve the issue you had in topic creation.
---
### Author:  [@rriemann](https://meta.discourse.org/u/rriemann)
#### Post date: [18 augustus 2018 om 16:38 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/8 "2018-08-18T16:38:53Z")
This bug report from 2017 is related:
> [@Username completition broken for names with accents like Régis](https://meta.discourse.org/t/username-completition-broken-for-names-with-accents-like-regis/65934):
>
> Hello, In my forum, people have real names with all different kind of accents. I tried to use the username completition launched by typing @ to mention a user while writing a response and found that I could not find users by their name if the name contains an é. You can test it on your own. Try to find @zogstrip by typing @ followed by Ré for Régis. The actual username lookup is https://meta.discourse.org/u/search/users?term=r&topic\_id=65888&include\_groups=true. Note that all characters from t…
---
### Author:  [@danekhollas](https://meta.discourse.org/u/danekhollas)
#### Post date: [23 augustus 2018 om 02:40 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/9 "2018-08-23T02:40:37Z")
Hm, just so I understand, this was fixed only partially, right? The full search is still sensitive to accents as far as I can tell. I am only confused because this was marked as done in
> [@Discourse Version 2.1](https://meta.discourse.org/t/discourse-version-2-1/86339):
>
> Focuswhite_check_mark add “welcome to our community” PM for users who successfully transition between TL0 (new signup) and TL1 (15 mins reading, 5 topics entered, etc) white_check_mark Improve [handling](https://meta.discourse.org/t/disclosure-of-s3-secret-access-key/87221/22) of site setting secrets @techAPJwhite_check_mark [2FA Backup codes](https://meta.discourse.org/t/2fa-backup-codes/87038) @majawhite_check_mark allow selecting a tag when moving posts to a new topic, if tags are enabled (right now you can only pick category) @majawhite_check_mark Upgrade ImageMagick to ImageMagick 7 (6 is …
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [23 augustus 2018 om 02:46 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/10 "2018-08-23T02:46:15Z")
Yes this is not complete yet, we still need to get to stripping accents from search here:
>
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [23 augustus 2018 om 15:17 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/12 "2018-08-23T15:17:15Z")
Ít̊’̊s̘ n̐ǫw͋ d̻oͭn͑e̐ 👌
[https://github.com/discourse/discourse/commit/2fcf2b899e6193701b5c56414980789f0d0f2cbd](https://github.com/discourse/discourse/commit/2fcf2b899e6193701b5c56414980789f0d0f2cbd)
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [23 augustus 2018 om 16:02 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/13 "2018-08-23T16:02:07Z")
This topic was automatically closed after 41 minutes. New replies are no longer allowed.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [23 augustus 2018 om 21:12 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/14 "2018-08-23T21:12:21Z")
Note search within topic is still going to have the issue, sadly it is much harder to sort out
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [31 augustus 2018 om 01:24 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/15 "2018-08-31T01:24:27Z")
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [31 augustus 2018 om 01:30 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/16 "2018-08-31T01:30:26Z")
OK this is absolutely not PR welcome territory cause it is a very tricky problem to solve.
In particular there are 2 very important things to keep in mind:
1. Diacritic stripping for search should be optional and default **off** on some languages like [Vietnamese](https://meta.discourse.org/t/do-not-strip-diacritics-for-search-in-vietnamese/96050/2). In Vietnamese you never want to strip diacritics cause you end up getting nonsensical results.
2. The excerpts we show for search results should always show the diacritics, cause otherwise it just looks like there are bunch of silly spelling mistakes even in French. This is a hard problem cause we lean on PG to create the excerpts.
Given (1) I am reverting this feature for now, and given (2) this is something we will tackle for 2.2 release.
Reverted per:
[https://github.com/discourse/discourse/commit/9b7cab589ac15a034d1c0e700230c1b3f63f8ba0](https://github.com/discourse/discourse/commit/9b7cab589ac15a034d1c0e700230c1b3f63f8ba0)
---
### Author:  [@danekhollas](https://meta.discourse.org/u/danekhollas)
#### Post date: [31 augustus 2018 om 06:52 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/17 "2018-08-31T06:52:56Z")
Do I understand correctly that there is currently no option to keep this feature, right?
In which case I might want to keep older version on our Czech site for a while…
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [31 augustus 2018 om 07:11 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/18 "2018-08-31T07:11:12Z")
Yeah, hold tight, expect us to redo the feature in the next 3-4 weeks. But if there are security fixes I do recommend just giving up and upgrading. (there was one yesterday)
I am curious, in Czech don’t the excerpts looks a bit “kindergarten” level of spelling when we strip the diacritics out of the excerpts or is it something people do not notice?
---
### Author:  [@danekhollas](https://meta.discourse.org/u/danekhollas)
#### Post date: [31 augustus 2018 om 07:21 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/19 "2018-08-31T07:21:46Z")
> [@sam](#):
>
> in Czech don’t the excerpts looks a bit “kindergarten” level of spelling when we strip the diacritics out of the excerpts or is it something people do not notice?
There are many people who just do not bother with diacritics because it is harder to type (we have basically QWERTY keyboard with accented characters instead of numbers on the top row). So while it does not look great, incomplete search is MUCH bigger problem for us, especially because people are expecting this to just work (i.e. habits from internet search engines).
I expect the the same is true for most Slavic languages.
Thanks for working on this! Good to see that Discourse is spreading and i18n issues are becoming more important. 😛
EDIT: You can multiply the pain of typing accented characters x10 on mobile. 🙂
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [31 augustus 2018 om 07:28 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/20 "2018-08-31T07:28:57Z")
We can add the feature back behind a site setting very very shortly post our upcoming release in 1 - 1.5 weeks, I just want to avoid adding strings now which we end up needing to translate.
Then we can spend the extra time needed on fixing excerpts after we get the optional feature in.
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [31 augustus 2018 om 07:34 UTC](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/21 "2018-08-31T07:34:17Z")
I also agree with @danekhollas. Having the search work is a lot better than having accents in 🇫🇷 search results. Most people would not even notice…
[Next page](https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198.md?page=2)