About Friendly urls and Non-Latin characters

Product Support

AI-generated summary

About Friendly URLs and Non-Latin Characters

The discussion revolves around modifying Discourse’s URL generation to support non-Latin characters. the220hq proposed changing the encoded_generator function in lib/slug.rb to use I18n.transliterate for better support of non-Latin characters. The modified function would generate friendly URLs without escape strings.

lhc_fl pointed out that some apps may not support non-Latin text links, implying that compatibility should be considered.

the220hq further explored customization options for transliteration using locale settings in locale.yml files.

However, the220hq encountered issues with their code changes being reverted after running ./launcher destroy app && ./launcher start app.

After experimenting, the220hq discovered the following best practices:

  • Avoid modifying ascii_generator as it can cause issues with template/theme/css compilation.
  • Use encoded_generator and hard-code locale settings if necessary, while keeping the system default locale setting (English).
  • Add customizations to slug.rb in a .yml file to persist changes between app rebuilds.
1

I tried encoded but actually the URL can be seen good on address of internet browser. However, when copy or send it, the real URL is a escape string and not friendly anymore.

I check the lib/slug.rb and suggest to change encoded_generator function likes this:

  def self.encoded_generator(string, downcase: true)
    # This generator will sanitize almost all special characters,
    # including reserved characters from RFC3986.
    # See also URI::REGEXP::PATTERN.
    string = I18n.transliterate(string, :locale => SiteSetting.default_locale)

    string = string.strip.gsub(/\s+/, "-").gsub(CHAR_FILTER_REGEXP, "")

    string = string.downcase if downcase

    string
    #CGI.escape(string)
  end

I tried with Viet Nam locale and got the desired results. Hope it works with another language.

2

I guess this is a compatibility consideration. Some APPs only support Latin text links and will directly cut off links in non-Latin text.

3

I’m not a ruby code writer, but I read the document and find this:

Setting a Hash in <locale>.yml:

i18n:
  transliterate:
    rule:
      ü: "ue"
      ö: "oe"

So this is the way for customize transliterate in setting, at the end of function, we got Latin text (friendly url) only. I tried with my native language, and no need to define custom hash setting.

4

Is there any way to keep the code change? Because if I did:

./launcher destroy app && ./launcher start app

The file ./lib/slug.rb turns back to original code. I’m sorry because I’m new in ruby on rails environment.

5

I’m so sorry because my annoycing!

I decide to change the ascii instead the encoded function. I don’t know why, but I got the different results when use encoded.

What is my wrong here? Thank you very much!

  def self.ascii_generator(string)
    #I18n.with_locale(SiteSetting.default_locale) { string.tr("'", "").parameterize }
    string = I18n.transliterate(string, :locale => SiteSetting.default_locale)
    string = string.strip.gsub(/\s+/, "-").gsub(CHAR_FILTER_REGEXP, "")
    string
  end
6

After play around. I found that:

  • Never change ascii_generator. It make discourse compile template/theme/css wrong.
  • Only use encoded_generator and maybe hard-code to your locale, keep the system default locale setting (English).
  • Add changed code of slug.rb in .yml to keep customize slug.rb for every time rebuild app.

Nice day!