Topic with Japanese in URL doesn't redirect if URL doesn't perfectly match

[seanblue]

In community.wanikani.com, opening any links to topics with Japanese in the fully qualified URL stopped working when opening them in a new tab or directly copying and pasting the link. Clicking the link to navigate within the same tab still works.

For example, opening this link in a new tab should navigate to

キノの旅 Home Thread (Intermediate Book Club) - Book Clubs - WaniKani Community

But instead it tries to navigate to

キノの旅 Home Thread (Intermediate Book Club) - Book Clubs - WaniKani Community

which fails to load.

If the link happens to match exactly, it works fine. But of course with topic renames this is often not the case.

I also tried to reproduce on try.discourse.org, but on that install Japanese characters never get added to the URL even when included in the topic title. I’m not sure why that’s the case, but without that happening I can’t demonstrate the bug there.

[Falco]

Both are links to topic 34890. Both load fine for me in Firefox. What is the problem?

[seanblue]

sigh It looks like it might be yet another Chrome bug. It works fine for me in both Firefox and Edge. Weirdly it works the first time in an incognito window, but then fails the second time. Same thing after clearing site cache/cookies and restarting my computer.

Chrome says the error is too many redirects.

---

Would you mind checking in Chrome to confirm if it’s an issue there in general and not just a problem for me? Just be sure to try to open the page multiple times since the first one seems to work fine. I appreciate the help!

[Falco]

I can repro it on Chrome mobile.

[seanblue]

Thanks. I guess I’ll report it to Google.

[pfaffman (Jay Pfaffman)]

On my android phone on chrome the second link redirects indefinitely.

[seanblue]

Still with Chrome? Just want to be sure before reporting it to them. I assume nothing changed related to this recently on Discourse? (Regardless, this would probably still be a Chrome issue since it just happens there, even if something did change on Discourse.)

[Falco]

@seanblue

Thanks. I guess I’ll report it to Google.

Wait a bit, this may be Discourse. Even a service worker bug.

[seanblue]

Okay, thanks for the update.

[seanblue]

Any update on this?

[sam (Sam Saffron)]

This is going to take a while to sort out, it is assigned out so it will not fall between cracks.

[seanblue]

I don’t expect to find any kind of solution or workaround. We just have to wait for it to be fixed.

[seanblue]

Are you still seeing this issue? It looks like maybe it’s fixed, unless I’m doing something differently this time.

[seanblue]

@seanblue

It looks like maybe it’s fixed

Well scratch that. It started happening again today. I have no idea how it got fixed for a bit.

---

@sam

This is going to take a while to sort out

Since this might take a while, I’d be open to workarounds for now. I mentioned in the OP about how on try (and I checked here on meta too) the Japanese characters never get added to the URL, effectively circumventing this issue. Is that a site or category setting that I can talk to my site admin about? Any other suggestions for a workaround besides that?

[YaserAlnajjar]

When I enter URL for an Arabic title into the browser like:

https://forums.coretabs.net/t/2456

I will get into an infinite redirections (and the generated link is not right I guess this is related to the encoding)

It should redirect instead into:

https://forums.coretabs.net/t/ماذا-يجب-ان-نتعلم-في-javascript-؟/2456

Why don’t I share links with their titles?

Cuz of the bad Arabic support in Twitter and Facebook:

• This bug didn’t exist before latest updates (last time I tried sharing a link was around two weeks ago, and it was perfectly fine).

[Falco]

I have dig into our codebase and looks like the error is kinda simple, but I would like to verify my assumptions.

We have a site setting named slug_generation_method which must be changed from the default ascii value to encoded in order to trigger this bug. When you change this site setting, we clear all slugs and generate those again.

What I don’t understand is why when the site setting is set to “encoded” we generate a slug like this:

[3] pry(main)> SiteSetting.slug_generation_method
=> "encoded"
[4] pry(main)> Slug.for(t.slug)
=> "キノの旅-home-thread-intermediate-book-club"

where I expected that “encoded” means something like

[5] pry(main)> CGI.escape(Slug.for(t.slug))
=> "%E3%82%AD%E3%83%8E%E3%81%AE%E6%97%85-home-thread-intermediate-book-club"

This appears to come from

https://github.com/discourse/discourse/pull/3370

The raw slug from the table is returned in the Location header 301 response when a topic slug doesn’t match, and IMO we should return a valid URL in there.

[sam (Sam Saffron)]

Yeah we should clean up the slug generation method for encoded so it relies less on browser magic.

[seanblue]

So are you saying the URL itself would show the encoded version? Or just that the redirect would internally handle it by using the encoded version? Either way, getting this to “just work” and not rely on browser quirks would be great.

[Arta_S (Arta)]

Hello,

Is this case resolved?
Because I am still facing this issue as I posted in the topic I initiated regarding this matter.

[Falco]

@Arta_S

이 문제는 해결되었나요?

아니요, 여기 Contribute > Bug 카테고리에서 해당 토픽이 아직 열려 있는 것으로 보아 해결되지 않았습니다.

@sam

네, 브라우저의 마법 같은 동작에 덜 의존하도록 인코딩된 슬러그 생성 메서드를 정리해야 합니다.

@sam 오늘 이 문제를 다시 살펴봤는데, 두 가지 접근 방법이 있습니다:

1. 슬러그 생성 설정이 **인코딩(encoded)**으로 되어 있을 때, 슬러그 컬럼에 실제 인코딩된 슬러그를 저장합니다. 인코딩된 슬러그를 사용하는 경우 현재 모든 슬러그를 비워두는 마이그레이션을 수행하여 시간이 지나면서 슬러그가 올바르게 재생성되도록 합니다.

2. 현재 UTF-8 슬러그를 유지하고, 301 리다이렉트 헤더로 전송할 때 실시간으로 패치합니다.

개인적으로 1번이 "더 정확"하다고 생각하며, 이는 클라이언트에 원본 슬러그를 전달하는 것을 더 어렵게 만듭니다. 그러나 슬러그 생성기만 패치하는 것만으로는 충분하지 않았습니다. 브라우저는 301 리다이렉트에서 인코딩된 URL을 받지만 다음 요청 시 이를 디코딩하기 때문에, 우리의 슬러그 비교가 실패하고 다시 리다이렉트가 발생합니다. 즉, 토픽 컨트롤러의 슬러그 비교 메서드도 패치해야 하고, 다른 곳들도 마찬가지일 것입니다.

이 방향으로 계속 진행할까요?