# Topic "popular links" panel domain extraction doesn't handle country TLDs
**URL:** https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156
**Category:** Bug
**Created:** [30 Marzo 2017, 4:32am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156 "2017-03-30T04:32:15Z")
**Posts on this page:** 15
**Page:** 2
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [31 Marzo 2017, 11:44pm UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/25 "2017-03-31T23:44:06Z")
> [@tgxworld](#):
>
> the heuristic algorithm will only produce kumamoto.jp which is not what we want
I think it _is_ what we want. Just a hint where the link going – some domain in Japan. That’s enough, and correct.
The perfect is the enemy of the good, my friends.
(turns out Kumamoto is [a prefecture in Japan](https://en.wikipedia.org/wiki/Kumamoto_Prefecture), I guess analogous to a “state” in the “united states” so that’s plenty good. The US equivalent might be `tulsa.oklahoma.us` which means seeing a link as `oklahoma.us` is **absolutely correct!** )
---
### Author:  [@Mittineague](https://meta.discourse.org/u/Mittineague)
#### Post date: [31 Marzo 2017, 11:52pm UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/26 "2017-03-31T23:52:05Z")
IMHO as long as the link works I don’t have any strong preferences what shows as link text.
Would be easy and good enough to simply do “between // and the next first / when there is one”?
The “authority” portion.
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [31 Marzo 2017, 11:59pm UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/27 "2017-03-31T23:59:01Z")
The expectations of what is correct here is alittle confusing for me.
> community.seqta.com.au seqta.com.au
_This provides me with a hint that I’ll be heading to a domain registered in Australia at the seqta domain_
vs
> [www.city.amakusa.kumamoto.jp](http://www.city.amakusa.kumamoto.jp/) kumamoto.jp
_This provides me with a hint that I’ll be heading to a domain registered in the Kumamoto prefecture in Japan_
Should I be reverting
[https://github.com/discourse/discourse/commit/7690cc6ca50d48c86449dd9c7bf16301aa31ebc4](https://github.com/discourse/discourse/commit/7690cc6ca50d48c86449dd9c7bf16301aa31ebc4)
first? Otherwise, we’ll be showing the full domain for all links.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [1 Aprile 2017, 12:01am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/28 "2017-04-01T00:01:51Z")
I strongly urge you to implement the algorithm I suggested.
- it is simple
- it works for the common cases
- it is closer to the original intent of the way the feature was written
- it fixes the country code TLD problem that was originally brought up
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [1 Aprile 2017, 12:21am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/29 "2017-04-01T00:21:19Z")
> [@codinghorror](#):
>
> it is closer to the original intent of the way the feature was written
Can I confirm that the original intent is to “Extract the domain without the subdomains” even though the approach was too simplistic?
> [@codinghorror](#):
>
> it fixes the country code TLD problem that was originally brought up
It only fixes that particular case though. If we introduce that solution, we’ll end up showing the full domain for domains like `m.abv.bg` which is what you didn’t want.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [1 Aprile 2017, 12:25am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/30 "2017-04-01T00:25:14Z")
> [@tgxworld](#):
>
> Can I confirm that the original intent is to “Extract the domain without the subdomains” even though the approach was too simplistic?
Isn’t this patently obvious?
- `meta.discourse.org` → `discourse.org`
- `www.youtube.com` → `youtube.com`
> [@tgxworld](#):
>
> If we introduce that solution, we’ll end up showing the full domain for domains like m.abv.bg which is what you didn’t want.
That sounds correct to me, the goal is to shorten, to reduce redundant or over specific information, but if it’s already quite short to begin with, why bother?
The actual edge condition would be something like
`crazilylongthinghere.abv.bg`
and I am willing to bet that is _incredibly_ rare.
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [1 Aprile 2017, 12:31am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/31 "2017-04-01T00:31:32Z")
There are differences between the two goals here to me 😕
`the goal is to shorten, to reduce redundant or over specific information`
vs
`Extract the domain without the subdomains`
The first goal being subjective while the second has a definite answer.
Assuming that a C extension to this problem is already available, are we against pulling it in?
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [1 Aprile 2017, 12:32am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/32 "2017-04-01T00:32:12Z")
Yes, I am _ABSOLUTELY_ against it. Far too much complexity and overhead for something we can handle with a simple heuristic. Please implement the heuristic I suggested.
Yet again, **[perfect is the enemy of good](https://en.wikipedia.org/wiki/Perfect_is_the_enemy_of_good)**.
---
### Author:  [@elijah](https://meta.discourse.org/u/elijah)
#### Post date: [7 Aprile 2017, 8:40pm UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/37 "2017-04-07T20:40:39Z")
> [@codinghorror](#):
>
> Couldn’t we have a simple regex that allows a few 2 and 3 letter dotted phrases at the end?
[https://meta.discourse.org](https://meta.discourse.org)./
➡ Discourse Meta [meta.discourse.org](http://meta.discourse.org).
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [24 Luglio 2017, 12:42am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/38 "2017-07-24T00:42:07Z")
I really, really object to this 💩
So at minimum we _PLEASE_ need to suppress the part of the string beginning with `www.` from the display here.. this is so hideous. I do not care about TLD perfection, what I care about is hideous awful repetitive output appearing for users.
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [24 Luglio 2017, 1:33am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/39 "2017-07-24T01:33:50Z")
OK this is done in
[https://github.com/discourse/discourse/commit/6593f0355f80a13f0f62993c7d776654141b61b4](https://github.com/discourse/discourse/commit/6593f0355f80a13f0f62993c7d776654141b61b4)
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [24 Luglio 2017, 2:58am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/40 "2017-07-24T02:58:11Z")
Much better thank you @tgxworld
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [28 Novembre 2017, 11:46am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/41 "2017-11-28T11:46:05Z")
I built [mini\_suffix](https://github.com/discourse/mini_suffix) today which is a super simple wrapper around [libpsl](https://github.com/rockdaboot/libpsl/tree/libpsl-0.19.1). Currently, it only exposes `psl_registrable_domain` for now because that is all we need.
### Benchmarks
>
```plaintext
PublicSuffix.domain total allocated memsize: 6574255
PublicSuffix.domain total retained memsize: 1133266
MiniSuffix.domain total allocated memsize: 8000
MiniSuffix.domain total retained memsize: 0
Warming up --------------------------------------
PublicSuffix.domain 4.503k i/100ms
MiniSuffix.domain 77.107k i/100ms
Calculating -------------------------------------
PublicSuffix.domain 47.521k (± 2.2%) i/s - 238.659k in 5.024541s
MiniSuffix.domain 875.595k (± 3.8%) i/s - 4.395M in 5.027237s
Comparison:
MiniSuffix.domain: 875594.7 i/s
PublicSuffix.domain: 47521.2 i/s - 18.43x slower
```
As this introduces a dependency on `libpsl`, I’ll need to introduce `libpsl` to our base image first before I can add `mini_suffix` to Discourse and use it.
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [29 Novembre 2017, 1:55pm UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/42 "2017-11-29T13:55:41Z")
Yay feeling pretty happy about this, it is finally perfect 😁
[https://github.com/discourse/discourse/commit/8491c5fba55bd680e8478b2a4cbbe43c50adb32b](https://github.com/discourse/discourse/commit/8491c5fba55bd680e8478b2a4cbbe43c50adb32b)
[https://github.com/discourse/discourse/commit/9fbe1436b6fe5aa3c1dfb3b711cf3af1801fab20](https://github.com/discourse/discourse/commit/9fbe1436b6fe5aa3c1dfb3b711cf3af1801fab20)
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [4 Dicembre 2017, 2:00am UTC](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156/43 "2017-12-04T02:00:01Z")
This topic was automatically closed after 4 days. New replies are no longer allowed.
[Previous page](https://meta.discourse.org/t/topic-popular-links-panel-domain-extraction-doesnt-handle-country-tlds/60156.md?page=1)