# Searching Chinese terms in middle of sentence

**URL:** <https://meta.discourse.org/t/searching-chinese-terms-in-middle-of-sentence/35440>\
**Category:** Feature\
**Created:** [2015年十一月10日 23:27 UTC](https://meta.discourse.org/t/searching-chinese-terms-in-middle-of-sentence/35440 "2015-11-10T23:27:41Z")\
**Posts on this page:** 5\
**Page:** 2

<div class="post-metadata">

**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\
**Post date:** [2016年十月6日 09:57 UTC](https://meta.discourse.org/t/searching-chinese-terms-in-middle-of-sentence/35440/21 "2016-10-06T09:57:54Z")

</div>

This is how stuff is being split in the tokenizer, does it look correct?

```plaintext
source: 我非常喜歡台灣
split: 我 | 非常 | 喜 | 歡 | 台 | 灣 |
source: 我們的總統目前不敢說台灣是個獨立的國家
split: 我 | 們 | 的 | 總 | 統 | 目前 | 不敢 | 說 | 台 | 灣 | 是 | 個 | 獨 | 立 | 的 | 國 | 家 |
source: 台灣主要的城市包括台北、台中、台南、高雄、台東和花蓮
split: 台 | 灣 | 主要 | 的 | 城市 | 包括 | 台北 | 、 | 台中 | 、 | 台南 | 、 | 高雄 | 、 | 台 | 東 | 和 | 花 | 蓮 |
source: 在我認識的人中，大部分不知道台灣在哪裡或以為是泰國
split: 在 | 我 | 認 | 識 | 的 | 人中 | ， | 大部分 | 不知道 | 台 | 灣 | 在哪 | 裡 | 或以 | 為 | 是 | 泰 | 國 |

```

---

<div class="post-metadata">

**Author:** ![tempogain](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tempogain/32/134055_2.png) [@tempogain](https://meta.discourse.org/u/tempogain)\
**Post date:** [2016年十月6日 14:29 UTC](https://meta.discourse.org/t/searching-chinese-terms-in-middle-of-sentence/35440/22 "2016-10-06T14:29:23Z")

</div>

Everything looks logical, except that there are a couple of oddities in the last one. 人中 doesn’t seem like an important enough term to preserve, and I would expect 哪裡 and 以為 to be preserved as units as opposed to what is shown. I’m not sure what the parameters are though, in other words what it should be doing.

---

<div class="post-metadata">

**Author:** ![riking](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/riking/32/170938_2.png) [@riking](https://meta.discourse.org/u/riking)\
**Post date:** [2016年十月8日 04:58 UTC](https://meta.discourse.org/t/searching-chinese-terms-in-middle-of-sentence/35440/23 "2016-10-08T04:58:18Z")

</div>

> [@tempogain](#):
>
> expect 哪裡 and 以為 to be preserved as units as opposed to what is shown

To help other English-speakers like me save their eyes, here’s what is shown:

> [@sam](#):
>
> `在哪 | 裡`

i.e. the first character is attached to the character before it instead of the one after it

---

<div class="post-metadata">

**Author:** ![tempogain](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tempogain/32/134055_2.png) [@tempogain](https://meta.discourse.org/u/tempogain)\
**Post date:** [2016年十月8日 13:37 UTC](https://meta.discourse.org/t/searching-chinese-terms-in-middle-of-sentence/35440/24 "2016-10-08T13:37:22Z")

</div>

> [@sam](#):
>
> 或以 | 為

Yes, and this pair as well 或以 | 為, that 以為 seems like a much more common term.

---

<div class="post-metadata">

**Author:** ![chapoi](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/chapoi/32/537252_2.png) [@chapoi](https://meta.discourse.org/u/chapoi)\
**Post date:** [2025年十二月4日 11:37 UTC](https://meta.discourse.org/t/searching-chinese-terms-in-middle-of-sentence/35440/25 "2025-12-04T11:37:06Z")

</div>



[上一頁](https://meta.discourse.org/t/searching-chinese-terms-in-middle-of-sentence/35440.md?page=1)