# How to add html markup or meta tags for external search engine
**URL:** https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220
**Category:** Support
**Created:** [5월 2, 2015, 1:03오전 UTC](https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220 "2015-05-02T01:03:44Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@mrosas](https://meta.discourse.org/u/mrosas)
#### Post date: [5월 2, 2015, 1:03오전 UTC](https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220/1 "2015-05-02T01:03:44Z")
We are implementing a search engine for our main site using [Swiftype](http://swiftype.com). Out of the box indexing of our discourse site includes page navigation in the search description which is not desirable. ([see here](http://www.looker.com/search2#stq=symmetric+aggregates&stp=1)).
Swiftype offers two ways to customize the search description.
1. Content Exclusion/Inclusion: add attribute to markdown to designate what should be excluded/included in search index (eg: ``)
2. Meta tags: add custom meta tags to indicate what should be included in search index (eg: ``)
I’m not finding a way to do this without directly modifying files and it’s pretty clear to me that would break once we updated Discourse.
Would really welcome ideas on the best approach to resolving this.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [5월 2, 2015, 2:25오전 UTC](https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220/2 "2015-05-02T02:25:19Z")
I am not seeing the problem in the link you provided?
What webcrawler agent does this software use? Are we correctly identifying it as a crawler? What happens when you set your user-agent to the same user-agent that crawler uses, and browse your site?
I would start there.
---
### Author:  [@mrosas](https://meta.discourse.org/u/mrosas)
#### Post date: [5월 2, 2015, 4:31오전 UTC](https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220/3 "2015-05-02T04:31:37Z")
> [@codinghorror](#):
>
> What webcrawler agent does this software use? Are we correctly identifying it as a crawler?
The user-agent is called [Swiftbot](https://swiftype.com/swiftbot) (it’s a commercial search tool). Not sure how you would identify as a crawler?
> [@codinghorror](#):
>
> I am not seeing the problem in the link you provided?
The problem is that Swiftbot is picking up extra stuff (login, navigation, etc) in the search description. I think it’s a limitation with the Swiftype crawler – but their solution requires either 1) add markup to exclude navigation or 2) add a meta tag with the desired description. Neither option appears to be non-trivial in Discourse.
See the difference in search results below:
**[Google Search Result](https://www.google.com/?gws_rd=ssl#q=site:discourse.looker.com+symmetric+aggregates)**
[ ](https://dl.dropboxusercontent.com/s/7bn5ff3ogm9plkz/Screenshot%202015-05-01%2020.59.56.png?dl=0)
**[Swiftype Search Result](http://www.looker.com/search2#stq=legacy+features&stp=1)**
[ ](https://dl.dropboxusercontent.com/s/kgdx5iwx9s42zfb/Screenshot%202015-05-01%2020.59.32.png?dl=0)
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [5월 2, 2015, 10:19오전 UTC](https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220/4 "2015-05-02T10:19:48Z")
Ok, I added `Swiftbot` to the webcrawler detection regex, so if it using that `user-agent`, we will serve it the crawler version of the page. You’ll need to be on absolute latest for this change to take effect.
---
### Author:  [@mrosas](https://meta.discourse.org/u/mrosas)
#### Post date: [5월 4, 2015, 12:41오전 UTC](https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220/5 "2015-05-04T00:41:33Z")
Thanks for adding that in so quickly. After recrawling the page, I’m getting an improved result (it dropped the “Log in”).
I’d still like to customize more precisely – but will see if I can get swiftype to help out on that end.
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [10월 1, 2018, 1:13오후 UTC](https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220/6 "2018-10-01T13:13:11Z")
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [10월 2, 2018, 12:36오전 UTC](https://meta.discourse.org/t/how-to-add-html-markup-or-meta-tags-for-external-search-engine/28220/7 "2018-10-02T00:36:35Z")