Trouble with `discourse remap` remapping `topic_links url`

Bug
1

I’m trying to

discourse remap community.x.com  staging.community.x.com

as described here (edit: Hmm. Maybe I should have made this a post over there?) and am getting this error:

Remapping topic_links url
Error: ERROR:  value too long for type character varying(500)
The remap has only been partially applied due to the error above. Please re-run the script again.
2

Can you share the value for the column url in the table topic_links where the length is that big? Is even a valid url?

psql
\x
SELECT * FROM topic_links WHERE length(url) > 450;
3 Likes
3

(Thanks for the SELECT!)

Looks like it’s a long http://site/new-topic?body=a%20bunch%20of%20stuff that’s to blame. Looks like it go truncated to 500 chars and so when we try to replace HOSTNAME with staging.HOSTNAME we’re out of luck.

P.S. If someone else wants to do that select, first do this:

./launcher enter app
su discourse
3 Likes
4

We could move that column to character varying(2000) (or even text and keep the validation on Ruby) but I’m not sure if it’s worth the hassle of a migration.

Since the remap is smart enough to just skip this line, I guess all is ok in the end?

1 Like
5

Unless topic_links is the last table it’s handling, it looks like it’s quitting rather than just skipping.

6

Hey @Falco, would truncating the URL in line 120 fix this?

change it to

    uri = UrlHelper.relaxed_parse(u.url[0..self.max_url_length])

https://github.com/discourse/discourse/blob/master/app/models/topic_link.rb#L120

EDIT: well, on my first attempt, this yielded

Remapping topic_links url
Error: ERROR:  duplicate key value violates unique constraint "unique_post_links"
DETAIL:  Key (topic_id, post_id, url)=(19484, 61049, https://staging.community.rstudio.com/t/birds-of-a-feather-bof-at-rstudio-conf-2019l/19328) already exists.
The remap has only been partially applied due to the error above. Please re-run the script again.

So short of a database migration to make the URL longer. . . maybe add another rescue on duplicate URLs and let them pass? So also add a

    rescue PG::UniqueViolation

that does nothing?

7

Since this is the very first time someone hits this, I would go with a simple approach of fixing the few affected rows.

Something like:

UPDATE topic_links
SET url = substring(url FOR 450)
WHERE length(url) > 450;
4 Likes
8

I’ll give that a shot. My solution was to add a next if parsed.length > 500 to self.extract_from(post) intopic_link.rb`. It seems like that would fix it for everyone. I think that it worked on the site where I am doing the remapping, but then I’m getting

Error: ERROR:  duplicate key value violates unique constraint "unique_post_links"
DETAIL:  Key (topic_id, post_id, url)=(19484, 61049, https://staging.community.rstudio.com/t/birds-of-a-feather-bof-at-rstudio-conf-2019l/19328) already exists.
The remap has only been partially applied due to the error above. Please re-run the script again.

More on this over at Migrate_to_s3 for Digital Ocean Spaces woes. I hit the same topic_link problem doing the S3 migrate on another site.