Post creation race condition

(Vikhyat Korrapati) #1

On our forum we have “chat” threads which see a fairly rapid rate of post creation. We recently switched to running Unicorn and Rails 4, and have also started seeing these errors around the same time:

 ActiveRecord::RecordNotUnique: ERROR: duplicate key value violates unique constraint "index_posts_on_topic_id_and_post_number"
DETAIL: Key (topic_id, post_number)=(2068, 9104) already exists. 

It looks like this happens when two Unicorn processes try to create a post at the same time, but end up setting the same post number as a result. This is the relevant part of the stack trace from New Relic.

(Sam Saffron) #2

Do you mind posting that screenshot again, you uncovered another bug with out image cleaning job

(Vikhyat Korrapati) #3

(Vikhyat Korrapati) #4

Would computing the post_number inside the same SQL query help?

(Sam Saffron) #5

I think we can fix this race condition, how frequently does this happen?

(Vikhyat Korrapati) #6

NewRelic reports that error as having occurred 38 times in the last 24 hours. It is not a show-stopper or anything since it will normally go through on retrying, but it is definitely a minor annoyance.

(Sam Saffron) #7

Belated … but I put a fix in place for this now:

Note, I chose to go with a redis mutex cause we need to ensure our post_numbers are sequential and this help avoid being left with holes on failures.

(Sam Saffron) #8