Unrecognized error type (ActiveRecord::StatementInvalid: PG::ProgramLimitExceeded

Contribute Bug

AI-generated summary

A user encountered an error, Unrecognized error type (ActiveRecord::StatementInvalid: PG::ProgramLimitExceeded), when processing an incoming email. The error occurs because an incoming email error message exceeds the maximum index row size for PostgreSQL.

Julien_Palard provided the full error message and backtrace. pfaffman suggested running a vacuum and reindex, and referenced the PostgreSQL 13 update for potential solutions.

Falco explained that the error is due to an incoming email error message being too large for indexing, and proposed indexing on a safe substring of the first 100 characters of the error as a solution.

1

A user on our self-hosted discourse just, while sending an email, triggered:

Message

Unrecognized error type (ActiveRecord::StatementInvalid: PG::ProgramLimitExceeded: ERROR:  index row size 2904 exceeds btree version 4 maximum 2704 for index "index_incoming_emails_on_error"
DETAIL:  Index row references tuple (1,9) in relation "incoming_emails".
HINT:  Values larger than 1/3 of a buffer page cannot be indexed.
Consider a function index of an MD5 hash of the value, or use full text indexing.
) when processing incoming email

Backtrace:
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.3.3/lib/patches/db/pg.rb:69:in `exec_params'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.3.3/lib/patches/db/pg.rb:69:in `exec_params'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:671:in `block in exec_no_cache'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
  /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
  /var/www/discourse/...

Backtrace

/var/www/discourse/lib/email/processor.rb:91:in `handle_failure'
/var/www/discourse/lib/email/processor.rb:28:in `rescue in process!'
/var/www/discourse/lib/email/processor.rb:17:in `process!'
/var/www/discourse/lib/email/processor.rb:14:in `process!'
/var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:23:in `process_popmail'
/var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:43:in `block (2 levels) in poll_pop3'
/usr/local/lib/ruby/2.7.0/net/pop.rb:667:in `each'
/usr/local/lib/ruby/2.7.0/net/pop.rb:667:in `each_mail'
/var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:40:in `block in poll_pop3'
/usr/local/lib/ruby/2.7.0/net/pop.rb:532:in `start'

application_version 6e4af0e36f8bec32d77bdb3c0051088efa8069e0

1 Like
2

Are you on PG13

You might try a vacuum and reindex. I think if you search for those you should find some examples, but PostgreSQL 13 update might give you some hints.

3

That means that an incoming email error message is too large for indexing. We should index on a safe substring of the first 100 characters of the error.

3 Likes