In file: crawler_detection.rb
def self.crawler?(user_agent)
!/Googlebot|Mediapartners|AdsBot|curl|HTTrack|Twitterbot|facebookexternalhit|bingbot|Baiduspider|ia_archiver|Wayback Save Page|360Spider|Swiftbot|YandexBot/.match(user_agent).nil?
end
Mozilla Firefox 34.0 → about:config → general.useragent.override
I add those bots that are not on the list. In fact, a lot of search engines:StackRambler, Teoma and etc.
As a result, I see a normal page
Am I doing something wrong?