Thank you ! That helped me understand.
I was trying to implement the same plugin system and my issue was that I was activating them later outside application.rb, in an initializer for example. Then the lib (and probably other folders) of the plugins are not added to the load path.