Mongodb as a Discourse Backend


(Stephanie Sunshine) #1

Hi there would it be possible to use Mongodb as the backend to Discourse using mongo_fdw ( GitHub - citusdata/mongo_fdw: DEPRECATED, moved to ) extension for Postgres?

It looks like you still get all the power of Postgres but the features of Mongo as well. It says there are some limitations, but I don’t know enough about Discourse to know if they would be a problem. I know there are real reasons why the devs decided to choose and ultimately stick with Postgres, but as an administrator I would like the freedom that Mongo gives me for clustering options. Maybe we can have both worlds?


(Sam Saffron) #2

Can you explain what particular problem you are trying to solve here?


(Stephanie Sunshine) #3

Clustered databasing with true automatic failover. I already have a nice mongodb cluster running and would like to deploy Discourse to it. I completely understand from reading other posts that Postgres was chosen for specific reasons, but looking at any of the automatic failover options and their problems, I would rather use what I currently have running rather than setup another system that doesn’t seem to have a real easy clear way to fail over and fail back without an administrators intervention. I think for the same reasons that a MySQL port seems to be off the table that a Mongodb port would be even more off the table, but with that adapter, I believe that I could use Mongo without causing any issues, but i’m not familiar enough with Postgres or Discourses internals to answer that question, hence why I’m here asking it. :smile:

So is it possible that this might work, or are there things missing from this adapter that would prevent it from ever working?


(Sam Saffron) #4

Its actually a massive undertaking moving Discourse to support any other DB engines let alone an nosql backend. It is not even on our distant roadmap.

That said, you can have active/active clustering with pg.


(Stephanie Sunshine) #5

Did you even look at what I was suggesting? It acts as a transparent translator so your apps see Postgres in all it’s glory but the real storage is in Mongo.


(Sam Saffron) #6

Its remote tables, perf would be atrocious, indexing / joining a nightmare.

This kind of scheme could be useful as a backup system possibly.


(Jens Maier) #7

Besides…

Once we started doing ugly MongoDB joins manually in the Diaspora code, we knew it was the first sign of trouble. It was a sign that our data was actually relational, that there was value to that structure, and that we were going against the basic concept of a document data store.

Jepsen: MongoDB (quote from the MongoDB 2.2 documentation)

In some failover situations primaries will have accepted write operations that have not replicated to the secondaries after a failover occurs. This case is rare and typically occurs as a result of a network partition with replication lag. When this member (the former primary) rejoins the replica set and attempts to continue replication as a secondary the former primary must revert these operations or “roll back” these operations to maintain database consistency across the replica set.

So, here’s the $1 billion dollar question: what does it mean when MongoDB says that a write (aka insert) is complete? The answer is none of the above. MongoDB v2.0 will consider a write to be complete, done, finito as soon as it has been buffered in the outgoing socket buffer of the client host. Read that sentence over again.

These quotes all refer to previous versions of MongoDB (as early as 2.0).

If the v2.6 documentation is to be believed, MongoDB now has strict consistency when and if all writes are directed to the primary node of the replication set.

However, that’s nothing new among relational databases. PostgreSQL is happy to oblige and even MySQL had replication years ago (altho with similar weak durability guarantees as MongoDB).


(Robin Ward) #8

Please don’t try and do this.

There’s no way this is going to work properly. You would have a much easier time just installing a postgres cluster and configuring that than this approach.