Quick question, what does not being crash safe mean in practice? Will I have to recreate all indexes in pg and this will trigger a restart of Elastic and recreation of all indexes there?
That's a good question and a point of improvement for the documentation.
What I was trying to get across by saying that it's not "crash safe" is that the Elasticsearch index is not WAL logged.
As such, if Postgres crashes (bugs, hardware failure, kernel oops, etc) and goes into recovery mode and has to recover, from WAL, transactions that touched blocks belonging to tables with ZomboDB indexes, the ZomboDB indexes are now inconsistent. :(
In this regard, it's akin to Postgres' "hash" index type.
That said, this may be a "simple matter of programming" to resolve. I have some ideas, but it hasn't been a priority yet.
The recovery path is to REINDEX TABLE foo; Of course, the total time it takes to reindex is highly dependent on how much data you have, but indexing is really fast. My MBP can sustain 30k records/sec indexing documents with an average size of 7.5k against a single-node ES cluster. We've seen >100k recs/sec on high-performance hardware with very large ES clusters.
It's also worth noting that if the ES cluster disappears, any transaction that tries to touch the index will abort.
