Thursday, 8 August 2013

Notifying postgres changes to java application

Notifying postgres changes to java application

Problem
I'm building a postgres database for a few hundred thousand products. I
will set-up an index (Solr or maybe ElasticSearch) to improve query times
for complex search queries.
The point now is how to let the index synchronized with the database?
In the past I had a kind of application that polled the database
periodically to check for updates that should be done, but I would have an
outdated index state time (from the database update to the index update
pull).
I would prefer a solution in which the database would notify my
application (java application) that something has been changed within the
database, and at that point the application will decide if the index needs
to be updated or not. To be more accurate, I would build a kind of
producer and consumer structure in wish the replica will receive
notifications from postgres that something changed, if this is pertinent
to the data indexed it is stored in a stack of updates-to-do. The consumer
would consume this stack and build the documents to be stored into the
index.
Possible Solutions
One solution would be to write a kind of replica end-point in which the
application would behave as a postgres instance that is being used to
replicate the data from the original database. Do someone have some
experience with this approach?
Which other solution do I have for this problem?

No comments:

Post a Comment