Does both the qpidd send data to Master Postgres

asurajpai
Participant V

Hi,

We have a 13 node installation of Edge.

There are 2 qpidd and 2 postgres (Master - Slave).

The question is does the both the qpidd nodes always send the

1) Queue analytics data to the master only.

Qpidd1 - Master Postgres

Qpidd2 - Master Postgres

Then Master slave replication will bring the both postgres databases in synch.

----OR----------

2) Queue analytics data as follows;.

Qpidd1 - Master Postgres

Qpidd2 - Slave Postgres

Then Master slave replication will bring the both postgres databases in synch.

What happens if one of the Qpidd servers are down. Will all the Queued data sent to the Master or slave?

1) Queue analytics data to the master only.

Qpidd1 - Master Postgres

Qpidd1 - Slave Postgres

Qpidd2___XXX_DOWN - XXXX

Then Master slave replication will bring the both postgres databases in synch.

----OR----------

2) Queue analytics data as follows;.

Qpidd1 - Master Postgres

Qpidd1 - Master Postgres only

Qpidd2___XXX_DOWN - XXX

Then Master slave replication will bring the both postgres databases in synch.

Please help us with the details for the above combination.

Solved Solved
0 5 883
1 ACCEPTED SOLUTION

Not applicable

Hi @asurajpai

The qpids always write to master PG. Slave pgs are readonly and and so cannot be written to. If the master PG is down, the messages remain inside the qpidd.

If qpidd is down, and it contains some messages, the messages are persisted to disk. Once the qpidd is back up, the ingestion from the same qpidd to the master postgres server starts again.

View solution in original post

5 REPLIES 5

Not applicable

Hi @asurajpai

The qpids always write to master PG. Slave pgs are readonly and and so cannot be written to. If the master PG is down, the messages remain inside the qpidd.

If qpidd is down, and it contains some messages, the messages are persisted to disk. Once the qpidd is back up, the ingestion from the same qpidd to the master postgres server starts again.

Hi @Sanjoy Bose, thanks for the quick reply.

Due to some reason one qpidd (Qpidd2) node got terminated.

Now with only one qpidd (Qpidd1) working,

1) Does all the qpidd (Qpidd1) data form the RMP1 and RMP2 is sent to the Master postgres server?

2) Can we add a new Qpidd server safely and redundancy HA achieved back?

3) Do we need to remove the UUID entries of Qpidd1 from the zookeeper? How to safely/ cleanly do it in case, if it is required.

Hi @Sanjoy Bose

Please help us with the above query.

@asurajpai

Adding few more points

1) The QPID servers queues have queue depth. So in case postgress server is down, then the queue depth will increase and if the queue is completely filled, then transaction data will be lost

2) In case of Different Data center architecture, if the request comes to DC1 - RMP, the RMP will send data to DC - 1 QPID, and then the DC- 1 QPID will send the data via queues to DC - 2 Master Postgress sever.

In this way, both the analytics data in DC1 and DC2 are in SYNC.

I hope this helps

Not applicable

Hi @asurajpai

Answers inline.

1) Does all the qpidd (Qpidd1) data form the RMP1 and RMP2 is sent to the Master postgres server?Yes

2) Can we add a new Qpidd server safely and redundancy HA achieved back?

Yes

3) Do we need to remove the UUID entries of Qpidd1 from the zookeeper? How to safely/ cleanly do it in case, if it is required.

It is better if you can remove.