How conflation works
Conflation is implemented as a lock-free, wait-free algorithm and can be scaled to meet demands of large numbers of concurrently subscribed clients.
Conflation policies
- How messages are matched
- Whether replacement is done in place or by appending
- How to merge the two messages
Conflation process
When conflation is enabled for a client, every time a new update is enqueued for the client it is considered for conflation.
- The Diffusion™ server checks whether a conflation policy is mapped to the topic that the update is published on.
- If a policy is mapped to the topic, the matching criteria of the policy is
used to scan the updates for that topic that exist on the queue (from newest
to oldest) for an existing update that matches the one being queued. If no
match is found, the new update is queued at the end of the queue as normal.
Note:
Fast caches and lookup mechanisms are used to find policies and updates in the queue. The whole client queue is not scanned when looking for a match, only updates for the same topic.
If default matching (by topic) is used, no comparison with the existing updates is required. This means that the conflation mechanism is as efficient as possible.
- If the matching criteria finds a matching update in the queue, the
conflation policy defines the following behaviors:
- Whether the update to queue as a result of conflation is the new update or an update produced by merging the content of the matching update and new update.
- Whether the update to queue replaces the matching update or the matching update is removed and the new update added to the end of the queue.
Conflation occurs on a client-by-client basis in the multiplexer thread. Results of merge actions are cached to ensure that merges of identical message pairs are not repeated for different clients.