Table of Contents
Just a second...

How conflation works

Conflation is implemented as a lock-free, wait-free algorithm and can be scaled to meet demands of large numbers of concurrently subscribed clients.

Conflation policies

Conflation policies configure how conflation takes place on topics. One or more conflation policies can be configured, each defining different conflation mechanisms. The policies can define the following behaviors:
  • How messages are matched
  • Whether replacement is done in place or by appending
  • How to merge the two messages

Conflation process

When conflation is enabled for a client, every time a new update is enqueued for the client it is considered for conflation.

  1. The Diffusion™ server checks whether a conflation policy is mapped to the topic that the update is published on.
  2. If a policy is mapped to the topic, the matching criteria of the policy is used to scan the updates for that topic that exist on the queue (from newest to oldest) for an existing update that matches the one being queued. If no match is found, the new update is queued at the end of the queue as normal.
    Note:

    Fast caches and lookup mechanisms are used to find policies and updates in the queue. The whole client queue is not scanned when looking for a match, only updates for the same topic.

    If default matching (by topic) is used, no comparison with the existing updates is required. This means that the conflation mechanism is as efficient as possible.

  3. If the matching criteria finds a matching update in the queue, the conflation policy defines the following behaviors:
    • Whether the update to queue as a result of conflation is the new update or an update produced by merging the content of the matching update and new update.
    • Whether the update to queue replaces the matching update or the matching update is removed and the new update added to the end of the queue.

Conflation occurs on a client-by-client basis in the multiplexer thread. Results of merge actions are cached to ensure that merges of identical message pairs are not repeated for different clients.