List Brokers

To provide massive scalability of a large messaging fabric you typically want to allow many brokers to be connected together into a network so that you can have as many clients as you wish all logically connected together - and running as many message brokers as you need based on your number of clients and network topology.

If you are using client/server or hub/spoke style topology then the broker you connect to becomes a single point of failure which is another reason for wanting a network (or cluster) of brokers so that you can survive failure of any particular broker, machine or subnet.

From 1.1 onwards of ActiveMQ supports networks of brokers which allows us to support distributed queues and topics across a network of brokers.

This allows a client to connect to any broker in the network - and fail over to another broker if there is a failure - providing from the clients perspective a HA cluster of brokers.

N.B. By default a network connection is one way only - the broker that establishes the connection passes messages to the broker(s) its connected to. From version 5.x of ActiveMQ, a network connection can be optionally enabled to be duplex, which can be useful for hub and spoke architectures, where the hub is behind a firewall etc.

The easiest way to configure a network of brokers is via the Xml Configuration. There are two main ways to create a network of brokers

Here is an example of using the fixed list of URIs

With static: discovery you can hard code the list of broker URLs. A network connector will be created for each one.

Networks of brokers do reliable store and forward of messages. If the source is durable, persistent messages on a queue or a durable topic subscription, a network will retain the durability guarantee. However networks cannot add durability when the source is non durable. Non durable topic subscriptions and temporary destinations (both queues and topics) are non durable by definition. When non durable sources are networked, in the event of a failure, inflight messages can be lost.

ActiveMQ relies on information about active consumers (subscriptions) to pass messages around the network. A broker interprets a subscription from a remote (networked) broker in the same way as it would a subscription from a local client connection and routes a copy of any relevant message to each subscription. With Topic subscriptions and with more than one remote subscription, a remote broker would interpret each message copy as valid, so when it in turns routes the messages to its own local connections, duplicates would occur. Hence default conduit behavior consolidates all matching subscription information to prevent duplicates flowing around the network. With this default behaviour, N subscriptions on a remote broker look like a single subscription to the networked broker.

However - duplicate subscriptions is a useful feature to exploit if you are only using Queues. As the load balancing algorithm will attempt to share message load evenly, consumers across a network will equally share the message load only if the flag conduitSubscriptions=false. Heres an example. Suppose you have two brokers, A and B, that are connected to one another via a forwarding bridge. Connected to broker A, you have a consumer that subscribes to a queue called Q.TEST. Connected to broker B, you have two consumers that also subscribe to Q.TEST. All consumers have equal priority. Then you start a producer on broker A that writes 30 messages to Q.TEST. By default, (conduitSubscriptions=true), 15 messages will be sent to the consumer on broker A and the resulting 15 messages will be sent to the two consumers on broker B. The message load has not been equally spread across all three consumers because, by default, broker A views the two subscriptions on broker B as one. If you had set conduitSubscriptions to "false", then each of the three consumers would have been given 10 messages.

By default a network bridge forwards messages on demand in one direction over a single connection. When dupex=true, the same connection is used for a network bridge in the opposite directions, resulting in a by directional bridge. The network bridge configuration is propagated to the other broker so the duplex bridge is an exact replica or the original. Given two brokers, broker A and broker B, a duplex bridge on A to B is the same as a default bridge on A to B and a default bridge on B to A. Note, if you want to configure more than one duplex network bridge between two brokers, to increase throughput or to partition topics and queues, you must provide unique names for each: eg:

This is all fine and well in small networks and environments whit small number of destinations and consumers. But as things starts to grow a default model (listen to everything, share everything) wont scale well. Thats why there are many ways you can use to filter destinations that will be shared between brokers.

Lets start with dynamically configured networks.