Tuesday, July 7, 2009

Multicast Addressing Pitfalls

The IP multicast model has been described as follows: “You put packets in at one end, and the network conspires to deliver them to anyone who asks”. That sounds pretty simple indeed, but as you spend more time with applications that deliver multicast services you find that in many cases you will ‘put packets in at one end, look at the network protocol analyzer, and see nothing that you expect’. The truth of the matter is that getting multicast applications to work as expected requires deep understanding of the network topology and the underlying routing devices.

This post reviews the multicast addressing technology along with in detail illustrations of some of the pitfalls that are often encountered when deploying multicast applications.

In my next post you can find walkthrough of C# sample application that allows sending and receiving multicast data over the network.

How does it Work?

In multicast applications, the sender defines IP multicast group address (from 224.0.0.0 to 239.255.255.255) to which is send data packets. Receivers inform the network that they are interested in receiving data packets sent to a certain group by sending Internet Group Management Protocol (IGMP) package to the closest network node (router, IGMP querier). The node is in charge of maintaining multicast distribution trees such that data packets sent to a multicast group reach all receivers which have joined the group.

image

Why Use Multicast?

The great thing about multicast addressing is that in order to distribute a message to multiple receivers - the sender have to dispatch only one message over the wire, while it’s the network (e.g. intermediary routers) responsibility to copy and distribute the message to all the receivers in the group. In case there aren’t any receivers that had joined the group – the network drops the message.  The fact that the sender is unaware of the receivers greatly improves the application ability to scale to a large receiver population

The Pitfalls

Developing application that sends and receives multicast streams over the network is considered a fairly simple task. You can get pretty fast to the point where everything seems to work fine in the integration labs, were all the PCs communicate through a single network card and connected through few well known switches. The problems start when the application first meet the client network that is often spitted by VLANs and filled with all kinds of Cisco goodness. Unavoidably, you’ll find yourself steering at the network sniffer trying to figure out why you don’t see packages from group X even though the IGMP package was sent to the router, or why you do see packages from group Y - even though your application had never requested to join this particular group (No IGMP package was sent).

Where is my Multicast?

Multihomed Network

In case a host is connected to two or more networks or have two or more network cards installed and enabled – senders must explicitly state to which network interface (represented by the unicast IP address of the network card) they want to send the multicast traffic, and receivers must explicitly state from which network interface they want to receive multicast traffic. If they don’t – the operation system takes the liberty to choose the network interface, and the multicast traffic has 50% chance of reaching to the intended destination.

image

A common mistake that developers make is not binding the socket used for sending multicast traffic to the appropriate network card endpoint, or binding the socket used for receiving multicast to the address Any (0.0.0.0).

This mistake is so common because developers often assume that UDP unicast socket and UDP multicast sockets that are being used to send data can be implemented alike, and that sockets that are being used to receive multicast data can bind to IPAddress.Any (0.0.0.0). This is true when the host has single network card, but not true for host that lives in multihomed network.

Firewall

In case the machine firewall is turned on, and the firewall is NOT configured to relay transports from a certain multicast group (and corresponding UDP port) – your application will not receive messages from that group.

image

Inter VLAN

In case the sender and the receivers don’t sit on the same VLAN, the receivers will not receive messages from the sender.

image

Switch doesn’t Support IGMP Snooping

In case the sender is connected to a switch that support IGMP snooping (which means that it relay multicast messages only through ports which sent IGMP message), and the receiver is connected to another switch that doesn’t support IGMP snooping - the multicast relaying mechanism will "breaks down" in the absence of an mrouter port, thus the receiver will not receive the messages from the sender.

image

If you want a fix for this solution, you must have the switches somehow learn or know of an mrouter port. When the switches know their mrouter port, the right Switch (that doesn’t support IGMP snooping) relays out the IGMP report that it receives through its mrouter port. From the perspective of the left Switch, it received merely another IGMP report. The left switch adds that port into its IGMP snooping table and begins sending out multicast traffic on that port as well. At this point, the right Switch receive the multicast traffic, and the application works as expected.

Why am I getting this Multicast?

Broadcasting Multicast Traffic

Switches that cannot understand multicast addresses usually broadcast multicast traffic to all the members of a LAN. Machines that are connected to such switches will see multicast traffic from groups that they’ve never registered to.

image 

Port Mirroring

In case the machine is connected to a router port that is configured as ‘port mirroring’ – the machine will receive multicast streams form all the groups.

image

Port mirroring, also known as a roving analysis port, is a method of monitoring network traffic that forwards a copy of each incoming and outgoing packet from one port of a network switch to another port where the packet can be studied. A network administrator uses port mirroring as a diagnostic tool or debugging feature, especially when fending off an attack. It enables the administrator to keep close track of switch performance and alter it if necessary. Port mirroring can be managed locally or remotely.

Multicasting over WAN

Source-specific multicast (SSM) make multicasting packages over the WAN eligible by reducing the amount of multicast routing information that the network must maintain. With SSM receivers supply the sender’s source address to the routers as a part of joining the group (by setting the receiver socket option MCAST_JOIN_SOURCE_GROUP).

A great presentation (ppt) that discusses multicast addressing can be downloaded from here.

Links

http://www.netcraftsmen.net/welcher/papers/multicast01.html

http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a008059a9df.shtml

6 comments:

  1. This post is quite advanced and I doubt the general internet audience is smart enough to understand even half what you said. On the other hand, I found it to be right up my alley and it clearly describes a lot of the technology I've been playing with lately. I appreciate your post. IGMP is not for the feint of heart.

    ReplyDelete
  2. This post is a very good guideline for system administrators. Awesome post.
    But I would like to suggest that you add a bit more regarding multicasting on VLANs.

    ReplyDelete
  3. Great post!
    You might add to section "Why am I getting this Multicast?" the case of multiple multicast IP addresses mapping to the same multicast MAC address.
    Regarding "Switch doesn’t Support IGMP Snooping" I would expect such switches to flood IGMP control and multicast data on the VLAN.

    ReplyDelete
  4. Thanks for the comment, can you please provide more details on your suggestions?

    ReplyDelete
  5. Refer to RFC1112 (section 6.4)
    For example: 225.1.1.1 and 226.1.1.1 are mapped to the same MAC address: 01:00:5E:01:01:01
    If a host wishes to join 225.1.1.1 the network card might capture also traffic to 226.1.1.1 (The IP stack should filter these frames before reaching the application).

    ReplyDelete
  6. thank you about this important post
    thank you

    ReplyDelete