jump to navigation

Mirror, mirror… January 15, 2013

Posted by jdstrand in security, ubuntu.
trackback

Recently I wanted to mirror traffic from one interface to another. There is a cool utility from Martin Roesch (of Snort fame) called daemonlogger. It’s use is pretty simple (note I created the ‘daemonlogger user and group on my Ubuntu system since I wanted it to drop privileges after starting):
$ sudo /usr/bin/daemonlogger -u daemonlogger \
  -g daemonlogger -i eth0 -o eth1

Sure enough, examing tcpcump output, anything coming in on eth0 was mirrored to eth1. I then used a crossover cable and used tcpdump on the other system and it all worked as advertised. This could be very useful for an IDS.

Then I thought how cool it would be to do this for multiple interfaces, but then I quickly realized that would be confusing for whatever was trying to interpret that traffic. I read that someone used a combination of daemonlogger and vtun for a remote snort monitor to work in EC2, which got me playing….

In my test environment, I again used a crossover cable to connect two systems (let’s call them the ‘server’ and the ‘monitor’). The server system runs vtund. The monitor system then connects as a vtun client. When this happens, tap devices are created on the server and the monitor. The vtund server is setup to start daemonlogger on connect to mirror the traffic from eth0 to the tap0 device. The monitor system can then listen on the tap device and get all the traffic. I then created a second vtun tunnel and mirrored from eth1 to tap1. The monitor could monitor on both tap devices. Neat!

To do accomplish this, I used this /etc/vtund.conf on the server:options {
 type stand;
 port 5000;
 syslog daemon;
 ifconfig /sbin/ifconfig;
}
default {
  type ether;
  proto udp;
  compress no;
  encrypt no;
  multi yes;
  keepalive yes;
  srcaddr {
   iface eth2;
   addr 10.0.3.1;
  };
}
internal {
  password pass0;
  device tap0;
  up {
   ifconfig "%d up";
   program /usr/bin/daemonlogger "-u daemonlogger -g daemonlogger -i eth0 -o %d";
  };
  down {
   ifconfig "%d down";
   program /usr/bin/pkill "-f -x '/usr/bin/daemonlogger -u daemonlogger -g daemonlogger -i eth0 -o %d'";
  };
}
external {
  password pass1;
  device tap1;
  up {
   ifconfig "%d up";
   program /usr/bin/daemonlogger "-u daemonlogger -g daemonlogger -i eth1 -o %d";
  };
  down {
   ifconfig "%d down";
   program /usr/bin/pkill "-f -x '/usr/bin/daemonlogger -u daemonlogger -g daemonlogger -i eth1 -o %d'";
  };
}

Ther server’s /etc/default/vtun has:RUN_SERVER=yes
SERVER_ARGS="-P 5000"

And here is the client /etc/vtund.conf:options {
  port 5000;
  ifconfig /sbin/ifconfig;
}
default {
  type ether;
  proto udp;
  compress no;
  encrypt no;
  persist yes;
}
internal {
  device tap0;
  password pass0;
  up {
   ifconfig "%d up";
  };
  down {
   ifconfig "%d down";
  };
}
external {
  device tap1;
  password pass1;
  up {
   ifconfig "%d up";
  };
  down {
   ifconfig "%d down";
  };
}

The monitor system’s /etc/default/vtun looks like this:CLIENT0_NAME=internal
CLIENT0_HOST=10.0.2.1
CLIENT1_NAME=external
CLIENT1_HOST=10.0.2.1

With the above, I was able to start suricata (a snort-compatible rewrite that is multithreaded) to listen to both tap0 and tap1 on the monitoring system and see all the traffic. Very cool. :)

Now, this isn’t optimized for anything– sending 2 interfaces’ worth of traffic through one interface is likely going to result in droppoed packets to the monitor if the links are busy. Could maybe use compression to help there. Also, the server really needs to have an extra interface for vtun traffic that isn’t mirrored, otherwise you end up mirroring the vtun traffic itself (in my test environment, eth2 on the server used 10.0.2.1 and the montior used 10.0.2.2). Finally, vtun doesn’t offer any meaningful line encryption so if you are doing this for anything resembling production, you’re going to want to use a dedicated network segment between the server and the monitor (a crossover cable worked fine in my test environment).

Enjoy!

Comments»

1. Matt Mossholder - January 15, 2013

Why not just use VLANs? It should be much more straight forward, and requires less configuration….

jdstrand - January 15, 2013

VLANs solve all kinds of problems and are really useful but a) I wanted to know how the snort guys used vtun to solve their EC2 problem and b) VLANs require a switch and NICs that support 802.1q. A switch of this type probably also supports port mirroring which could also be used instead. I was curious about how to achieve switch port mirroring for arbitrary interfaces using network hardware that didn’t support it; I came up with what I deemed a cool hack, and shared it. YMMV :)

2. Joel Wirāmu Pauling - January 16, 2013

I would have throught you could just use bond mode 3 with the in kernel linux driver to accomplish the same thing. Bond mode 3 is the broadcast mode and traffic is replicated to either interface.

I have not tried to include an interface as part of two bonds before but I can see no reason why it shouldn’t work with a bit of fiddling. This has the advantages ( and disadvantages) of not being userspace.

If you are wanting to not drop packets have a look at the ring_buffer pcap implementation.

http://www.ntop.org/products/pf_ring/

@aenertia

jdstrand - January 16, 2013

Interesting, that’s definitely worth investigating.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: