Back in the day, I used an open source program called NistNET to emulate a WAN for my company’s network test lab on a linux machine.\u00a0 I was able to solve a multitude of issues and test our applications in a WAN environment with this product.\u00a0 Unfortunately, NistNET is no longer maintained, and until recently I had no open source tool for emulating a network in my arsenal.\u00a0 The other day, while playing Call of Duty 2 with some friends on my dedicated linux server, I decided that I was tired of having an unfair advantage since my latency to the server was 1 ms, and they all had 50-70 ms or more, so I went on a search for something I could use to add delay to my connection to the server (one of my buddies says I am too honest).\u00a0 After some searching, I came upon netem, which to my surprise is and has been part of the linux kernel for some time.\u00a0 I know, some of you linux guys and gals out there are saying “tell me something I don’t know”\u00a0 but, ashamedly, I didn’t know about this one perhaps because I don’t do much of that kind of work any more.<\/p>\n
Anyway, using Netem, I could do exactly what I wanted.\u00a0 I can add enough delay to my client to game server traffic (actually, its server to client, which I will explain later) to make it seem like I have a lot more network between the server and me than I actually do.\u00a0 Whether or not this decreases my advantage in the game is yet to be seen.<\/p>\n
Anyway, several “effects” are present on most wide area networks today.\u00a0 A common effect, latency, can have a drastic effect on the way network communications protocols behave.\u00a0 Latency is also one of the key issues with playing on line games, especially those that require fast reaction to on screen events.\u00a0 High latency creates what gamers refer to as lag.<\/p>\n
The Netem function of linux provides the capability to modify the parameters of egress traffic (ie, traffic exiting the machine and destined for another point on on the network).\u00a0 With netem, its possible to create artificial delay, thus creating latency.\u00a0 Other possibilities are rate limiting (controlling the amount of bandwidth traffic can use), causing packet loss, and jitter.\u00a0 Packet loss can result in very poor performance with TCP applications.\u00a0 Jitter, also known as variable delay, is bad for real time streaming applications such as voice over IP.<\/p>\n
Anyway, you could probably see why this kind of stuff would be important to a network engineer, especially in a lab environment.<\/p>\n
So – on to how to use netem.\u00a0 Netem is controlled by the tc command, which is part of the iproute2 package and is included in with most linux distributions.<\/p>\n
Using the tc command, we can easily tell a linux host to delay all packets exiting a network interface using this command: We can also add variable delay (jitter) as most wide area networks (such as the internet) have some jitter associated with them.\u00a0 The following command will add +\/- 10ms of jitter to the 80ms delay shown in the last example: To see what qdisc (short for queuing discipline) parameters have been applied to an interface (in this case eth0) use the following command: Now, for the important part – how do you turn this off? It took a while to find this in the netem documentation: So this is great, but not necessarily what I am looking for.\u00a0 Adding delay wholesale to the server would also increase my fellow gamers latency as well as mine, and the idea is to level the playing field.<\/p>\n That is ok, since netem\/tc has a way to only place qdisc’s on specific traffic.\u00a0 In my test network, I have two machines.\u00a0 One running Windows 7 (this case the Call of Duty Client) and one running Ubuntu 9.10 (the COD2 server).\u00a0 The Windows machine has an IP of 192.168.0.15, and the server 192.168.0.14.<\/p>\n On the linux server, I run the following commands as root: These commands do exactly what I was wanting – making my delay to the game server about equal to my friends.<\/p>\n So far, it seems to work, however it’s not optimal.\u00a0 The reason its not optimal is because only the packets coming from the server to my Windows client machine are being delayed.\u00a0 A true internet connection would have delay in both directions.\u00a0 Since netem only affects the egress of traffic from a network interface, technically you would have to delay the traffic as it leaves the client PC, and delay the traffic as it leaves the server back towards the client.\u00a0\u00a0 Since Windows doesn’t have a netem facility (at least not without some expensive commercial software such as that from Shunra) the best way to do this would be to run Call of Duty 2 on Linux using wine (which is another article for a another time).\u00a0 That way I could induce delay on both machines, and get a “more perfect”\u00a0 simulation of the internet.<\/p>\n To show existing filters such as those set by the last set of commands you can use the following commands: Netem has several capabilities that I didn’t cover, which can be found at the following links:<\/p>\n The Linux Foundation has an overview page here<\/a>. Next up – Client IPSEC VPNs from a linux laptop to a Juniper Netscreen VPN\/FIrewall device. Coming Soon!<\/p>\n","protected":false},"excerpt":{"rendered":" Back in the day, I used an open source program called NistNET to emulate a WAN for my company’s network test lab on a linux machine.\u00a0 I was able to solve a multitude of issues and test our applications in a WAN environment with this product.\u00a0 Unfortunately, NistNET is no longer maintained, and until recently […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4,8],"tags":[33,42,47,49,59],"_links":{"self":[{"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=\/wp\/v2\/posts\/41"}],"collection":[{"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=41"}],"version-history":[{"count":0,"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=\/wp\/v2\/posts\/41\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=41"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=41"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.linuxpoweruser.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=41"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n
\ntc qdisc add dev eth0 root netem delay 80ms
\n<\/code>
\nThis will add 80ms of delay to all packets leaving the eth0 interface.\u00a0 To test the result of this command, just do a ping from you machine before issuing the command, and then after:
\n
\nping -n 192.168.0.15<\/code>
\n
\nPING 192.168.0.15 (192.168.0.15) 56(84) bytes of data.
\n64 bytes from 192.168.0.15: icmp_seq=1 ttl=64 time=0.103 ms
\n64 bytes from 192.168.0.15: icmp_seq=2 ttl=64 time=0.394 ms
\n64 bytes from 192.168.0.15: icmp_seq=3 ttl=64 time=0.070 ms
\n64 bytes from 192.168.0.15: icmp_seq=4 ttl=64 time=0.101 ms
\n^C
\n--- 192.168.0.15 ping statistics ---
\n4 packets transmitted, 4 received, 0% packet loss, time 2998ms
\nrtt min\/avg\/max\/mdev = 0.070\/0.167\/0.394\/0.131 ms
\n<\/code>
\nEnter the tc command for adding delay to eth0:
\n
\ntc qdisc add dev eth0 root netem delay 80ms
\n<\/code>
\nThen ping again:
\n
\nping -n 192.168.0.15
\nPING 192.168.0.15 (192.168.0.15) 56(84) bytes of data.
\n64 bytes from 192.168.0.15: icmp_seq=1 ttl=64 time=80.0 ms
\n64 bytes from 192.168.0.15: icmp_seq=2 ttl=64 time=80.0 ms
\n64 bytes from 192.168.0.15: icmp_seq=3 ttl=64 time=80.0 ms
\n64 bytes from 192.168.0.15: icmp_seq=4 ttl=64 time=80.4 ms
\n^C
\n--- 192.168.0.15 ping statistics ---
\n4 packets transmitted, 4 received, 0% packet loss, time 3004ms
\nrtt min\/avg\/max\/mdev = 80.073\/80.164\/80.414\/0.246 ms
\n<\/code>
\nNotice the difference in delay (~80ms).<\/p>\n
\n
\ntc qdisc add dev eth0 root netem delay 80ms 10ms
\n<\/code>
\nNow lets do the ping again:
\n
\nPING 192.168.0.15 (192.168.0.15) 56(84) bytes of data.
\n64 bytes from 192.168.0.15: icmp_seq=1 ttl=64 time=72.6 ms
\n64 bytes from 192.168.0.15: icmp_seq=2 ttl=64 time=84.6 ms
\n64 bytes from 192.168.0.15: icmp_seq=3 ttl=64 time=86.7 ms
\n64 bytes from 192.168.0.15: icmp_seq=4 ttl=64 time=84.0 ms
\n^C
\n--- 192.168.0.15 ping statistics ---
\n4 packets transmitted, 4 received, 0% packet loss, time 3004ms
\nrtt min\/avg\/max\/mdev = 72.648\/82.023\/86.752\/5.510 ms
\n<\/code>
\nLooks even more like a real internet connection now.<\/p>\n
\n
\ntc qdisc show dev eth0
\n<\/code>
\nSample output follows:
\n
\nqdisc netem 8003: root limit 1000 delay 80.0ms\u00a0 10.0ms
\n<\/code>
\nThe last part of the output shows that a delay of 80ms +\/- 10ms is applied.<\/p>\n
\n
\ntc qdisc del dev eth0 root
\n<\/code>
\nThis will remove all queuing discipline parameters from the eth0 interface on your system.<\/p>\n
\n
\ntc qdisc add dev eth0 root handle 1: prio
\ntc qdisc add dev eth0 parent 1:3 handle 30: netem delay 140ms 10ms distribution normal
\ntc filter add dev eth0 protocol ip parent 1:0 prio 3 u32 match ip dst 192.168.0.15\/32 flowid 1:3
\n<\/code>
\nThis set of commands creates a simple priority queuing discipline, attaches a basic latency netem at hook 3, and then tells all traffic to 192.168.0.15 to be priority 3, thus subject to the netem delay of 140ms +\/- 10ms (with a normal statistical distribution of jitter).<\/p>\n
\n
\ntc filter show dev eth0
\n<\/code>
\nand
\n
\ntc qdisc show dev eth0
\n<\/code>
\nHere is an example output:
\n
\ntc filter show dev eth0
\n<\/code>
\n
\nfilter parent 1: protocol ip pref 3 u32
\nfilter parent 1: protocol ip pref 3 u32 fh 800: ht divisor 1
\nfilter parent 1: protocol ip pref 3 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:3
\nmatch c0a8000f\/ffffffff at 16
\n<\/code>
\n
\ntc qdisc show dev eth0
\n<\/code>
\n
\nqdisc prio 1: root bands 3 priomap\u00a0 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
\nqdisc netem 30: parent 1:3 limit 1000 delay 140.0ms\u00a0 10.0ms
\n<\/code>
\nIn summary, netem is a perfect example of what I love most about Linux – flexibility and utility beyond what most commercial Operating Systems (especially those from Redmond) offer out of the box for free. While the way I have implemented it is not quite perfect, netem can provide what I am looking for in terms of simulating network conditions. As I mentioned above, perfection could be achieved by using linux as the client as well as the server, or by making a bridge between the two machines with a dual interface linux machine doing netem delay on both interfaces.<\/p>\n
\nHere<\/a> is pdf file showing more details on netem and tc usage.<\/p>\n