I do a lot of web development work, which usually doesn’t require a lot of knowledge of low-level networking details. But from time to time it becomes necessary to work below the HTTP protocol, to debug a broken remote procedure call, or reverse engineer a third party ajax app. These tools make many low-level networking tasks a breeze. These are all command line utilities, by the way, since that’s where I’m most comfortable.

slurm

slurm is a network load monitor that displays an ascii graph of incoming and outgoing bandwidth usage for an interface in real time. It’s a simple and easy to use utility that comes in handy whenever you get the urge to check out how much traffic is crossing an interface on your box.

Slurm

netstat

netstat is the workhorse of the network engineer’s toolbox. You can use it to discover lots of useful information about the state of interfaces, open sockets, routing interfaces, and protocol information for the transport layer (TCP, UDP, etc) and the internet layer (IP, IGMP, ICMP, etc).

The -i flag displays interface statistics such as the current MTU (Maximum Transmission Unit — the largest packet that the interface can pass without segmentation), and the number of packets sent (TX) and received (RX) over an interface with errors (RX-ERR/TX-ERR) or without (RX-OK/TX-OK). 

root@www:~# netstat -i
Kernel Interface table
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0   1500 0  120434205      0      0      0 126839757      0      0      0 BMRU
lo    16436 0   7330467      0      0      0  7330467      0      0      0 LRU

The -s flag shows a summary of protocol-level statistics for all interfaces. Statistics are typically shown for the TCP, UDP, ICMP, and IP protocols, with some variation between implementations.

root@www:~# netstat -s | head
Ip:
    127757501 total packets received
    10 with invalid addresses
    0 forwarded
    2 with unknown protocol
    0 incoming packets discarded
    127757286 incoming packets delivered
    127064471 requests sent out
    1 fragments dropped after timeout
    12 reassemblies required

ngrep

ngrep, developed by Jordan Ritter (who also wrote the backend for the original Napster) is probably my favorite network utility due to it’s power and simplicity. It is a simple command line tool built on top of the PCAP (Packet CAPturing) library that allows you to specify an extended regular or hexadecimal expression to match against the payloads of raw packets. It is particularly useful for debugging plaintext protocol interactions such as HTTP, SMTP, FTP, etc (and incidentally, has become a popular tool for crackers).

From time to time I’ve needed to view the traffic being passed between a web server and web browser, and ngrep is just the tool for this job. I’ve found myself using ngrep more and more frequently, as ajax has become more popular, to debug asynchronous client-server interaction. To view HTTP traffic we can simply tell ngrep to dump all traffic going to or from port 80 (the default port for HTTP). I’ve also specified -W byline mode which makes ngrep respect linebreaks, and provides more readable output.

root@www:~# ngrep -W byline port 80
interface: eth0 (74.52.115.96/255.255.255.248)
filter: (ip or ip6) and ( port 80 )
####
T 204.111.140.165:61680 -> 74.52.115.98:80 [AP]
GET /home HTTP/1.1.
Host: vino2vino.com.
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3.
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5.
Accept-Language: en-us,en;q=0.5.
Accept-Encoding: gzip,deflate.
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7.
Keep-Alive: 300.
Connection: keep-alive.
Referer: http://vino2vino.com/wine/161992.
Cookie: __utma=269925878.2131118343.1174418491.1177962568.1177967641.216; __utmz=269925878.1177552137.203.30.utmccn=(referral)|utmcsr=blog.vino2vino.com|utmcct=/2007/03/10/wine-review-widget-live-on-widgetbox/|utmcmd=referral; PHPSESSID=993c6f697bde63ce5129b08e1a6e31d7; __utmc=269925878.
.

##
T 74.52.115.98:80 -> 204.111.140.165:61680 [A]
HTTP/1.1 200 OK.
Date: Mon, 30 Apr 2007 22:03:03 GMT.
Server: Apache/2.2.3 (Debian) PHP/5.2.0-10.
X-Powered-By: PHP/5.2.0-10.
Expires: Thu, 19 Nov 1981 08:52:00 GMT.
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0.
Pragma: no-cache.
Keep-Alive: timeout=15, max=100.
Connection: Keep-Alive.
Transfer-Encoding: chunked.
Content-Type: text/html; charset=UTF-8.
.
3164.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <title>Welcome to Vino2Vino</title>

If you were receiving 404 errors for an ajax request and wanted to review the raw data being sent across the wire you could run ngrep -W byline '404' port 80, which will only display packets matching the regular expression ‘404′.

root@www:~# ngrep -W byline '404' port 80
interface: eth0 (74.52.115.96/255.255.255.248)
filter: (ip or ip6) and ( port 80 )
match: 404
###################################
T 74.52.115.100:80 -> 204.111.140.165:61700 [AP]
HTTP/1.1 404 Not Found.
Date: Mon, 30 Apr 2007 22:09:24 GMT.
Server: Apache/2.2.3 (Debian) PHP/5.2.0-10.
Content-Length: 295.
Keep-Alive: timeout=15, max=98.
Connection: Keep-Alive.
Content-Type: text/html; charset=iso-8859-1.
.
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

There are a number of additional examples of ngrep’s functionality on the usage section of the ngrep website.

netcat

netcat is another tool commonly used by network security professionals and crackers alike. I find it useful for replaying HTTP sessions or simply displaying the results of an HTTP request on the command line (you can also do this with telnet, but not quite as gracefully — and there are many things that netcat can do that telnet can’t).

root@www:~# echo -e "GET / HTTP/1.0\\r\\n\\r\\n" | \\
> netcat www.google.com 80
HTTP/1.0 200 OK
Cache-Control: private
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=3d88cbffaaaba042:TM=1177972009:LM=1177972009:S=BjrteegKWiXHr_wx; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.com
Server: GWS/2.1
Date: Mon, 30 Apr 2007 22:26:49 GMT
Connection: Close

<html><head><meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"><title>Google&lt/title><style><!--
body,td,a,p,.h{font-family:arial,sans-serif}
.h{font-size:20px}
.h{color:#3366cc}
.q{color:#00c}
--></style>

vnstat

vnStat is a network traffic monitor, like slurm. But rather than displaying real-time traffic information, vnStat collects network utilization statistics and stores this information in a database for future use. Once you’ve told vnStat to collect statistics (by running vnstat -u -i <interface>), you can use it to generate utilization reports over various time periods.

Here’s a report showing network utilization over the past few days:

root@www:~# vnstat --days

        eth0

            day         rx      |     tx      |  total
        ------------------------+-------------+--------------
           12.04.    511.83 MB  |  519.66 MB  |    1031 MB
           13.04.    369.43 MB  |  460.56 MB  |  829.99 MB
           14.04.    564.80 MB  |  644.63 MB  |    1209 MB
           15.04.    610.19 MB  |    1111 MB  |    1722 MB
           16.04.    577.28 MB  |  756.98 MB  |    1334 MB
           17.04.    714.81 MB  |  903.13 MB  |    1617 MB
           18.04.      1071 MB  |    1218 MB  |    2290 MB
           19.04.    666.18 MB  |  783.72 MB  |    1449 MB
           20.04.      1264 MB  |    1643 MB  |    2908 MB

And one showing utilization over the past 24 hours:

root@www:~# vnstat --hours
 eth0                                                                     17:40
  ^                  t
  |                  t
  |                  t
  |                 rt
  |                 rt
  |                 rt
  |                 rt
  |                 rt        t                    t
  |                 rt    rt rt rt  t  t rt rt rt rt rt rt     t
  |   t        t rt rt rt rt rt rt rt rt rt rt rt rt rt rt rt  t  t  t
 -+--------------------------------------------------------------------------->
  |  18 19 20 21 22 23 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17    

 h   rx (kB)    tx (kB)      h   rx (kB)    tx (kB)      h   rx (kB)    tx (kB)
18       9016      17456    02      26788      32898    10      22850      29821
19       6651       8465    03      22640      27922    11      24140      25697
20       3964       5550    04      18516      25529    12      14044      19099
21       6369      10896    05      19193      26345    13       7385      21880
22      18948      18296    06      20406      27190    14       9550      14396
23      72491     101025    07      23438      30176    15       9670      10728
00      14985      19733    08      23385      26043    16       7623       9054
01      21986      27116    09      25692      31459    17       8934       7894

There are tons of other useful tools for monitoring and manipulating information that is being passed across your machine’s network interface(s). These are simply the most useful tools that I use on a regular basis. Feel free to add a comment if you think I left something out.