Squeeze Your Gigabit NIC for Top Performance - Page 2

Gigabit network cards are becoming more and more common, but getting maximum speed depends on a the right mix of hardware, software, and finesse. Here's how to squeeze top performance out of your gigabit gear using Linux, FreeBSD, and Windows.

By Charlie Schluting | Posted Feb 24, 2007
Page 2 of 2   |  Back to Page 1
Print ArticleEmail Article
  • Share on Facebook
  • Share on Twitter
  • Share on LinkedIn

Configuring Linux and FreeBSD

For both Linux and FreeBSD we're using the sysctl utility. For all of the following options, entering the command 'sysctl variable=number' should do the trick. To view the current settings use: 'sysctl <variable name>'

  • Maximum window size:
    • FreeBSD:
      kern.ipc.maxsockbuf=262144
    • Linux:
      net.core.wmem_max=8388608
  • Default window size:
    • FreeBSD, sending and receiving:
      net.inet.tcp.sendspace=65536
      net.inet.tcp.recvspace=65536
    • Linux, sending and receiving:
      net.core.wmem_default = 65536
      net.core.rmem_default = 65536
  • RFC 1323:
    This enables the useful window scaling options defined in rfc1323, which allows the windows to dynamically get larger than we specified above.
    • FreeBSD:
      net.inet.tcp.rfc1323=1
    • Linux:
      net.ipv4.tcp_window_scaling=1
  • Buffers:
    When sending large amounts of data, we can run the operating system out of buffers. This option should be enabled before attempting to use the above settings. To increase the amount of "mbufs" available:
    • FreeBSD:
      kern.ipc.nmbclusters=32768
    • Linux:
      net.ipv4.tcp_mem= 98304 131072 196608

These quick changes will skyrocket TCP performance. Afterwards we were able to run ttcp and attain around 895 Mb/s every time – quite an impressive data rate. There are other options available for adjusting the UDP datagram sizes as well, but we're mainly focusing on TCP here.

Windows XP/2000 Server/Server 2003
The magical location for TCP settings in the registry editor is HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

We need to add a registry DWORD named TcpWindowSize, and enter a sufficiently large size. 131400 (make sure you click on 'decimal') should be enough.Tcp1323Opts should be set to 3. This enables both rfc1323 scaling and timestamps.

And, similarly to Unix, we want to increase the TCP buffer sizes:

 ForwardBufferMemory
80000 NumForwardPackets 60000 

One last important note for Windows XP users: If you've installed Service Pack 2, then there is another likely culprit for poor network performance. Explained in 'knowledge base article 842264(http://support.microsoft.com/?kbid=842264), Microsoft says that disabling Internet Connection Sharing after an SP2 install should fix performance issues.

The above tweaks should enable your sufficiently fast server to attain much faster data rates over TCP. If your specific application makes significant use of UDP, then it will be worth looking into similar options relating to UDP datagram sizes. Remember, we obtained close to 900Mb/s with a very fast Pentium 4 machine, server-class motherboard, and quality Intel network card. Results may vary wildly, but adjusting the above settings are a necessary step toward realizing your server's capabilities.


Charlie Schluting is the author of Network Ninja, a must-read for every network engineer.

Comment and Contribute
(Maximum characters: 1200). You have
characters left.
Get the Latest Scoop with Enterprise Networking Planet Newsletter