Branch Office Routers: Build or Buy?
How many times has your friendly Linux administrator said "I can do that in Linux for only $1000 in hardware!"
Many discussions have taken place in various forums both for and against building your own router. We'll try to combine some of the important considerations into one place.
First, one point of confusion needs to be taken care of. You simply cannot push line-rate gigabit through PC hardware. Sorry, most low-end Cisco routers can't even do that. When we're talking about access-lists and possibly stateful filtering, and the added processing involved in that, PC hardware can't come close. You're never going to replace important Internet routers with software. More information about hardware forwarding can be found in a past article about TCAMs.
What we are talking about is the possibility of running a branch office router on a BSD machine. Most network engineers prefer some flavor of BSD to Linux, so we'll assume BSD in this discussion. The likely cause for this widespread disdain is because Linux kernel 2.4 had a tendency to deadlock when fed more than 100K routes.
There are two software routing packages available: quagga and gated. Gated is non-free, but copes better with a full Internet routing table. Quagga is the successor to zebra, and is free. Both applications can speak OSPFv3, BGP4, and RIPv2 for both IPv4 and IPv6.
The considerations, from a technical standpoint, are twofold: memory and Packets Per Second, or PPS. PPS is a measure of how fast packets can be forwarded. The network card and its driver make all the difference here. A good Intel NIC should be high on the shopping list. Memory is important because of the memory requirements inherent in BGP routing. For example, 256MB is barely enough for a full Internet routing table at its current size. Latency will increase, since nothing is going to be forwarded in hardware with a home-built solution, but it's normally tolerable.
T1 and T3 cards can be obtained for both PCI slots and Soekris boards (MiniPCI). A modest FreeBSD machine can certainly handle T1 traffic, including routing daemons and filtering, without too much trouble. There have even been reports of Soekris boards running FreeBSD and pushing a DS3 at line-rate while handling 80,000 BGP routes.
There are many fine points to argue about using PC hardware for routers. Perhaps Linux 2.6 has fixed everything and it is just as stable as BSD, perhaps you can push 45Mb/s on a 550 MHz PIII machine with full stateful filtering. Lots of things are possible, but let's talk about whether or not you should.
Like most things IT, great care needs to be taken when deciding whether or not to deploy a home-built router. The financial aspects go far beyond the initial hardware purchases. If a solution can be built for less than $1,000, and the alternative router from Cisco costs $5,000, this doesn't necessarily mean the $1,000 options is the cheapest.
A funny thing about PC hardware is that there are moving parts, and sporadically inferior components. Hard drives die all the time, and CPU and memory frequently need to be replaced as well. The really great thing about buying a Cisco 2600-class router is that there are no untested components, and there are no moving parts. If something does go wrong for some reason, a replacement is just a quick phone call away. The saved configuration file is loaded into the replacement, and you're back in operation almost immediately.
If the custom-built computer fails, it's normally necessary to rebuild from scratch. That means installing the OS, restoring configurations from backup (hopefully) and basically reconfiguring everything. Before you reach for that feedback link above... yes, it's possible to take a hard-drive image of each router, and use that to restore the computer in the event of catastrophic failure. That method doesn't quite scale, though. Even if you have similar hardware on many routers so that it's possible to use the same image all the time, you're still backup up more than one simple Cisco config file. You have to save password files, firewall configurations, hostname-specific configurations, and the list goes on.
Complete and total failure isn't the only concern. What will you do when your OS starts locking up for no apparent reason? You will definitely see some bugs, especially if you're handing the router a full Internet routing table. Home-built routers require on-staff expertise far beyond the ability to read a howto document to "just get it working." Sometimes it's even necessary to hack some code.
The most important consideration, aside from PC hardware quality, is that of time and skill. When you apply kernel updates on the BSD machine, will something else break? It's hard to say. The great part about spending the money on a Cisco router is that it only has one purpose. It is your router. Updates to unrelated components (there are none in a Cisco) couldn't break your existing router. Cisco service plans include security updates and bug fixes in the form of a single easy-to-update software image.
If building a software router is your business's core focus, then by all means you'll want to roll your own. If you're planning on dedicating an employee to maintaining these hand-rolled router boxes, then you've thought well about this problem. If not, then you need to sit down and carefully weigh the (dis)advantages of doing so. Routers that you purchase from a company have been carefully configured to do just that: route. You'll save money in the long run, both in terms of hardware and man-hours spent working on it.