Custom Router

The objective was to build a custom network router with overkill specs (future-proof) using pfSense and an Intel ITX motherboard. I wanted it to have a compact, rack-mountable design. This would be my primary router/firewall for my home network. At the very least, it needed to handle multiple VPN connections, a proxy server, and any other experiments I might do, including an IDS/IPS (intrusion detection system/intrusion prevention system). This kind of demand would require some decent processing power, plenty of RAM, and low latency storage.

Project Log

Date: Early 2014

I chose to use an old Dell 24-port switch for the chassis.

After looking at a lot of different motherboard options, I decided to go with the Jetway NF9HQL. http://www.jetwaycomputer.com/NF9H.html

I chose the Jetway because, at the time, it had the best specs for the price. It has a dual-core Intel Atom D525 1.8GHz, it takes DDR3 SODIMM, has 4x gigabit Ethernet ports, a Mini PCI-e/mSATA slot, a PCI-e x1 slot, and has integrated power (takes a 12V AC adapter instead of an ATX PSU). These features were really attractive to me. Its compact design and integrated power would ensure there was plenty of room in the chassis for air flow and future expansion. Also, most of the other motherboards I was considering had only a PCI port, so the PCI-e was a plus.

As an experiment, I installed the operating system on an SD card and used an SD card/SATA adapter instead of a typical hard drive. I had been using Raspberry Pi computers for a couple years and never had trouble with the SD cards. It doesn't necessarily meet my low latency storage objective, but I still wanted to try it.

After I gutted the chassis, I had to do some precision Dremel work on the back for the I/O panel and the VGA port. I can't tell you how pleased I am that I was able to cut it with such precision that the back plate snapped into place. It took some steady cutting with a cut-off wheel and about a half hour of grinding down to the exact size for the plate. The VGA port also required a funny shaped hole, and I'm surprised it turned out so well. Patience is the key.

The next challenge was to build a bracket for mounting the motherboard. The stand-offs in the chassis obviously didn't fit the Jetway motherboard. I found that the best way of figuring out where the screw holes needed to be was to put liquid paper (white-out) on each chassis stand-off, then press the plate down onto it in the correct orientation. Paint would have been better, but I didn't have any. This would leave a mark exactly where I needed to drill. I did one stand-off at a time because the liquid paper would dry in a couple seconds.

Another challenge was the fact that the motherboard needed to be a very exact height from the bottom of the chassis so that the I/O would line up with the back plate. I went to the hardware store and picked out plastic washers of varying thicknesses: somewhat thick ones and a bunch of the thinnest ones I could find. That way I could stack them to control the height. In the end, I actually found a single plastic washer that was the right height when stacked with 2 of the metal nuts I was using. The motherboard fit very nicely.

The SD card adapter only had two screw holes in the back, so in the front I just glued two plastic washers to the metal plate and allowed the adapter to sit on top of them while the back end had the luxury of being secured with screws.

Date: Mid - Late 2014

The router was starting to take shape. I took the RAM and network card from an unused laptop I had lying around. I took some wi-fi antennas off of a couple routers, but I had to order the antenna connectors. The operating system was installed and I was ready to begin bench testing.

I was particularly worried about heat. I ran it on full network load for a few hours, periodically checking the temperature. The system temperature reading and my infrared thermometer agreed; the temperature was well below the unit's maximum operating temperature of 60° C (140° F). The pictures below are in Fahrenheit.

I bought a black acrylic plate and cut it to the size of the face plate. I made the mistake of buying extruded acrylic as opposed to cast acrylic. As I understand it, cast acrylic is higher quality and easier to work with (but extruded is also ideal for some applications). I struggled to find a good way to cut and drill the plate. The acrylic had a tendency to melt and bead up on the edge like slag. I found that a circular bench saw worked best for cutting. It was higher RPM and the teeth took more of a "bite" rather than a "rub" (like a Dremel cut-off wheel). Also, when drilling, I discovered that you should press very lightly and spin the drill at max speed. I cracked the corner off of a perfectly good plate because I was pressing too hard, even when I thought I was being gentle. Literally just the weight of the drill is all the pressure you should apply. Also, at low speed, the drill bit tended to snag the acrylic and crack it. Someday, I may redo the face plate, but for now I'll just leave it with a rough edge. It won't be noticeable as it will be mounted on a rack between other appliances.

Here is a webpage that does a very good job explaining the best ways to work with acrylic. I wish I had found this page before I did this project. http://www.bcae1.com/plexi.htm

I also found an excellent site for ordering custom plastics. I didn't order mine from here, but I wish I did. http://www.tapplastics.com/

To finish off the face plate, I added a nice LED button.

The operating system is a fantastic FreeBSD based router OS called pfSense (https://www.pfsense.org/). My previous employer, a managed service provider (MSP), used it for their main firewall. We also tended to install it for many of our customers. It's very powerful and is capable enough for most enterprise applications. I'm not going to go into detail about the configuration process simply because I'm writing this in retrospect. I may eventually write some tutorials for pfSense.

Here are some nice features of pfSense:

• quick and easy to setup
• robust and reliable
• scales to any performance demand
• has an easy to use web interface
• tons of additional packages and a nice graphical package manager
• redundancy and failover
• very granular control over routing, subnetting, packet shaping, and services
• a backup/restore system that works like a dream
• It's FREE!

There was one very disappointing thing about pfSense at the time I built this: it had terrible wireless support. pfSense is based on FreeBSD, and at the time BSD did not have support for 802.11n. pfSense 2.1 had support for N mode, but it ran at B/G speed. FreeBSD 10 finally added full support for 802.11n, but until pfSense upgraded to FreeBSD 10, I was stuck using my old DLink router exclusively for wireless.This was an ugly solution, as getting rid of the DLink was the whole point of this build.

Finally, in pfSense 2.2, they upgraded to FreeBSD 10.1 with full 802.11n wireless support, and there was much rejoicing.

Here's kind of a rough description of my setup (as of 2014):

• WAN interface connected to my 40Mbps DSL modem.
• LAN interface for my internal network.
• Wireless interface that is bridged with the LAN interface, so LAN and wireless hosts are on the same subnet.
• DNS server that resolves hosts in my local domain and forwards requests for external hosts.
• DHCP server: leases reserved for certain hosts, a range for LAN hosts, and a separate range for wireless hosts (I can tell just by IP whether a host is wired or wireless).
• OpenVPN with user and certificate authentication. I have a private VPN and a guest VPN for my friends, both on their own subnets. Each friend has a user account, and is able to login and change their password ONLY. They then have limited access to certain servers (mostly for playing old school games).
• Squid reverse proxy for some web servers.

Date: Mid 2015

In my area, 2015 was the hottest Summer on record (as of this writing). I was disturbed to find my pfSense box struggling to stay cool. The single CPU fan was really giving it all she had, and I decided I would add another fan. The beast had been running flawlessly for almost a year. I didn't anticipate any immediate hardware failure, I was more worried about longevity. I found a small fan lying around, and made a little bracket for mounting and directing air flow. The fan was a little too tall for the chassis, so I mounted it at an angle.

I also added a potentiometer to control fan speed. I fount that a 0-25ohm range was ideal. If the range is too high, like if you used a 10k pot, it would be too sensitive. In other words, just budging the dial a tiny bit would be the difference between fully on and fully off; you wouldn't have a reasonable gradient to control the speed.

I mounted the fan and the potentiometer, and this is where disaster struck...

As you can see in the photo below, I no longer have the SD card device. During the process of mounting the fan, I must have damaged the card adapter somehow. I'm not sure how this happened; I was grounded while I worked on it and I put electrical tape over the top of the card adapter while I was mounting the fan so the metal plate didn't touch it. The SD card was fine and I had configuration backups stored on my server. I was able to throw in this 500GB laptop hard drive, reinstall pfSense, and restore the configurations from backup in no time. Enterprise solutions!! Woooo!! To be fair, the SD card idea held up until I ruined it.

And then... more bad. The new fan, which had previously been very quiet, had a horrible rattle when I turned it on. Again, I have no idea what happened. I didn't have another fan small enough, so I ordered one on eBay. It was no big deal waiting at this point as Summer was coming to an end and the cool Fall air meant the heat problem was no longer urgent.

Date: February 26, 2016

I don't often torrent things (anymore) but occasionally it is the most practical method for legally downloading massive files, such as a 300GB rainbow table. Last week, I was torrenting a digital monstrosity over the course of a few days, and I experienced, first hand, the extreme demand of an unabated torrent stream.

One day, my Internet suddenly cut out. My CenturyLink modem had been needing the occasional restart, so I wasn't surprised. I immediately pinged my pfSense box to make sure it was still up. It's usually up, but this time it was down. I go into my office to find it unpowered. Furthermore, it wouldn't turn back on. Panic mode: phase 1. I immediately pulled out my multimeter and tested the 12V AC adapter. It was putting out a clean 12V! Panic mode: phase 2.

I proceeded to disassemble my router and troubleshoot. I swapped the RAM, unplugged the hard drive, and tried a different power button. All of the capacitors on the motherboard looked nice and flat; none of them were bulging or split. Panic mode: undefined.

I couldn't accept that the board was so dead that it wouldn't even power on, let alone POST. Something was still fishy about the power adapter, as it was very hot to the touch. I rummaged through all my tubs of power adapters, but none of the 12V ones had an Amp rating high enough.

I decided to pull out a spare Mini ITX power supply. To use an ATX or ITX PSU as a general purpose power supply, you need to enable it by bridging the green wire and any black wire. I just used a paper clip. I then cut one of the yellow 12V wires and a black wire. Luckily, I had some spare N-type power connectors from RadioShack. A little bit of solder, shrink wrap tubing, and hot glue makes a proper connection (don't get lazy; take the time to do things right!).

Seeing my pfSense box light up was a beautiful sight! I've now ordered 2 spare power supplies, 3 Amp rated this time (the original was 1.5 Amps, but it probably died because it was cheap). I assume the "tabletop" power supplies (laptop style) will be more reliable, so I went with one of those. They're more expensive, but worth it.

Lessons Learned:

1. Not all products come with power supplies rated to accommodate additional loads (i.e. hard drives, fans, etc.) under full demand (i.e. downloading the entire Internet).
2. Just because a power supply is putting out the correct voltage doesn't mean it's working!
3. Always have spare parts on hand!

Date: March 15, 2016

This project is finally coming to a close. I have my new power adapter, so I shouldn't have problems with drawing too many Amps. I finally installed a fan in the chassis. I decided to just hot glue it in. If I need to replace it I can just rip it out. I decided to not use the potentiometer to control the fan speed because the new fan is pretty darn quiet. Typically, small fans are louder than large fans, but this one is very nice.

It is now a finished project and it's time to put it on the rack!! I will open a new project if I want to make further modifications.

One last surprise: high gain antennas for the win!!

Possible Future Hardware Changes

1. Add a second wireless card for guest wireless
2. Move the chassis fan to the other side so it's closer to the CPU
3. Extend the VGA ribbon cable so it's not sitting on top of the CPU heatsink/fan
4. Add LED indicators for network activity and user login
5. Clean up the edges of the face plate or order a pre-cut/pre-drilled piece
6. Replace the hard drive with a solid state drive
7. Put the potentiometer back in series with the fan to control the speed again
8. Put dust filter pads behind the vents
9. Touch up or repaint the chassis

Possible Future Software Changes

1. Captive portal for guest wireless and VPN so users can be directed to the servers they want to use
2. Web content filtering on guest wireless and VPN
3. Intrusion prevention and network based anti-virus
4. Uninterruptable power supply (UPS) should signal router to shutdown
5. Email/SMS alerts
6. ARP monitoring
7. RADIUS authentication (just for fun, I guess)