Posts tagged network

Just got unifi-ed

Don't Panik Featured Life

When Telekom Malaysia (TM) announced their Fiber to the home broadband initiative, I was rather excited as broadband speeds in Malaysia have stagnated over the past couple of years. Initially when I jumped on to the DSL bandwagon in 2001 with a paltry (by today’s standard) speed of 384kbps down and 128kbps up. It was a big improvement over any 33.6 or even 56kbps modems of the day. Fast forward 10 years and internet speeds have doubled or tripled many times in neighboring countries and all we’re stuck with is 2mbps DSL. 4mbps is available but few areas could even qualify with the poor quality of cabling or exchange equipment. My home couldn’t even get past 1.8mbps due to the distance from the exchange.

It took TM about a year plus to reach my backwater housing area. The good thing is many others have sacrificed their effort and time to enable TM to iron out the kinks in the system and to improve the roll-out, delivery and implementation service. After all, there are thousands and hundreds of thousands of homes to install.

The good is that they no longer need eight guys to install like for my parents home, a year back. The bad is that they still are not coordinated with their contact center and appointment scheduling system. It still takes an entire day but most of it will be spent waiting and waiting or cleaning up. If you’re lucky, everything goes well after install otherwise, you’ll end up waiting for new modems, BTUs, and more technicians.

For me, the install was a breeze. Minus the five hours waiting for the installers to show up. When the appointment time said 9:30am to 2:30pm, I assumed the install will complete within that window rather than start at 2:45pm! Anyway, my install was through the ceiling so all was needed was a hole drilled through the outside wall to get into the ceiling, pull the fiber and drop it into my central cable drop. My home, fortunately for the TM installers, had a central cable drop where all ethernet, phone, cable TV, etc are laid down so drilling through the wall took the longest time. The Brits build houses to last in the 60s.

Fortunately for the installers, I also provided the ceiling light and ladder otherwise they would have come to grief with such a miserable ladder of theirs. Also to their benefit, my entire house is wired with CAT5e (long story why it’s not CAT6) so again, the rest of the setup was a breeze. Plug the Fiber BTU here, LAN to WAN port, IP TV out to Port 2 (next to my TV), Phone out to Phone Distribution panel and viola, IP TV works, we got internet and the rooms have phone lines! Hallelujah!

Since I had a Cisco VPN Router & SPI firewall going, I wanted to junk the crappy DLINK DIR615 that came with the package. The router has caused grief to many users due to it running custom firmware, lack of security (all wide open settings), poor wireless and network performance, and more. The problem with using your own router is that the incoming network has three VLANs and if you didn’t care about the IP-TV, you can go ahead. I paid for all services so I damn well want the access.

Solution was simply replace the DIR615 with a VLAN bridge. I used a MikroTik RB250 for this purpose. If anyone wants a pre-configured RB250, drop me an email. I have the RouterBoard Rb750GS as well but for the moment, I wanted to continue using my Cisco router.

Here’s the TM Fiber Broadband Termination unit (BTU)

The crappy D-Link DIR615 which was quickly replaced…

My Cisco VPN and IPS/IDS Firewall

The Huawei IP TV Set-top box (STB). Still a bit laggy when watching internet streaming content

The awesome MikroTik RB250 VLAN Bridge.

I really need to fix my cabling mess! For now, Yellow is WAN link, white is Internal network, Blue is POE, green is for Voice and now Red for IPTV.

 So how does it perform? The good is that it works pretty much as it is advertised. No complaints until the service starts going down or becomes unreliable but overall, my satisfaction is high. Minus the setup, installation part of course. Your mileage may vary and depending on your home and where you want certain components, i.e. IPTV, Wireless, phone, etc, the process might cost you a fair bit and be more trouble than in my case.

Storage & Backup again

Photography 1 Comment

Seems like there’s a raft of people I know who recently lost some stuff due to disk crashes or are looking at ways to store that ever growing pile/stack/stash/gigabytes/terabytes/petabytes of data that people nowadays are generating. Even non-photographer friends I have are either busy downloading TV series or accumulating lots of media here and there such as e-books, songs, photos of their children, etc. Generally, media is usually the biggest culprit. I mean how large can a collection of excel spreadsheets or word documents be?

My approach is a two-tier storage system with my QNAP NAS playing primary on-line storage and a custom-built machine serving as the full storage backup. The following is a simple sketch of my network with my QNAP and my storage server. They’re connected to my Cisco switch using Link Aggregation (802.3ad) which offers both redundancy and faster transfer speeds. The little diagram on the right is how my content is stored between the server and the NAS. In a nutshell, my main storage server contains everything but the NAS has a little sub-set of the data.

Basically, my server stores everything in 1TB & 1.5TB disks, mirrored of course. I use rsync/robocopy to mirror as I’ve mentioned much earlier about using cheaper RAID cards. The mirrored pair is exposed on the network with one disk primarily for reading and the other for writing. What basically happens is that I back up to my NAS and my Storage Server (when it’s switched on – which is like at least a few times a week), pulls the backup from the NAS. When working on photos (especially for paid clients), I usually back-up simultaneously to both my QNAP and my Storage server. You can see below where I have my disks…

In case you are wondering (which you will), my disks are all short-stroked. My two swap disks are 150Gb 10,000rpm Raptor disks. Also, if you noticed, my storage server seems rather short of space… well, that’s because all the files are using NTFS junction points since I don’t have a RAID array… there’s actually 3 sets of mirrored disks. Total capacity is 1TBx 2 and 1.5TB x 4 / 2 due to the mirror. Usable capacity is about 3.4TB. Using junction points, 2001 and 2002 directories can reside on Pair A while 2003 and 2004 can be actually on Pair B. A bit complex but cost wise, all you need is a system with more than 4 SATA ports and you’re set. Recovery is also easier. Currently, the Samsung HD154UI 5,400rpm 1.5TB disks are good value but I’ll be moving away from them considering their slow-as-snail spin-up time and generally overall poor performance. I’m testing Hitachi’s affordable (runs a little hot though) 2TB, 5-platter, 7K2000 drive and see if it’ll give me back some performance lost to the Samsungs. Green drives may consume less power but the slow speed may result in high power consumption if we end up leaving more systems on just to complete back-ups and disks synchronization.

So far, I’ve been satisfied with my Seagate 7200.12 1TBs but looking for 1.5TB and 2TB drives are tough if you’re trying to avoid the green marketing crap.

Unifi, decent Broadband at last?

Life

As one of the first TM Streamyx customers way back in 2001, ten years is a long time to wait for decent broadband. Streamyx was good when it launched but like anything in Malaysia, it just stagnates. Anyways, my parent’s house is among the first few areas for Unifi, unlike where I stay now which is low on TM’s priority list just because the exchange we use is pretty run down.


The speed is great but the installation purely sucks. Don’t get me wrong, the guys did a decent job and were quite clean but I think the first thing Telekom Malaysia should have done is not to assume each household has a single PC. Secondly, the installers should not assume the cable drop is next to the phone table and next to the TV set. So here goes the installer bitching about my complicated setup… see below and trying to charge my mom RM12 per meter run of CAT5 cable. Good thing I have a box (a thousand feet) handy. CAT5 cable run in my parent’s home, measured from the switch is about 85 feet (26m) to the TV area, 75 feet (23m) to the guest room, 65 feet (20m) to Rooms 1 and 2.

Original ADSL setup

Since I wasn’t around when the completed the installation, they managed to hoodwink my parents into believing that all the PCs are already connected. As my parents didn’t know how to run any network testing, all they did was try to load some websites. They even forgot to testing printing. My mom tested printing from her PC (in the guest room) though so she thought everything worked. Yes, it works but half the house is still on Streamyx!! See below diagram.

Initial Unifi Fiber Setup

I dropped at 4:30pm and first thing I did was to do a cable trace. Not liking what I found, I turned off the ADSL connection and immediately two PCs lost internet connectivity. It’s obvious why. These are the things I immediately configured or changed:

  1. Change the Router’s Administrator password (defaulted to blank!)
  2. Turned off the Wireless on TM’s DIR-615. (If you intend to use it, rename the Wireless Network Name/SSID)
    * The default SSID uses the customer’s name, i.e.: subscribername@unifi.
  3. Disable Remote Access to the router
  4. Enable Firewall, including blocking scan, DOS attacks, service filters, the whole enchilada….
  5. Adjusted the DHCP port range so it didn’t conflict with my other devices that were statically assigned

Here’s how the network finally looks like:

How it looks like in the end

It looks like I’ve to be around and end up doing a fair bit of work if (when TM decides to upgrade the Sungei Way exchange) I subscribe to Unifi and their installers come to my home. It’s way more complicated that my parents!

By the way, I didn’t play around much with the phone service or the IPTV service. It’s not that attractive to me at the moment but hopefully it’ll change in the future. I’m hoping for pure SIP services from TM in terms of the voice side…

NAS Purchasing Questions

Photography

Seems like after posting the NAS entries, I got a question from a friend trying to archive his photos as well as other data, i.e. media files, documents, backups, etc. So how do you decide which NAS to buy?

For everyone and anyone who asks me, I’ll always go with a dual-drive NAS. If you decide to go with a single drive NAS, you might as well go with USB disks like Seagate FreeAgents or WD Elements. So, basically you have to calculate how much storage you need. One area to start with is to see how much space you use in one year and how much that grows. My average storage needs (photos only) are about 600Gb/year. Chances are that won’t increase much unless I upgrade my cameras. I do budget a 10% increase. I do have other needs also so by the time I added everything in, it’s a large number.

Secondly, ask yourself f the NAS is your primary backup or not. If it is, it will need to be able to store all your backups. It’s good to take into account a 3-year plan so if you need 600Gb/year, then your NAS will be or must be able to grow to accomodate at least 2TB. This actually then dictates the size of your NAS. If it’s secondary storage (like in my case) where you have backups somewhere else (Hard drives, DVDs, BDs, etc) then the requirements are different. I wanted past 3-years of media (Photos & videos) on my NAS while my main backup stores everything. This required at least a capacity of 2.2TB at least which is why I went with a 4-drive unit. Most 2-drive NAS units max out at 2TB (2x2TB disks in mirrored/RAID1 config).

Finally, the next thing you need to consider is how fast you want to get the data off the NAS. I assume you’d be connecting at least via gigabit speeds. The specifications of the NAS play a part in the speed (excluding poor network speeds) aspect so normally I look at what CPU the NAS has along with the memory.

So what are my choices/recommendations?

Budget: DLink DNS-323
Cheaply priced (RM439/USD130), it’s the lowest cost 2-drive NAS I would buy myself.

I’ve only had experience with QNAP and Synology NAS units so that’s what I’ll stick with. You can go over to Smallnetbuilder and check their NAS reviews but not all are available locally in Malaysia. Here’s a handy performance chart for QNAP and Synology NASes.

Two-bay units:
Synology : DS210j or DS209
Qnap : TS-210 or TS-219

Two-bay units (higher performance):
Synology DS210+
Qnap TS-239 Pro II

Four-bay units:
Synology : DS410j
Qnap: TS-410 or TS-419P

Four-bay units (higher performance):
Synology : DS410
Qnap: TS-439 Pro II

note: this list above is from May 2010. If you need more space, you can consider the 5 and 6 bay models.

Storage strategies

Photography

A few people have asked me to elaborate on my storage strategy so here it is. Some background information is needed first. I started taking photography seriously in 1998. Buying my first SLR back then was a big thing and working through the lenses and other options. In late ’99, was basically scanning the prints on a flatbed scanner. Quality was amazing, or at least what I thought was amazing until I stumbled upon a very expensive (at that time, I wasn’t making much a month) option; the slide scanner. By skipping the printing stage and scanning directly from film, it delivered unmatched results from anything else unless you’re comparing to a drum scan. Since then, I’ve been scanning almost all my slides and negatives but most of what I shot were from my travels. Film costs a fair bit, especially if you shot transparency film such as Fuji Velvia, Kodachrome or Elitechromes, so the everyday life doesn’t often get captured on film.

So with digital scans dating all the way back to 2000 (the 2700 dpi scans tend to produce on average 14MB LZW compressed TIFF files), storage is key, along with a proper backup strategy. I went with CD-Rs and then DVD-Rs but they never were convenient (not fun, not quick, and laborious process) meant that you will almost never will do it timely enough so I decided to stick with hard drives. They get cheaper and are inherently quicker to access if they’re stuck into a spare machine that can be booted up anytime you needed to.

I upgraded scanners so the file sizes got bigger and films like Velvia and EliteChrome 100 pushed the resolution (grain?) further. Then came my first DSLR, a 300D (my shortest-lived DSLR ownership) and my 20D several months later. So came RAW files, 16-bit PSDs, and the story continues until I hit my 5D Mark II with 21-megapixels, Full HD movies. Backup is getting more important but also important is accessibility.

I needed to backup and offload my main PC but also need to access the “older” stuff occasionally. Leaving your main storage server switched on/off daily or 24×7 wasn’t something I want to do on a daily basis so I got a NAS which I quickly outgrew within a year so that too got upgraded to a 4-bay unit.

I have a storage server with 6x1TB disks. It’s all RAID 1. Mirrored. So I get a net space of 3TB. I have a “_photos” folder shared out that further breaks down by year, i.e. “2001”, “2003”, along with some odd directories. I also have a “_videos” folder where I back up my created DVDs (ISO format) and Adobe Premiere Pro project files. The problem is occasionally accessing stuff from say, 2006. My main PC only stores past 16-odd months so I ended up going to my storage server regularly. So what’s running 24×7 is my NAS box with 4x500Gb in RAID5. That gives me about 1.4TB of disk space. That way, I get that past 2-years on it; i.e. 2008 & 2009 along with my 60Gb music collection on the built-in iTunes server. My IP cameras also record to NAS and they keep the past-7 days recording. I also have imaged backups of all the machines in my home so that if the primary disk crashes, recovery is just a click away. Keeping everything in sync is the unix utility rsync (the windows equivalent is DeltaCopy).

With the growing file sizes, I’ll be moving my storage server to 1.5TB disks by Q4 this year and my NAS will grow to 4x1TB*… one can never have enough disk space!

* That’s one thing I like about my NAS. I can live swap the disks and expand it without powering down and losing my data. I can easily move from 500Gbx4 to 1TBx4 and later to 2TBx4.

QNAP TS-439 Pro unboxing and mini-review

Don't Panik
The big box

If you’ve been following this blog for a while, you’ll have noticed a few posts alluding to the search of some network storage to replace my 2-drive unit. I finally settled on the QNAP TS-439 Pro. Read the previous post about it or go to QNAP’s site to read more. Basically, I needed a compact, four-drive unit. Why four drives? It allows either two mirrored sets of disks (note: not necessarily RAID1) or RAID5 (with or without a spare). Going larger (5, 6 or 8 drive units) adds exponentially to the price but I wanted the use of the few 500Gb disks I have lying around. 500Gb is quite small but if I put them in a RAID5 set, I get 1TB (n-1). Later when I move some of my main storage disks to 1.5TB disks, I get another 500Gb to slot in as a spare drive (3x500Gb + 1 spare).

Other highlights for me were the hot-swap disk trays (lockable), multiple USB ports for printer sharing and dual-Gigabit LAN ports. The other stuff are frills. The dual gigabit Ethernet ports give you a fair bit of flexibility. You can configure two separate links for two separate networks, or some form of load-balancing, etc. Mine is configured for link aggregation group (LAG) mode. If you want to read more on that, which can get technical, Google for 802.3ad. In a nutshell, what LAG does is to create an aggregation group of network links that share the same speed and duplex settings. This requires a switch that supports LAG though and few consumer grade switches support it. The TS-439 Pro does have another option of using Adaptive load balancing. This bonds the network interfaces on the 439 and allows it to aggregate without any switch support. There’s a slight drop in performance but if you don’t have a switch that supports 802.3ad, it is the way to go. Did I mention two links means redundancy?

Pretty basic and unassuming, low power, low heat and quiet.

My primary use is to serve as my iTunes and media server (uPNP/TwonkyMedia), download stuff (BitTorrent), IP camera recorder, and a common file store to use between my PCs and notebooks in the house. My plan is to copy my photos there from camera, then once they’re sort of done (processing, etc), they can be moved to my File Server (which has 6x1Tb) in a RAID1+0 config. I’ve used jperf to do some testing to check the LAN bandwidth/switch speeds and for the TS-439, some regular file transfer timings.

First off, my gigabit LAN between Win Server 2003 R2 (dual-core Athlon, 6x1TB, 3Gb RAM, single link, the onboard NIC is died earlier this year so I’m running an Intel Gigabit CT off the PCIe x1 slot) and my Vista 64 SP2 client. I ran jperf over a few minutes with a five second interval. Min speed reported at 601,274Kbits/sec, max speed reported at 879,761Kbits/sec. Average the 30 readings and I get an average of 795,208Kbits/sec, or 776Mbits/sec or about 97Mbytes/sec. So let’s see how the 439 Pro transfers — remember, I have link aggregation on the TS-439.

1. Transfering 14,521Mbytes of photos. What I have is a mix of small and large files sizes. Large as in on average 26,912kb and small as in about 9kb (the XMP sidecar file). Transfer took about 320 seconds. 45.3MBytes/sec.

2. Transfering only the CR2 files took faster, clocking in about 261 seconds, or 55.5MBytes/sec.

3. Dual client access. Transfer a directory of photos and sidecar XMP files (12,672Mbytes and 11,760Mbytes). Start the transfer on client A then client B. Same target on NAS. Client A finished at 262 seconds. Client B finished at 302 seconds. Note that client B started transferring at roughly the 30 second mark of Client A. Average transfer speed is about 48.3Mbytes/sec for client A and 38.9Mbytes/sec for client B. Not too shabby considering both were hitting the NAS at the same time for about 87Mbytes/sec. Good enough for me.

Testing single disk setup

I’ve used it as a mapped drive for working on my photos and from Lightroom 3 beta 2 and Bridge CS4, the images feel very snappy. Not as quick of course compared to my local SATA drive but it’s not that obvious you’re working through a network. The lag on bridge in terms of refresh for example is noticeable only when you have three clients hitting the NAS but the fact is that its not terribly slow. My guess is that if Bridge refreshes my 800 photo directory on my local drive in 5 seconds, it takes about 8 or 9 on the NAS and perhaps 15+ seconds if you have multiple clients hitting F5 at the same time. Overall, I’m pretty satisfied. Maybe it’s time to look for dual-Gigabit port NICs for my client machines as well but I think the NAS will get faster when I up the RAM to 2Gb.

Fancy cover flow style admin interface

Overall, I’m pretty happy. It serves up my music to my iTunes, I can also listen to music/look at photos/watch movies on my Popcorn Hour C-200 via Twonkymedia server, record from all four of my IP cameras and work on a shared folder of photos. Not cheap but probably more cost effective to run than a standalone PC or server (if you factor in licensing, power, heat and noise). You can do other things with it as well though even thought I don’t such as running a Web or FTP server, etc.

If you want a more in-depth review, you can read it here.

Most administrative interfaces are web based now…