Stephen Foskett, Pack Rat

The first storage performance horseman is spindles: If you don’t have enough disk units, performance will suffer. I have been laying out storage on enterprise arrays since the dark ages, and one of the first lessons I learned was allocating data to avoid hotspots. I remember spending hours back in the 1990’s hunched over custom Excel spreadsheets trying to get my storage layout just right, balancing the workload across every available disk.

This is how we used to avoid hotspots in 1998: Carefully planning every detail of the storage layout.

Each disk drive consists of a spindle of spinning platters with read/write heads move back and forth. Each time you access a piece of data that’s not in cache, the drive moves its arm over the platter to access the correct piece of data. Since each drive can only access one piece of data at once, and since caches can only hold so much data, tuning a system to minimize the number of requests per drive is essential.

Manual storage array layout was an art, but we never fooled ourselves into thinking our designs were optimal. There were just too many intractable problems, so we had to compromise at every turn:

• We usually had no performance data to base our layout decisions on, so we had to rely on guesses and rules of thumb
• Workloads tend to change over time and manual layouts are painful to modify
• The smallest unit of allocation was an entire LUN or drive, so even the best disk layout mixed hot and rarely-accessed data everywhere
• Much of the allocated space was unused, so we used expensive disks to store nothing

One might think that, 10 years later, advances in technology would have solved these basic issues. But for many people using many of the so-called modern mainstream enterprise storage systems, these problems remain. Continue Reading »

I’ve been using Dreamhost as my hosting provider since 2000, mostly happily. But last year I began receiving enough traffic that I could no longer rely on shared hosting. Last Fall, I switched all of my domains to a virtual private server at Dreamhost - an easy upgrade that doubled my hosting bill but promised all the performance I would need. Sadly, this was not the case.

Dreamhost never could get their flexible resource provisioning control panel to work right with my server, and their support staff was only slightly responsive. I limped along for three months, waiting for them to get everything working right and suffering through two unexplained multi-hour server outages. After losing my server for an hour and a half yesterday without any response or explanation whatsoever from Dreamhost support, I decided that enough was enough. After nine years as a customer, I am moving my sites elsewhere.

I looked around for a rock-solid Xen VPS/Linux provider with high performance and affordable prices. After checking into many that were recommended, I narrowed my search to two: Linode and Slicehost. Both differ from Dreamhost in that they offer unconfigured servers rather than point-and-click convenience and management. But after almost 20 years in UNIX administration, I figured I was up to the job.

After examining the two, I decided that Slicehost, which was recently acquired by Rackspace, offered the best solution for me. One deciding factor was Linode’s failure to offer backup services of any kind - that’s just bizarre! When all is done, I will be spending just $5 more per month than my Dreamhost VPS cost, and will hopefully get much better performance and stability.

So I spent much of Friday evening setting up Ubuntu on my slice, copying data, configuring DNS, and installing lighttpd, php, MySQL, Wordpress, Mediawiki, and the rest. After some teething issues, I believe that everything is now up and running properly on my new host! Please let me know (by clicking the “Email Me” link in the upper left corner) if you see any errors or issues!

Does data encryption throw storage efficiency out the window? Not always!

One of the great ironies of storage technology is the inverse relationship between efficiency and security: Adding performance or reducing storage requirements almost always results in reducing the confidentiality, integrity, or availability of a system.

Many of the advances in capacity utilization put into production over the last few years rely on deduplication of data. This key technology has moved from basic compression tools to take on challenges in the fields of replication and archiving, and is even moving into primary storage. At the same time, interconnectedness and the digital revolution has made security a greater challenge, with focus and attention turning to encryption and authentication to prevent identity theft or worse crimes. The only problem is, most encryption schemes are incompatible with compression or deduplication of data! Continue Reading »

As I expected, EMC’s Iomega subsidiary today rolled out the StorCenter Pro ix4-100 a big brother to the popular but plain StorCenter ix2 NAS device. This new model add hot-swappable drives (there are four now), RAID-5, and a longer warranty to make it suitable for small office use. It uses a blacked-out version of the chassis from the non-LifeLine StorCenter Pro 150d.

Like the ix2, the Pro ix4 is certified for VMware ESX using NFS. iSCSI is still absent, though EMC folks have talked about adding this in short order.

This is the third member of the LifeLine-powered NAS family from Iomega: The confusingly-named single-drive Home Media Network Hard Disk Drive, the dual-drive StorCenter ix2, and this four-drive StorCenter Pro ix4-100. The “-100″ nomenclature suggests that more 4-drive models might be on the way, perhaps a “-200″ model sharing its rackmount chassis with the existing non-LifeLine StorCenterPro 200rl.

With NetApp exiting the small-business market by ending their StoreVault line, who would have thought big old EMC would be the one to step up? At $799 or $1,299 for the 2 and 4 TB versions, the ix4 is about the same price as a Drobo, which is $749 or $999 plus $199 for the DroboShare NAS.

Frankenweenie saves young Victor in Tim Burton's macabre short film

When is a solution integrated and when is it a Frankenstein-like mashup of tangled tech? Apparently, that line is crossed when it’s your competitor’s offering…

In my time in the storage industry, I’ve seen enough franken-storage come and go to make me skeptical whenever a new “integrated” solution is announced. But a lot of this stuff works just fine, so I also know that integrated solutions aren’t always bad!

The latest industry blog flame war centers around NetApp’s recently-announced solid state storage solution, which pairs a V-Series NAS head and a Texas Memory Systems RamSan-500 flash storage system. Perhaps NetApp’s Val Bercovici did get a bit over-excited in his post on the topic, but he wasn’t just talking about the RamSan: He was laying out how NetApp’s WAFL technology can work in an SSD world, and using some recent performance test numbers on that solution as well as their PAM cache cards as an illustration of this.

The next thing you know, we have EMC’s Storagezilla and IBM’s Barry Whyte calling the company out for what they (and others. like Storagebod) see as an underwhelming product offering. That’s all well and good, and I’ll let the reader decide if NetApp’s moves warranted a press release, but now things have gotten uglier… Continue Reading »

Storage Magazine has returned with an online edition

Although the dead-tree version of TechTarget’s excellent Storage Magazine is no more, the company today released the premiere issue of its online counterpart! Available as both a web-based magazine and a PDF download, Storage Magazine Online continues with many of the same editors and writers, including Rich Castagna at the helm.

You can still subscribe to the online edition too, so you needn’t miss an issue! Existing qualified subscribers should receive an email today outlining the shift from paper to bits.

One article I’d personally like to highlight is my own piece on consolidated archiving. I put a great deal of effort into the article, and I welcome your comments!

I wish the TechTarget all the best with this transition, and urge you all to check out their work. Now if they would only add all of the back issues as downloadable PDFs as well!

I usually don’t write about other peoples’ articles on this blog, preferring to stick to my own independent work. But this time I’m making an exception.

If you use or are interested in VMware ESX 3.x and iSCSI, you simply must go read Chad Sakac’s post on the topic. Co-written with just about everyone in the industry (including EMC, VMware, NetApp, Dell/EqualLogic, and HP/Lefthand), Chad has put together a “cheat sheet” on the ins and outs of iSCSI connectivity and performance in ESX 3.x.

Top takeaways (and I’ve been preaching about these for a while myself, too!)

• Ethernet link aggregation doesn’t buy you anything in iSCSI environments
• iSCSI HBA’s don’t buy you much other than boot-from-SAN in ESX, either
• The most common configuration (ESX software iSCSI) is limited to about 160 MB/s per iSCSI target over one-gigabit Ethernet, but that’s probably fine for most applications
• Adding multiple iSCSI targets adds performance across the board, but configurations vary by array
• Maximum per-target performance comes from guest-side software iSCSI, which can make use of multiple Ethernet links to push each array as fast as it can go

Thanks for putting together such a great article, guys!

Disk Inventory X is an amazing tool to zoom into your full disk and figure out what's taking up all the space!

I’m not an inherently tidy person. That’s one reason that I upgraded my MacBook Pro’s hard disk to 320 GB! But as a storage and data management consultant, I generally keep my computers much cleaner than my office.

So I was surprised to see a pop-up window on my Mac telling me that my hard drive was full! It was time to investigate data management applications for OS X. Luckily, I found some great tools, and these helped me isolate the space-wasting items quickly and easily.

Who’s To Blame?

When it comes to data storage, out of sight is usually out of mind. The primary culprit in the rampant over-saving of data is a lack of perspective and visualization: We tend to focus on the items we think are at fault rather than the true space wasters.

Consider your PC or Mac: Do you really know what is taking up all of your disk space? Unless you have a good tool, the answer is probably not. You might think that your “Documents” folder takes up most of the room, since you use it all the time and it has so many files in it. But even the bloated files produced by Microsoft Office pale in comparison to multimedia photo, music, and video files. And it is usually the folders that you don’t actively manage that are the worst space-wasters!

All operating systems have their secret corners that fill up with cruft, and many applications do a terrible job of cleaning up after themselves. In my PC days, I used to use Glary Utilities to clean out the corners of Windows and the awesome treemapping visualizations in WinDirStat to find where my storage was going. But since switching to my Mac I wasn’t aware of similar utilities. Continue Reading »

Google's first page accounts for more than two thirds of my web site traffic!

I’m not blogging to get traffic; I’m blogging because I have something to say. But, being a curious person, I do measure my blog traffic, and I’ve become interested in how the search engine optimization gurus ply their trade. So a recent tip on using a custom Google Analytics filter to determine “front page” placement on that search engine piqued my curiosity.

After just one week of monitoring, the results speak for themselves: Google’s first page of results accounts for more than two thirds of my web site traffic, and less than a quarter of my visits come from a source other than Google! It should come as no surprise that the search giant’s first page has come to dominate Internet traffic, but it was amazing for me to see it demonstrated so graphically using data from my own blog.

Since the first of the year, I’ve noticed that my traffic has been up by about 50%, too. I was puzzled, since the uptick comes across all topics and posts. My friend Louis Gray suggests that Google might be responsible for this as well: Google updated their Page Rank results, potentially moving many of my pages into the first page of results. Continue Reading »

I’ve been talking about storage capacity utilization for my entire career, but the storage industry doesn’t seem to be getting anywhere. Every year or so, a new study is performed showing that half of storage capacity in the data center is unused. And every time there is a predictable (and poorly thought through) “networked storage is a waste of time” response.

The good news is that this is no longer a technical problem: Modern virtualized and networked servers ought to have decent utilization of storage capacity, and technology is improving all the time. Consider the compounded impact of modern technology on storage capacity utilization:

• Shared storage (SAN and NAS) allows different servers to share a common pool of storage, reducing the likelihood that excess capacity will be stranded in isolated “puddles”. Pervasive use of NAS technology, and the rise of simple and inexpensive iSCSI SANs, means that every system in the modern data center can use shared storage.
• Organizational and architectural optimization allows storage to be provisioned from a common pool rather than building “stovepipe systems” with their own resources. Quicker provisioning also helps reduce over-provisioning.
• Network connectivity allows servers to share resources, including storage, on a peer-to-peer or client-server basis, ultimately resulting in things like cloud computing.
• Managed and utility services reduce the impact of low utilization, potentially focusing on efficiency or perhaps passing the buck to a service provider.
• Thin provisioning might help certain systems to keep less storage in reserve.

So why don’t things get better? It’s hard to be sure why people don’t use these pervasive tools to improve storage utilization, but I do have some ideas… Continue Reading »