Virtual Servers, Storage, and SAN – Why your servers are slow

Alright folks, this is a technical topic, but I’m going to try to explain it simply.  Bear with me!

If your organization has more than five servers, you are probably using a virtualization system like VMWare or Microsoft’s HyperV.  These systems give huge benefits in terms of high-availability, disaster recovery, and management ability.  But if they aren’t designed well, they tend to have resource problems.

All computer systems have four basic types of resources available to them which affect their performance.

These are kind of like muscles in a body.  If your tasks are too hard for the muscle, it  won’t be able to keep up with the work.  And if one set of muscles is overwhelmed (think of trying to swim with a weak arm), everything suffers.   The four basic resources are processor speed (CPU), fast memory (RAM), slow memory (hard disk), and network.

Most system administrators know all about processor speed and fast memory.  They know not to over-allocate the CPU or RAM.  There are clear views of exactly how much CPU and RAM you have and how much is being used, so they can see their usage.

But the hard drive – the slow memory – only displays capacity in terms of storage space – 400 GB used out of 1 TB.

Left out of the view is data transmission speed – which is made of two pieces… the communication path between your processor and the storage, and the speed of accessing the data itself.   This transmission speed is often the piece that degrades first when you have multiple virtual machines using it.

The rule of thumb is that you want the average transmission (both the read or write from the storage, as well as the network path to the processor) to take less than 10 milliseconds.   Personally, I aim for 3 milliseconds in a smoothly functioning system.

Once you start seeing 50 ms, 100 ms, or more, the virtual machines will be obviously “slow”

If you are running more than a handful of virtual machines, you will definitely need a professional Storage Area Network (SAN) solution such as Netapp, EMC, vSAN, or Hyperconverged Infrastructure.  This will increase the speed of accessing the data itself.  In addition, you need a fast communication path, AKA network.  These days, that communication path is a high speed storage switch such as Brocade or Cisco with matching high speed network cables.   iSCSI can run at speeds of 10gbps or 40gbps ethernet.  Fibre Channel can run at speeds of 8gbps, 16gbps, or faster.   1gbps Ethernet is not fast enough to handle multiple virtual machines – you need to look at 10gbps if this is your situation.

In summary, if your virtual machines are SLOW, check the disk transmission (latency) performance.

This is often the main bottleneck.    Specific performance measures are disk read latency,  disk write latency, disk read i/o, disk write i/o.   Isolate the bottleneck to either the storage device, or to your network.  Then upgrade the one (or both) that is causing the most pain.  Most medium sized businesses are fixed by upgrading to a professional SAN and installing dedicated 10gbps switches for the communication path.

As always, if you are near the Frederick MD, Baltimore MD, or Northern DC suburbs, I can help you with this upgrade.  Give Kieri Solutions a call!