Re: [PLUG] vmware and i.o

Since you mentioned a dev environment, the other thing that is common is to keep

adding more and more VM's to that environment without tracking over-commit.

VMware will let you overcommit CPU cores, and overcommit RAM on the host, and certainly

allow you to overcommit available disk IOPS to VM's ( Overallocate means that you can allocate 200+ cores

to VM's when you only have 24 real cores / 48 threads available.

In many dev environments most of the VM's are idle, and you can get away with overcommit, but

then if too many CPU cycles or RAM is needed elsewhere by previously idle VM's you get random

performance drops in your VM. Internal VM stats are not very useful here either.

There are design rules to follow for Dev vs Production, and monitoring tools etc. that will let you

see what is going on, and if you don't keep track of overcommit eventually the wheels will fall off.

Note that overcommit is not always bad - you can run more guest CPU and RAM on VMware on

the same host hardware than with other Hypervisors, but you need to be aware of the limitations.

Lee

From: "Lee H. Marzke" <lee@marzke.net>
To: "Philadelphia Linux User's Group Discussion List" <plug@lists.phillylinux.org>
Cc: rmorgan466@gmail.com
Sent: Wednesday, February 21, 2018 10:01:40 PM
Subject: Re: [PLUG] vmware and i.o

If you have only ESX there is very little monitoring available. If you have vSphere the vCenter
appliance provides a wealth of monitoring and performance statistics.

vCenter provides latency statistics for VM disks, and for the underlying physical LUN's as well
as throughput. Since it is not uncommon with physical disks for one VM to use more than it's
fair share of the disk - your problems may be from a neighbor VM and not your problem so you
really can't debug this from inside the VM.

VMware Enterprise Plus versions use Storage I/O control to throttle back I/O on any runaway VM's automatically
so this doesn't happen, and it makes sure each VM gets it's fair share.

Note: VMware support generally says that 80% of support calls for performance are ultimately tied back to disk
performance issues. You didn't mention if you were using local disk ( RAID level, number of spindles )
or iSCSI. Generally these days the trend is to move towards all-flash storage which eliminates many of
these storage problems as disk latencies often drop from 20-40ms down to 1-2 ms, and everything
works much smoother.

However all of this is the responsibility of your VMware administrator.

If you need specific help you can contact me off-list.

Lee

From: "Rita" <rmorgan466@gmail.com>
To: "Philadelphia Linux User's Group Discussion List" <plug@lists.phillylinux.org>
Sent: Wednesday, February 21, 2018 9:02:51 PM
Subject: [PLUG] vmware and i.o
at our lab we use vmware esx (i think) for virtualization. i feel like we are i/o bound for our processes. The process runtime varies too much but I can't provide if its i/o issue or not. The i.o wait percentage is sometimes at 0 and sometimes at 20%. I understand a lot of times I am using the page/buffers which is cached data.

Sometimes, I feel the underlying disks the VM is using is having problems but being a guest I can't provide that. Is there a way to prove that from a guest point of view?

--
--- Get your facts first, then you can distort them as you please.--

___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug

--
"Between subtle shading and the absence of light lies the nuance of iqlusion..." - Kryptos

Lee Marzke, lee@marzke.net http://marzke.net/lee/
IT Consultant, VMware, VCenter, storage, infrastructure, cloud migration options
+1 800-393-5217 voice/text
+1 484-348-2230 fax

___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug

"Between subtle shading and the absence of light lies the nuance of iqlusion..." - Kryptos

Lee Marzke, lee@marzke.net http://marzke.net/lee/
IT Consultant, VMware, VCenter, SAN storage, infrastructure, SW CM
+1 800-393-5217 voice/text

+1 484-348-2230 fax