[PLUG] I/O monitoring on Linux? At all?

[gabriel rosenkoetter <gr@eclipsed.net>]

Under Solaris, I've got iostat(1M), which can be told to repeat at
a granularity of 1 second or lower any number of times and, with
the -xcn flags, gives me output like this on a Sun E450:

         cpu
 us sy wt id
  3  7  4 86
                              extended device statistics
  r/s  w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 fd0
  0.0  0.0    0.3    0.4  0.0  0.0    0.0   28.1   0   0 c0t0d0
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    2.2   0   0 c0t1d0
  0.1  0.0    0.3    0.0  0.0  0.0    0.1    1.1   0   0 c1t6d0
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    1.7   0   0 c2t3d0
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c2t5d0
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c3t2d0
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c3t4d0
  0.3 14.1   18.5  882.4  0.0  0.2    0.0   12.6   0   5 c2t3d5
  0.3 14.1   18.5  882.3  0.0  0.2    0.0   13.2   0   5 c2t3d6
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c2t5d5
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c2t5d6
  0.3 14.0   18.5  882.2  0.0  0.2    0.0   12.7   0   5 c3t2d5
  0.3 14.0   18.5  882.3  0.0  0.2    0.0   13.3   0   5 c3t2d6
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c3t4d5
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c3t4d6
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c4t0d0
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c4t1d0
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    1.8   0   0 c4t8d0
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c4t10d0
  0.0 12.2    0.0  697.4  0.0  0.0    0.0    2.5   0   3 rmt/0
  0.0  2.1    0.0  107.8  0.0  0.0    0.0    1.8   0   0 rmt/1
  0.0  0.0    0.0    0.0  0.0  0.0    0.0    0.2   0   0 cyclone:vold(pid384)
 21.0 20.8   10.7   10.6  0.0  0.1    0.0    1.9   0   6 golf-ge0:/xy/bfd

On NetBSD, I've got iostat(8), which has similar features and with
the -x option gives me output like this on my crappy IA32
workstation:

device       KB/t      t/s     time     MB/s
fd0          1.92        0     0.00     0.00
sd0          6.26        3     0.02     0.02
cd0         57.52        0     0.00     0.00
md0          0.00        0     0.00     0.00

(Note that we're already going downhill here, as there's no way to
split out reads from writes. But it's something, at least.)

Under Linux I've got... pretty much nothing. I mean, there's
vmstat(8), which has a "block I/O" field. But that tells me across
every block device in the ENTIRE system, which, you can imagine,
is pretty much useless.

Sure, sure, there's sar(1). If I want to collect data for two days
and analyze it later. That's just swell, but how do I go about
debugging an I/O problem while it's *happening*?

As near as I can tell, especially based on some of the more
discouraging posts here:
  http://slashdot.org/askslashdot/99/06/14/225203.shtml
(particularly the third to last post, presuming they won't get
resorted in an archived discussion even if you're logged in, and I
don't think they will) there simply isn't any way to get the
information that I want when I want it, and that those who've forged
ahead and made a way have certainly not had the manpower to maintain
the changes.

Please tell me it ain't so! How can Linux hope to compete with a
real Unix if you can't do performance monitoring on something as key
as I/O? Who in their right mind would attach a RAID to a system so
crippled?

I *must* be missing something...

-- 
gabriel rosenkoetter
gr@eclipsed.net

Attachment: pgpGPRuJPD41N.pgp
Description: PGP signature