Bad disk IO performance problem during reads and writes

Bad disk IO performance problem during reads and writes

Post by sl143 » Thu, 21 Oct 2004 02:28:23


I'm using V480s with A1000s as storage using Veritas DB edition,
solaris 8 running oracle.

We're seeing poor IO performance and I'm having trouble pinpointing
the problem. IO looks fine when it's mostly writes, but tanks the
moment reads increase:



Lookin' good:

r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t5d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t4d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t0d0
5.1 124.5 52.0 1154.4 0.2 0.8 1.5 6.3 2 23 c2t5d1
5.7 120.7 54.3 1124.0 0.4 0.8 2.9 6.4 2 22 c2t5d2
5.0 131.9 51.2 1212.8 0.2 0.9 1.4 6.3 2 24 c2t5d3
5.1 128.8 52.0 1188.8 0.4 0.9 2.8 6.4 3 24 c2t5d4
6.2 127.4 65.6 1177.6 0.5 1.0 3.6 7.7 3 27 c5t4d1
4.9 128.0 44.9 1180.2 0.2 1.0 1.4 7.5 2 26 c5t4d2
5.9 122.8 52.8 1139.2 0.3 0.9 2.4 7.1 3 26 c5t4d3
4.9 124.1 44.8 1151.2 0.2 0.8 1.6 6.5 2 24 c5t4d4


Crap-o-rific:

r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.4 0.0 0.2 0.0 0.0 0.0 0.0 0.0 0 0 c2t5d0
0.4 0.0 0.2 0.0 0.0 0.0 0.0 0.0 0 0 c5t4d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t0d0
47.8 5.4 531.3 172.8 0.9 4.9 17.6 92.5 16 51 c2t5d1
51.8 5.4 559.3 172.8 1.4 5.3 24.1 93.3 17 54 c2t5d2
48.1 5.8 532.9 176.0 0.9 4.7 16.1 87.0 16 52 c2t5d3
46.6 5.4 515.9 172.8 0.5 4.4 9.7 84.4 10 50 c2t5d4
51.3 6.0 545.7 182.4 2.5 8.3 43.2 144.8 32 84 c5t4d1
49.6 5.7 535.3 179.3 1.8 7.9 32.2 142.0 30 79 c5t4d2
52.5 6.2 560.9 183.2 1.9 6.9 32.2 117.1 26 74 c5t4d3
50.0 5.6 552.1 179.2 2.5 7.3 44.3 131.8 27 78 c5t4d4


The examples above are from the standby, and we see the same IO
profile on the primary.

I've checked write-cache, prefetch, read-ahead. I've looked for HW
problems, too.

Neither of these hosts have been touched for months -- I'm very
certain of that.


Thoughts appreciated.
 
 
 

Bad disk IO performance problem during reads and writes

Post by Michael Vi » Thu, 21 Oct 2004 03:04:19

In article < XXXX@XXXXX.COM >,



What are the volumes in Veritas? You've probably been told that RAID5
is a Bad Idea(tm) for write-heavy applications.

--
DeeDee, don't press that button! DeeDee! NO! Dee...