service time/latency difference

Post Reply
mohuddle
Posts: 62
Joined: Thu May 08, 2014 4:43 pm

service time/latency difference

Post by mohuddle »

4 node, 8 host ports, 8 disk ports, 8 cage 7400, mostly FC ( 160 ) with some SSD for a handful of AO LUNs.

statpd -d 5 -p -devtype FC
Noticed during some benchmarks for some new software that disk latency was 8ms - 10ms ( Cur and Avg, total ). Now it's 11ms - 13ms.

statvlun -vvsum -ni -rw -d 5
Looking at all LUN's ( well, all active LUNs ) for total, it's only 1.65ms to 1.25ms .

statport -rw -ni -disk -d 5
Also, looking at the ports I see 9.01ms - 6.16ms ( Cur - Avg for total ).

Finally, looking at the subset of LUNs doing the actual benchmark, they are 3.39ms to 1.92ms and floated up to 6ms and even 10ms and a little higher at times.

All that to ask: when looking at all PDs, all VVs and all Ports, why aren't they a bit closer together in their service times?

Any cause for concern when PDs ( or VVs or Ports ) get over a certain time?
apol
Posts: 267
Joined: Wed May 07, 2014 1:51 am

Re: service time/latency difference

Post by apol »

Hi,

as far as I know,

PD-times show the bare metal service times from the discs, which means all IOPS covered by cache are not included here.

VV-Service-time is local storage-view (not sure about what's included here or not)

vlun-servicetime is closer to what the server sees, including time spent for mirroring, acknowledging IOPS from the server, maybe retransmits because of san-problems etc.

port-servicetimes: not sure where this fiits...
When all else fails, read the instructions.
JohnMH
Posts: 505
Joined: Wed Nov 19, 2014 5:14 am

Re: service time/latency difference

Post by JohnMH »

As above

Statvlun should provide a very similar view to the host, statport -host should provide similar albeit an aggregate view, so look at these as front end which benefit from cache, anything else is backend so you'll typically see higher latencies.

Statpd shows how heavily loaded the backend disks are taking into account the write amplification required for raid. In your case with statport you're looking at all I/O traversing the backend disk ports inc raid overheads.
Post Reply