Page 1 of 1
3par 7400 linux & oracle
Posted: Thu Nov 20, 2014 12:12 pm
by steveg
Hi,
We have just got our 1st 3par 7400 to run a oracle DB.
during our testing we never see any write merges on the multipath devices ( using deadline scheduler), for example:
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util
dm-3 0.00 0.00 0.00 2018.00 0.00 8.38 8.50 0.37 0.18 0.14 28.00
dm-5 0.00 0.00 0.00 1801.00 0.00 8.39 9.54 0.37 0.20 0.15 27.50
dm-11 0.00 0.00 0.00 3818.00 0.00 16.77 8.99 0.74 0.19 0.14 51.90
Has anyone else seen this behaviour ?
Thanks
Steve
Re: 3par 7400 linux & oracle
Posted: Tue Nov 25, 2014 10:53 am
by Richard Siemers
Linux is foreign to me, I suspect/believe "write merging" is the same thing as "write coalescing" in other vocabularies... which 3PAR ports have settings to enable/disable interupt coalescence, and in recent versions/patches have changed the default to be off. Maybe thats related?
Code: Select all
showport -par
N:S:P Connmode ConnType CfgRate MaxRate Class2 UniqNodeWwn VCN IntCoal
4:0:1 disk loop auto 4Gbps disabled disabled disabled enabled
4:0:2 disk loop auto 4Gbps disabled disabled disabled enabled
4:0:3 disk loop auto 4Gbps disabled disabled disabled enabled
4:0:4 disk loop auto 4Gbps disabled disabled disabled enabled
4:5:1 host point auto 4Gbps disabled disabled disabled disabled
4:5:2 host point auto 4Gbps disabled disabled disabled disabled
4:5:3 host point auto 4Gbps disabled disabled disabled disabled
4:5:4 host point auto 4Gbps disabled disabled disabled disabled
Re: 3par 7400 linux & oracle
Posted: Wed Nov 26, 2014 11:48 am
by steveg
Hi,
thanks, i'll take a look. my current thinking is that the linux kernel & device mapper is now obeying FLUSH and FUA commands and sendning them down to the 3par where on our old EVA devices and kernels it was being ignored somewhere along the OS I/O path.
Cheers
Steve
Re: 3par 7400 linux & oracle
Posted: Tue Dec 23, 2014 8:16 am
by steveg
Hi,
just to let everyone know in case they stumble into the same problem. HP upgraded the 3par from 3.1.3 to 3.2.1 P04 and that disabled the WCE bit on the presented SCSI devices and now we are seeing write merges and its looking quite a bit better from early results
Issue ID 104168 in the 3.2.1 release notes.
Re: 3par 7400 linux & oracle
Posted: Tue Dec 23, 2014 9:12 am
by afidel
steveg wrote:Hi,
just to let everyone know in case they stumble into the same problem. HP upgraded the 3par from 3.1.3 to 3.2.1 P04 and that disabled the WCE bit on the presented SCSI devices and now we are seeing write merges and its looking quite a bit better from early results
Issue ID 104168 in the 3.2.1 release notes.
Hmm, so they lie to the OS and say that they aren't caching writes. Not sure I'm a fan, sure it improves performance but it might lead to issues. I mean the write cache is supposed to be nonvolatile, but bugs happen and if the OS has failed to issue a flush because it's assuming there is no cache them that could lead to data corruption.
Re: 3par 7400 linux & oracle
Posted: Tue Dec 23, 2014 9:53 am
by steveg
I dont know if the particular version we had on the 3par was different to prior versions so it might just be a bug in that particular version of the 3par OS we hit.
Prior to 3.2.1 when running sdparm -g WCE against a 3par volume
WCE 1 [cha: n, def: 0, sav: 0]
now its
WCE 0 [cha: n, def: 0, sav: 0]
Our HP EVA's all report write cache disabled as well so maybe all vendors do it.
Re: 3par 7400 linux & oracle
Posted: Tue Dec 23, 2014 2:01 pm
by JohnMH
All vendors with a proper writeback cache solution will do this. The need to be able to Implement this is where the storage can't guarantee data and cache integrity through end to end integrity checks and a minimum of mirrored cache with destage to permanent media. i.e a non engineered roll your own solution.
If you were to obey this then you undermine much of 3PAR's design goal and may as well deploy some cheap JBOD storage.