virtual grind thoughts from the virtual world

10Sep/140

Dell Compellent 6.4 Write Latency

Recently, I started seeing abnormal write latency across a number of pods in our environment. After spending a few hours looking at best practices and firmware on the compute and storage side, I decided to look at the switching layer. As expected, things like IO, buffers, buffer credits, etc. all looked fine.

Thinking it may be a cosmetic issue in Dell's Enterprise Manager, I engaged Compellent Support to have a look. After a week or so of analysis, I finally reached a senior engineer, who on a whim, asked me to test disabling SNMP at the controller level. In an instant, the abnormal write latency disappeared. What is alarming is that this was not just cosmetic, but actually felt all the way down to the host level.

I will say that I do not see this behavior in 6.3 but the change made a huge difference in 6.4.4.2 and 6.4.5.4. You can disable SNMP in Storage Center Manager by navigating to Storage Management -> System -> Access -> Configure SNMP Server, then click on "Stop Agent". If SNMP is not running, the link will say "Start Agent".

If you are running any FluidOS version 6.4.1 to 6.4.10, you may want to upgrade or disable SNMP and feel the difference!

Please note the change was made around 11AM, screenshots attached (click to view):

Controller View

Host View

Virtual Machine View

Comments (0) Trackbacks (0)

No comments yet.


Leave a comment

No trackbacks yet.