8440 Node Replacement Problems
Posted: Tue Jan 07, 2025 4:09 pm
Hi all!
I have an 8440 3Par that has lapsed support due to it being used for DEV work.
It's an 8440, 4 node SAN, running 3.3.1 MU5 OS and has encryption enabled w/ FIPS capable SED's.
Right before the holidays, we had node 2 fail. I had previously replaced a node on a different 3par that had an actual hardware failure. I believe I was successful because I was able to take the good node m.2 drive out of the bad node, and put it in a known good node, and presto, I was good, but this time, the node HDD definitely failed.
So, here is what I attempted, and where we are at now.
I took a known good node from a spare SAN that had been reset to BITB (same node #2 and same 3.3.1 MU5 code) and tried to use that. I connected to the mfg port, opened WHACK and ran the following commands:
Whack>unset sys_serial_10
Whack>unset w19
Whack>ahci init
Whack>ahci erase (this command seems to error out talking about a password?)
Whack>reset
After this completes, the node rebooted and came up, back to the login prompt.
I then issued a node rescue to node 2, which started, and then completed successfully. All of the messages were good, the rescue completed successfully and rebooted itself again.
After it ends up back at the login prompt, I am back on my ssh session into the SAN, running a shownode displays Nodes 0,1,3 and it shows them all as normal for a brief moment and then they all go to degraded. If i try to run another node rescue it tells me that node 2 is already part of the cluster. If I try to run admithw it tells me that node 2 isn't present.
The only relevant error I get in showalert is this:
Encryption is enabled on the system but the node drive is not encrypted (for node 2)
I am really at a loss here. What am I doing wrong? Does anyone have a step by step guide to follow?
PS, because of that ahci erase error, I bought a m.2 reader and completely removed all partitions and the read-only flag on a spare drive, I tried this entire process again with a completely bare drive, and the results were the same.
I have an 8440 3Par that has lapsed support due to it being used for DEV work.
It's an 8440, 4 node SAN, running 3.3.1 MU5 OS and has encryption enabled w/ FIPS capable SED's.
Right before the holidays, we had node 2 fail. I had previously replaced a node on a different 3par that had an actual hardware failure. I believe I was successful because I was able to take the good node m.2 drive out of the bad node, and put it in a known good node, and presto, I was good, but this time, the node HDD definitely failed.
So, here is what I attempted, and where we are at now.
I took a known good node from a spare SAN that had been reset to BITB (same node #2 and same 3.3.1 MU5 code) and tried to use that. I connected to the mfg port, opened WHACK and ran the following commands:
Whack>unset sys_serial_10
Whack>unset w19
Whack>ahci init
Whack>ahci erase (this command seems to error out talking about a password?)
Whack>reset
After this completes, the node rebooted and came up, back to the login prompt.
I then issued a node rescue to node 2, which started, and then completed successfully. All of the messages were good, the rescue completed successfully and rebooted itself again.
After it ends up back at the login prompt, I am back on my ssh session into the SAN, running a shownode displays Nodes 0,1,3 and it shows them all as normal for a brief moment and then they all go to degraded. If i try to run another node rescue it tells me that node 2 is already part of the cluster. If I try to run admithw it tells me that node 2 isn't present.
The only relevant error I get in showalert is this:
Encryption is enabled on the system but the node drive is not encrypted (for node 2)
I am really at a loss here. What am I doing wrong? Does anyone have a step by step guide to follow?
PS, because of that ahci erase error, I bought a m.2 reader and completely removed all partitions and the read-only flag on a spare drive, I tried this entire process again with a completely bare drive, and the results were the same.