HPE EVA Storage

EVA 6400 never failed over

 
JimD4231
Occasional Contributor

EVA 6400 never failed over

Hi Everyone,

         Our primary SAN had some disk groups go off-line last week but they never failed over to the secondary SAN. These are both EVA 6400 units. We replicate each disk group synchronously. Here is some of the log when the SAN had issues. Thank you for your help.

 

Aug/28/2017 11:58:34.685880, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 141, SCMI command FAILED
Aug/28/2017 11:58:34.686004, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 142, WWN: 5000-1FE1-501E-AB90 Serial: PAFUCC29SZC00P Path: \\.\Scsi1:-0-0-0
Aug/28/2017 11:58:34.686110, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 146, Cmd: 6023-SCMI_F_PHYSICAL_STORE_GET_DOWNLOAD_CONDITION Result: SCMI_S_LDAD_DOWNGRADED
Aug/28/2017 11:58:34.686198, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 151, Cmd.pver: 21100 Cmd.exec: 1 Cmd.cver: 0 Cmd.class: 6
Aug/28/2017 11:58:34.686301, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 156, Cmd.fn: 6023 Cmd.ctx: 0 Cmd.hndl: 2044-07-10-20000024-B65666BB-00000000-00000000
Aug/28/2017 11:58:34.686416, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 167, 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Aug/28/2017 11:58:34.686515, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 170, Rsp.pver: 21100 Rsp.mas: Master Rsp.exec: 1 Rsp.cver: 4 Rsp.status: 215 Rsp.qual: 0
Aug/28/2017 11:58:34.686590, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 177, Rsp.ctx: 0 Rsp.datlen: 0
Aug/28/2017 11:58:34.686677, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 186, 00000000 00000000 00000000 00000000 00000000 00000000
Aug/28/2017 11:58:34.686759, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 189, Cmd_cookie: 0 Rsp_cookie: 0 Rsp_timeout: 120
Aug/28/2017 11:58:34.686825, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 195, Stack trace:
Aug/28/2017 11:58:34.686891, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 196,
Aug/28/2017 11:58:34.686957, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 197, End of stack trace
Aug/28/2017 11:58:34.687143, cveva, 1444, 6624, 1, sma_ps, 350, ScmiMgmtApi::ps_getDownloadCondition() command failed! (215) Recontructing or reverting is in progress.
Aug/28/2017 11:58:34.689605, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 141, SCMI command FAILED
Aug/28/2017 11:58:34.689714, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 142, WWN: 5000-1FE1-501E-AB90 Serial: PAFUCC29SZC00P Path: \\.\Scsi1:-0-0-0
Aug/28/2017 11:58:34.689822, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 146, Cmd: 6023-SCMI_F_PHYSICAL_STORE_GET_DOWNLOAD_CONDITION Result: SCMI_S_LDAD_DOWNGRADED
Aug/28/2017 11:58:34.689911, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 151, Cmd.pver: 21100 Cmd.exec: 1 Cmd.cver: 0 Cmd.class: 6
Aug/28/2017 11:58:34.690013, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 156, Cmd.fn: 6023 Cmd.ctx: 0 Cmd.hndl: 2044-07-10-20000024-B65666BB-00000000-00000000
Aug/28/2017 11:58:34.690129, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 167, 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Aug/28/2017 11:58:34.690227, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 170, Rsp.pver: 21100 Rsp.mas: Master Rsp.exec: 1 Rsp.cver: 4 Rsp.status: 215 Rsp.qual: 0
Aug/28/2017 11:58:34.690302, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 177, Rsp.ctx: 0 Rsp.datlen: 0
Aug/28/2017 11:58:34.690390, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 186, 00000000 00000000 00000000 00000000 00000000 00000000
Aug/28/2017 11:58:34.690471, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 189, Cmd_cookie: 0 Rsp_cookie: 0 Rsp_timeout: 120
Aug/28/2017 11:58:34.690538, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 195, Stack trace:
Aug/28/2017 11:58:34.690604, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 196,
Aug/28/2017 11:58:34.690670, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 197, End of stack trace
Aug/28/2017 11:58:34.690847, cveva, 1444, 6624, 1, sma_ps, 350, ScmiMgmtApi::ps_getDownloadCondition() command failed! (215) Recontructing or reverting is in progress.
Aug/28/2017 11:58:37.287602, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 141, SCMI command FAILED
Aug/28/2017 11:58:37.287724, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 142, WWN: 5000-1FE1-501E-AB90 Serial: PAFUCC29SZC00P Path: \\.\Scsi1:-0-0-0
Aug/28/2017 11:58:37.287828, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 146, Cmd: 6023-SCMI_F_PHYSICAL_STORE_GET_DOWNLOAD_CONDITION Result: SCMI_S_LDAD_DOWNGRADED
Aug/28/2017 11:58:37.287914, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 151, Cmd.pver: 21100 Cmd.exec: 1 Cmd.cver: 0 Cmd.class: 6
Aug/28/2017 11:58:37.288016, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 156, Cmd.fn: 6023 Cmd.ctx: 0 Cmd.hndl: 2044-07-10-20000024-B65666BB-00000000-00000000
Aug/28/2017 11:58:37.288131, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 167, 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Aug/28/2017 11:58:37.288228, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 170, Rsp.pver: 21100 Rsp.mas: Master Rsp.exec: 1 Rsp.cver: 4 Rsp.status: 215 Rsp.qual: 0
Aug/28/2017 11:58:37.288302, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 177, Rsp.ctx: 0 Rsp.datlen: 0
Aug/28/2017 11:58:37.288389, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 186, 00000000 00000000 00000000 00000000 00000000 00000000
Aug/28/2017 11:58:37.288469, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 189, Cmd_cookie: 0 Rsp_cookie: 0 Rsp_timeout: 120
Aug/28/2017 11:58:37.288534, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 195, Stack trace:
Aug/28/2017 11:58:37.288600, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 196,
Aug/28/2017 11:58:37.288681, cv_scmi, 1444, 9704, 1, ScmiTransportManager, 197, End of stack trace
Aug/28/2017 11:58:37.288879, cveva, 1444, 9704, 1, sma_ps, 350, ScmiMgmtApi::ps_getDownloadCondition() command failed! (215) Recontructing or reverting is in progress.
Aug/28/2017 11:58:56.112330, cveva, 1444, 1668, 2, sob_discovery, 175, SOB discovery
Aug/28/2017 11:58:56.112505, cveva, 1444, 1668, 2, sob_discovery, 182, ******************************************************************
Aug/28/2017 11:58:56.112634, cveva, 1444, 1668, 2, sob_discovery, 183, *
Aug/28/2017 11:58:56.112772, cveva, 1444, 1668, 2, sob_discovery, 184, * Starting SOB Discovery Cycle
Aug/28/2017 11:58:56.112891, cveva, 1444, 1668, 2, sob_discovery, 185, *
Aug/28/2017 11:58:56.113045, cveva, 1444, 1668, 2, sob_discovery, 186, ******************************************************************
Aug/28/2017 11:58:56.131627, ccb2_fsscsi, 1444, 1668, 0, dev_win, 224, Device has: 8 busses. 0 ms
Aug/28/2017 11:58:56.131887, ccb2_fsscsi, 1444, 1668, 0, dev_win, 224, Device has: 8 busses. 0 ms
Aug/28/2017 11:58:56.132505, ccb2_fsscsi, 1444, 1668, 0, dev_win, 224, Device has: 8 busses. 0 ms
Aug/28/2017 11:58:56.133415, ccb2_fsscsi, 1444, 1668, 0, dev_win, 224, Device has: 1 busses. 0 ms
Aug/28/2017 11:59:17.279723, cveva, 1444, 1668, 2, sob_discovery, 399, ******************************************************************
Aug/28/2017 11:59:17.279831, cveva, 1444, 1668, 2, sob_discovery, 400, * Config
Aug/28/2017 11:59:17.279898, cveva, 1444, 1668, 2, sob_discovery, 401, *
Aug/28/2017 11:59:17.282479, cveva, 1444, 1668, 2, sob_discovery, 417, * Node Array Ctlr SCMI Ping Init Pass Pass Mnged Disc Mast Firmware
Aug/28/2017 11:59:17.282596, cveva, 1444, 1668, 2, sob_discovery, 417, * WWN Name Type Lock set knwn here rung path version
Aug/28/2017 11:59:17.282705, cveva, 1444, 1668, 2, sob_discovery, 417, * ------------------- ---------- ------ ---- ---- ---- ---- ---- ----- ---- ---- -------------------
Aug/28/2017 11:59:17.282814, cveva, 1444, 1668, 2, sob_discovery, 417, * 5000-1FE1-501E-AB90 XBE1C01SN1 HSV400 No Yes Yes No No Yes No Yes CR18CBdelp-09534000
Aug/28/2017 11:59:17.282922, cveva, 1444, 1668, 2, sob_discovery, 417, * 5000-1FE1-501E-AE10 XBE1C02SN1 HSV400 No Yes Yes No No Yes No Yes CR18CBdelp-09534000
Aug/28/2017 11:59:17.282991, cveva, 1444, 1668, 2, sob_discovery, 417, *
Aug/28/2017 11:59:17.283088, cveva, 1444, 1668, 2, sob_discovery, 417, * Path Vendor Product Product Node Serial Valid
Aug/28/2017 11:59:17.283189, cveva, 1444, 1668, 2, sob_discovery, 417, * Rev WWN Number
Aug/28/2017 11:59:17.283290, cveva, 1444, 1668, 2, sob_discovery, 417, * ---------------- ------ ------- ------- ------------------- -------------- -----
Aug/28/2017 11:59:17.283390, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi1:-0-0-0 HP HSV400 9534000 5000-1FE1-501E-AB90 PAFUCC29SZC00P Valid
Aug/28/2017 11:59:17.283490, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi1:-0-4-0 HP HSV400 9534000 5000-1FE1-501E-AB90 PAFUCC29SZC00P Valid
Aug/28/2017 11:59:17.283591, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi2:-0-0-0 HP HSV400 9534000 5000-1FE1-501E-AB90 PAFUCC29SZC00P Valid
Aug/28/2017 11:59:17.283691, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi2:-0-5-0 HP HSV400 9534000 5000-1FE1-501E-AB90 PAFUCC29SZC00P Valid
Aug/28/2017 11:59:17.283792, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi1:-0-1-0 HP HSV400 9534000 5000-1FE1-501E-AB90 PAFUCB49SY8036 Valid
Aug/28/2017 11:59:17.283893, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi1:-0-5-0 HP HSV400 9534000 5000-1FE1-501E-AB90 PAFUCB49SY8036 Valid
Aug/28/2017 11:59:17.283993, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi2:-0-1-0 HP HSV400 9534000 5000-1FE1-501E-AB90 PAFUCB49SY8036 Valid
Aug/28/2017 11:59:17.284093, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi2:-0-4-0 HP HSV400 9534000 5000-1FE1-501E-AB90 PAFUCB49SY8036 Valid
Aug/28/2017 11:59:17.284194, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi1:-0-2-0 HP HSV400 9534000 5000-1FE1-501E-AE10 PAFUCC29SZC02Q Valid
Aug/28/2017 11:59:17.284294, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi1:-0-6-0 HP HSV400 9534000 5000-1FE1-501E-AE10 PAFUCC29SZC02Q Valid
Aug/28/2017 11:59:17.284394, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi2:-0-2-0 HP HSV400 9534000 5000-1FE1-501E-AE10 PAFUCC29SZC02Q Valid
Aug/28/2017 11:59:17.284494, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi2:-0-6-0 HP HSV400 9534000 5000-1FE1-501E-AE10 PAFUCC29SZC02Q Valid
Aug/28/2017 11:59:17.284599, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi1:-0-3-0 HP HSV400 9534000 5000-1FE1-501E-AE10 PAFUCC29S3I00U Valid
Aug/28/2017 11:59:17.284700, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi1:-0-7-0 HP HSV400 9534000 5000-1FE1-501E-AE10 PAFUCC29S3I00U Valid
Aug/28/2017 11:59:17.284800, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi2:-0-3-0 HP HSV400 9534000 5000-1FE1-501E-AE10 PAFUCC29S3I00U Valid
Aug/28/2017 11:59:17.284901, cveva, 1444, 1668, 2, sob_discovery, 417, * \\.\Scsi2:-0-7-0 HP HSV400 9534000 5000-1FE1-501E-AE10 PAFUCC29S3I00U Valid
Aug/28/2017 11:59:17.284969, cveva, 1444, 1668, 2, sob_discovery, 417, *
Aug/28/2017 11:59:17.285031, cveva, 1444, 1668, 2, sob_discovery, 420, *
Aug/28/2017 11:59:17.285111, cveva, 1444, 1668, 2, sob_discovery, 421, ******************************************************************
Aug/28/2017 11:59:17.285286, cveva, 1444, 1668, 2, sob_discovery, 244, ******************************************************************
Aug/28/2017 11:59:17.285364, cveva, 1444, 1668, 2, sob_discovery, 245, * Stats
Aug/28/2017 11:59:17.285434, cveva, 1444, 1668, 2, sob_discovery, 248, * CCB object counts:
Aug/28/2017 11:59:17.285524, cveva, 1444, 1668, 2, sob_discovery, 275, ******************************************************************
Aug/28/2017 11:59:17.285597, cveva, 1444, 1668, 2, sob_discovery, 232, SOB discovery complete
Aug/28/2017 11:59:24.954959, cveva, 1444, 10172, 1, ApLicInfra, 1034, Error in OvLicGetLicenseDetail 149155000-1FE1-501E-AB90
Aug/28/2017 11:59:25.078588, cveva, 1444, 10172, 1, ApLicInfra, 1034, Error in OvLicGetLicenseDetail 105000-1FE1-501E-AB90
Aug/28/2017 11:59:25.355781, cveva, 1444, 10172, 1, ApLicInfra, 1034, Error in OvLicGetLicenseDetail 14175
Aug/28/2017 11:59:25.642713, cveva, 1444, 10172, 1, ApLicInfra, 1034, Error in OvLicGetLicenseDetail 14161
Aug/28/2017 11:59:30.972846, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 141, SCMI command FAILED
Aug/28/2017 11:59:30.972961, cv_scmi, 1444, 6624, 1, ScmiTransportManager, 142, WWN: 5000-1FE1-501E-AB90 Serial: PAFUCC29SZC00P Path: \\.\Scsi1:-0-0-0