topic Re: Deploy HCI failed in HPE SimpliVity

Deploy HCI failed

louisdang — Sat, 20 Mar 2021 13:27:20 GMT

Hi,

i have problem with deploy node HCI, before i have deleted two card raid of simplivity and created new raid then i deploy omni.

Next, i use Deployment manager to install but test is oke, install is failed.

Pls help me.

Re: Deploy HCI failed

Rajini_Saini — Mon, 22 Mar 2021 07:33:37 GMT

Hi @louisdang,

Thank you for choosing HPE.

When you clear the Front and Rear drive raids, Please recreate only the Rear drives raid and leave the front drives without any raid configuration.
Also from the screenshot, it looks like a Factory reset issue.
Please confirm if you used the tiny deploy installer image or the Large deploy installer file for Factory reset.
You may refer to the factory reset procedure in the below link;
If using Tiny image: https://support.hpe.com/hpesc/public/docDisplay?docId=sf000067656en_us&docLocale=en_US
If using the large file, then please use a bootable USB key to factory reset the node [best practice]

regards,
Rajini Saini

Re: Deploy HCI failed

gustenar — Mon, 22 Mar 2021 19:27:44 GMT

@louisdang

Do you have anything connected to the back of that server, like a KVM? Also make sure the configuration for the rear disks is appropiate for the type of server you have.

Re: Deploy HCI failed

louisdang — Wed, 24 Mar 2021 08:20:24 GMT

Hi Rajini Saini,

Thank you for support. I deployed succesfull. But there was another error.

Actions completed:-

1.Putty in to the OVC.
2.svt-policy-create --name <Name of test policy>
3.svt-datastore-create --name <datastore_name> --policy <Name of test policy> --size [nGB|nTB].

Re: Deploy HCI failed

Rajini_Saini — Wed, 24 Mar 2021 08:38:28 GMT

Hi @louisdang,

Glad to know deployment was successful.

From the screenshot, I see that when you check for the cluster summary of the DR site you receive an error - Failed to authenticate with the server.
Please confirm if it's only for the DR cluster or all the clusters on site have the same issue.
Next, How many nodes in the DR site.
Vcenter in linked mode?

Login to anyone of the OVC from DR site and share the output of below commands, we will have to look into the logs;
cd /var/svtfs/0/log/
cat jauth.log | grep -i error
cat eventmgr.log | grep -i error

regards,
Rajini Saini

Re: Deploy HCI failed

louisdang — Fri, 26 Mar 2021 07:03:10 GMT

Hi Rajini Saini,

it's only for the DR cluster issue. DR Site have 3 nodes and Vcenter in linked mode.

Send you log:

2021-03-22T19:27:39.767Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator Correctable ECC Error Threshold
2021-03-22T19:27:39.778Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Virtual Controller using swap memory, error.
2021-03-22T19:27:40.359Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator ARM Correctable ECC Error Threshold
2021-03-22T19:27:40.448Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Virtual Machine Availability Zone Policy Compliance Error
2021-03-22T19:27:40.470Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator ARM DDR3 Correctable ECC Error
2021-03-22T19:27:40.491Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator PCIe correctable error
2021-03-22T19:27:40.531Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator uncorrectable ECC Error. Call Support
2021-03-22T19:27:40.641Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator Correctable ECC Error
2021-03-22T19:27:40.683Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator over temperature error has occurred.
2021-03-22T19:27:40.728Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Device Monitoring Error
2021-03-22T19:27:40.739Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Physical Drive Health State Error
2021-03-22T19:27:40.759Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity SSD Array Alert - Wear Level Errors
2021-03-22T19:27:40.790Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator firmware POST error
2021-03-22T19:27:40.811Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity SSD Array Alert - Wear Level Warnings and Errors
2021-03-22T19:27:40.963Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator firmware error. Call Support if restart fails.
2021-03-22T19:27:40.984Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator POST error. Call Support.
2021-03-22T19:27:41.006Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Logical Drive Health State Change Error
2021-03-22T19:27:41.017Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator firmware boot error
2021-03-22T19:27:41.036Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator DMA error event. System will restart.
2021-03-22T19:27:41.104Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Battery Backup Health Error
2021-03-22T19:27:41.145Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Omnicube Accelerator recovered from correctable soft memory error
2021-03-22T19:27:41.174Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator power supply error. Call Support.
2021-03-22T19:27:41.185Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator NVRAM persistence error. Call Support.
2021-03-22T19:27:41.195Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Storage adapter firmware uncorrectable error. Call Support.
2021-03-22T19:27:41.205Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator II error. Call Support.
2021-03-22T19:27:41.237Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity OmniCube Accelerator ARM DDR3 uncorrectable ECC Error. Call Support
2021-03-22T19:27:41.256Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Accelerator PCIe uncorrectable error. Call Support.
2021-03-22T19:27:41.285Z INFO pool-4-thread-1 [c.s.h.h.v.VMWareControllerImpl] setAlarms(VMWareControllerImpl.java:3291) - Created new alarm: SimpliVity Backup Availability Zone Policy Compliance Error
2021-03-22T19:27:42.822Z ERROR pool-4-thread-2 [c.s.u.n.ThriftConnector] connectTransport(ThriftConnector.java:124) - Connection to address x.x.x.x on port 9190 failed. ConnectException:Connection refused (Connection refused)
2021-03-22T19:27:42.828Z ERROR pool-7-thread-1 [c.s.u.n.ThriftConnector] connectTransport(ThriftConnector.java:124) - Connection to address x.x.x.x on port 9190 failed. ConnectException:Connection refused (Connection refused)
2021-03-22T19:33:16.750Z ERROR pool-2-thread-1 [c.s.d.s.i.s.DigitalVault] getById(DigitalVault.java:163) - Requested record not found
2021-03-22T19:33:16.766Z ERROR pool-2-thread-2 [c.s.d.s.i.s.DigitalVault] getById(DigitalVault.java:163) - Requested record not found
2021-03-22T19:33:18.773Z ERROR pool-2-thread-2 [c.s.d.s.i.s.DigitalVault] getById(DigitalVault.java:163) - Requested record not found
2021-03-22T19:33:18.775Z ERROR pool-2-thread-2 [c.s.d.s.i.s.DigitalVault] getById(DigitalVault.java:163) - Requested record not found
2021-03-22T19:38:50.395Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT 5191c240-ab47-4e67-a28e-93d4dded55cf, Session for ticket {SVT-T-TAG}4d22983d-d4d9-4cb0-9c0b-a61393af2877 has become invalid.
2021-03-22T19:38:52.385Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT 6f277c7e-f573-495c-98fc-ce4984ec4c9e, Session for ticket {SVT-T-TAG}197a4b4e-bcec-4321-8822-842748ead242 has become invalid.
2021-03-22T19:39:00.174Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT 83ae61a8-2282-48d4-a171-654b34c8a92d, Session for ticket {SVT-T-TAG}2b3c4152-dbf6-47a9-99ab-97049995d354 has become invalid.
2021-03-22T21:33:37.904Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT f429c0d0-bda7-43e4-b69e-de7507924dc9, Session for ticket {SVT-T-TAG}4832f5f3-da06-4263-ae3d-4a1799684838 has become invalid.
2021-03-22T21:33:38.227Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT f0e013d9-841d-44b5-9ead-1b70453f343f, Session for ticket {SVT-T-TAG}3dfbc98e-77a8-4182-ac53-866889d5b322 has become invalid.
2021-03-22T21:33:58.908Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT fbd94d03-aff6-4de5-b092-7d49413f65c6, Session for ticket {SVT-T-TAG}3d4864a6-e648-4582-a763-84d1d5b7e606 has become invalid.
2021-03-22T21:33:59.488Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT f3ebcffc-97af-4b41-b8b3-9e7e07af24f7, Session for ticket {SVT-T-TAG}d101a4bd-17de-44d3-b177-03bcc3fcbc7a has become invalid.
2021-03-22T22:06:35.484Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT 1627920f-4fb4-4850-9c76-62986b633597, Session for ticket {SVT-T-TAG}7a409cd8-159e-4491-9626-38cebadf4be8 has become invalid.
2021-03-22T22:06:35.823Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT 02200c36-e17a-47b3-9eef-5eae1f9f49b6, Session for ticket {SVT-T-TAG}bb96bc79-341c-47df-b7a3-76df54396adc has become invalid.
2021-03-22T22:06:54.506Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT 12c7bb37-ef85-4acd-a36b-eff099a91f81, Session for ticket {SVT-T-TAG}c76e131f-6df0-43f8-b427-3df6bffb145a has become invalid.
2021-03-22T22:06:55.068Z ERROR pool-3-thread-1 [c.s.j.s.s.BaseSessionControl] tickleHmsSessions(BaseSessionControl.java:512) - SLT 77edcd79-4cd7-402f-be49-3437a7694f39, Session for ticket {SVT-T-TAG}25195aea-c1cc-49b3-897c-0533027c0aaa has become invalid.
2021-03-22T22:29:39.681Z ERROR pool-4-thread-9 [c.s.j.s.s.VMwareSessionControl] authenticate(VMwareSessionControl.java:225) - SLT e8a9b98c-7bb3-4d14-b12e-d7c19846f656, No token for owner HMS: 10.225.44.100
2021-03-22T22:29:39.881Z ERROR pool-4-thread-10 [c.s.j.s.s.VMwareSessionControl] authenticate(VMwareSessionControl.java:225) - SLT 2074d099-da61-4687-af35-62d99d1ccb3e, No token for owner HMS: 10.225.44.100
2021-03-23T08:34:46.497Z ERROR pool-2-thread-1 [c.s.d.s.i.s.DigitalVault] getById(DigitalVault.java:163) - Requested record not found
2021-03-23T08:34:46.524Z ERROR pool-2-thread-1 [c.s.d.s.i.s.DigitalVault] getById(DigitalVault.java:163) - Requested record not found
2021-03-23T08:35:00.392Z ERROR pool-2-thread-1 [c.s.d.s.i.s.DigitalVault] getById(DigitalVault.java:163) - Requested record not found
2021-03-23T08:35:00.400Z ERROR pool-2-thread-1 [c.s.d.s.i.s.DigitalVault] getById(DigitalVault.java:163) - Requested record not found
2021-03-23T12:09:51.556Z ERROR pool-3-thread-1 [c.s.h.h.v.VMWareSessionImpl] connectOnce(VMWareSessionImpl.java:514) - Remote Exception occurred while connecting to vSphere Server to create a session.
2021-03-23T12:09:51.556Z ERROR pool-3-thread-1 [c.s.h.h.v.VMWareSessionImpl] connectOnce(VMWareSessionImpl.java:518) - Failed session ID: b88c6b240949407adba0da60d36809a1cff24602
2021-03-23T12:09:51.556Z ERROR pool-3-thread-1 [c.s.h.h.v.VMWareUtil] handleRemoteException(VMWareUtil.java:2926) - Type: 'java.rmi.RemoteException' Detail: 'VI SDK invoke exception in connection to https://x.x.x.x/sdk; nested exception is:
2021-03-23T12:09:51.628Z ERROR pool-3-thread-1 [c.s.h.h.v.VMWareSessionImpl] connect(VMWareSessionImpl.java:347) - Connect attempt 1 to HMS server https://x.x.x.x/sdk with type credentialSessionId error: Type: 'java.rmi.RemoteException' Detail: 'VI SDK invoke exception in connection to https://x.x.x.x/sdk; nested exception is:
2021-03-23T12:09:54.645Z ERROR pool-3-thread-1 [c.s.h.h.v.VMWareSessionImpl] connectOnce(VMWareSessionImpl.java:514) - Remote Exception occurred while connecting to vSphere Server to create a session.

Re: Deploy HCI failed

Rajini_Saini — Wed, 24 Mar 2021 08:56:54 GMT

Hi @louisdang ,

Thank you for the logs. [You may mask the IPs in the logs if required]

I see a lot of authentication errors, between the nodes/OVC to the Vcenter.
Hence request you to log a SimpliVity support case and they will be able to help you further with detailed log analysis and correct the authentication issue using elevated command access on the OVCs.

regards,
Rajini Saini

Re: Deploy HCI failed

louisdang — Wed, 24 Mar 2021 09:00:55 GMT

Hi Rajini Saini,

Current i sent request to HPE support, but I have not received help yet.

Can you support me how to fix step by step?

Re: Deploy HCI failed

Rajini_Saini — Wed, 24 Mar 2021 09:44:27 GMT

Hi @louisdang,

Please help me with the SimpliVity support case number.

regards,
Rajini Saini

Re: Deploy HCI failed

louisdang — Wed, 24 Mar 2021 09:46:52 GMT

Hi Rajini Saini,

My HPE Support Case is 5354396327.

Can you help me?

Re: Deploy HCI failed

Rajini_Saini — Wed, 24 Mar 2021 12:57:04 GMT

Hi @louisdang,

Thank you for the case details. I received an update that one of our Support engineers will be sharing the meeting link with you.

regards,
Rajini Saini

Re: Deploy HCI failed

louisdang — Fri, 26 Mar 2021 07:16:14 GMT

Hi Rajini Saini,

Thank you support, someon has support me but can't resolve. Current, We have elevated the case to Level 3 and we are awaiting for an update from them but After 24 hours have passed, I still have not received your support from level 3. Can you urge them to help me?

Thank you very much.