HP Network Protector Service Unavailable

[SHawkins]

Hi Guys!

Has anyone else seen the error message

"HP Network Protector Service Unavailable"

"Backing Service has become unavailable"

"Application or service modules may have stopped or undeployed"

This happened to my Network Protector app after leaving it alone for a couple of weeks and not logging in to it or changing anything.

That error message pops over the UI, and is the very first thing I see after I log in to the Network protector app.

Any ideas how to resolve it?

I tried restarting the server that the VAN is on.

I also tried uninstalling and reinstalling the Network Protector app.

I've included a few screenshots.

[sdnindia]

Hello SHawkins,

“Service Unavailable" or  “may have stopped” or “undeployed" occur when either the application is uninstalled or it’s not running.

Please make sure the application is installed and service is running.

Are you still facing the problem even after restarting / reinstalling?

Thanks,

HP SDN Team

[EricAtHP]

I have seen this happen when the server is assigned less than 4GB of RAM too.

Check the protector service: "service sentunneld status"

[SHawkins]

Hi guys!

Yes - I did try restarting, and also tried uninstalling and reinstalling the app.

It still doesn't work.

Also - the VM I'm using has 16GB RAM and do show that the sentunneld is running.

I've got a HP support case open - Case#4649577762 - but I'm not sure exactly what's going on with it.

tadmin@MI-VM-HP-VAN:~$ free
total used free shared buffers cached
Mem: 16434636 2665384 13769252 13464 204324 999008
-/+ buffers/cache: 1462052 14972584
Swap: 16777212 0 16777212
tadmin@MI-VM-HP-VAN:~$ service sentunneld status
sentunneld start/running, process 900
tadmin@MI-VM-HP-VAN:~$

[SHawkins]

It's Network protector v1.1.12.1433 - running on VAN 2.3.5.

tadmin@MI-VM-HP-VAN:~$ sudo service sdna status
[sudo] password for tadmin:
sdna start/running, process 893

tadmin@MI-VM-HP-VAN:~$ sudo service sdnc status
sdnc start/running, process 890

I can share the "all logs" file privately if anyone would like to take a look.

I've sent if over to HP support last Tuesday. I think the engineer that was working on my case elevated it to L3 on Wednesday, but is out of the office this week, so I don't really know where things stand.

I've sent an e-mail in to the HP global support e-mail to see if I can get the ball moving again.

[Oliver Wehrli · Accepted solution]

Hi SHawkins,

This problem can occur when the process of the Cassandra database is not running.

You can verify the status of the process by running the following shell script as root user:

sudo /opt/sdn/cassandra/bin/caServer.sh status

If the service is reported up and running and the error persists, try manually restarting the database daemon

sudo /opt/sdn/cassandra/bin/caServer.sh restart

Hope that helps.

[SHawkins]

Thanks Oliver! Also, thank you previous posters for your suggestions!

Starting Cassandra, uninstalling the NetworkProtector App & Reinstalling it with Cassandra actually running fixed it.

It's really weird why Cassandra wouldn't have started on it's own, even after a reboot, and why it stopped in the first place.

That got me going though :-)

tadmin@MI-VM-HP-VAN:~$ sudo /opt/sdn/cassandra/bin/caServer.sh status
[sudo] password for tadmin:
Checking status of Cassandra daemon......
could not access pidfile for Cassandra

tadmin@MI-VM-HP-VAN:~$ sudo /opt/sdn/cassandra/bin/caServer.sh start
Starting Cassandra daemon......

[sdnindia]

Hello SHawkins,

Doing a follow up to check if you need any further assistance with respect to the query posted.
Please do let us know  if your problem is solved.

If you have more questions on the same topic please do reply on the same thread or open a new post if new topic.

Thanks,
HP SDN Team

[ldw643]

Hi, I am a student ICT and I had the same problem. Sow I started searching and found this:

Installation guide network protector (just click on link)

and go to page 19 where you find this:

A.8 The controller ceases to operate when running Network Protector
A.8.1 Problem description
In a controller installation supporting the Network Protector application, if the Cassandra service
fails, Network Protector and the controller cease to operate.

A.8.2 Solution
Restart the Cassandra service:
1. Verify that the SDNC service is started (sudo service sdnc status). You should see
the following output:
sdnc start/running, process nnnn

2. Use the following command to determine whether Cassandra is running:
ps –ef | grep –i Cassandra
• If Cassandra is not running, you will see output similar to the following:
sdn 16941 16487 0 15:01 pts/0 00:00:00 grep --color=auto cassandra
In this case, go to step 3.
• If Cassandra is running, output similar to the following example appears, which indicates
that Cassandra is not causing the controller to halt.
sdn 22162 1 6 15:09 ? 00:00:05 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -ea
-javaagent:/opt/sdn/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities
-XX:ThreadPriorityPolicy=42 -Xms1024M -Xmx1024M -Xmn256M -XX:+HeapDumpOnOutOfMemoryError
-Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled
-XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB -XX:+UseCondCardMark
-Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199
-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
-Dlog4j.configuration=log4j-server.properties -Dlog4j.defaultInitOverride=true
. . .
org.apache.cassandra.service.CassandraDaemonsdn 22431 18086 0 15:11 pts/0 00:00:00 grep
--color=auto Cassandra

3. If Cassandra is not running, disable Network Protector on the controller.

4. If there is no Cassandra process running, then determine whether the Cassandra .pid file is
present in /opt/sdn/cassandra/data/cassandra.pid.
• If the .pid file is present, delete it (rm –rf
/opt/sdn/cassandra/data/cassandra.pid) and go to step 5.
• If the .pid is not present, go to step 5.

5. Change to the Cassandra bin directory:
cd /opt/sdn/cassandra/bin

6. Use this command to restart the Cassandra server:
./caServer.sh start

7. Use this command to verify that Cassandra has started:
./caServer.sh status

You should then see output similar to the following:
Checking status of Cassandra daemon......Cassandra is running

8. Restart the Network Protector application from the controller

Now it should work :)

Let me know if it is not, but that did it for me. I used the HP VAN SDN controller v2.4 (released since begin this month I thought)

[Oliver Wehrli]

I ran into this issue before as well and I can confirm that the above guidelines will resolve the "Service unavailable" message under the described circumstances.

[Payton]

I installed the sdn controller version 2.5.15 with the network protetor version 1.3.13. I have the same error "HP Network Protector Service Unavailable". I tried the steps from the post above.

- the service sentunneld is running

- cassandra database is running

ny ideas how to resolve it?

[Joetel]

Hi Payton,

Just wanted to ask whether your Network Protector installation did function for some time after installation and then ceased to work, or that it didn't function directly after installation?

Best regards,

Wouter

HP SDN CoE Team

[Payton]

Hy Wouter

The Network Protector works fine for some time after the installation.

Best regards

[Joetel]

Hi Payton,

And Casandra is running, correct? Could you in that case (again) perform the trouble shooting procedure in the HP Network Protector SDN Application - 1.3 Installation Guide , appendix A.5, page 12 and post the output of steps? Thanks!

Best regards,

Wouter

HP SDN CoE Team

[Payton]

Hy Wouter

Cassandra is running correct:

$ sudo /opt/sdn/cassandra/bin/caServer.sh status

....

+ echo 'Cassandra is running'
Cassandra is running
+ exit 0

Troubleshooting A.5

$ ps -ef | grep cassandra | grep -v grep | wc -l
1

[Joetel]

Hi Payton,

Hmm. Can you also do the casandra restart (A6) and post the output back? Thanks.

Best regards,

Wouter

HP SDN CoE team

[Payton]

Hy Wouter

$ sudo service sdnc status
sdnc start/running, process 822

---

opt/sdn/cassandra/bin$ sudo ./caServer.sh stop

......

+ echo 'Cassandra daemon stopped successfully'
Cassandra daemon stopped successfully
+ exit 0

---

opt/sdn/cassandra/bin$ sudo ./caServer.sh start

.....
+ echo 'Cassandra daemon started successfully'
Cassandra daemon started successfully
+ exit 0

---

opt/sdn/cassandra/bin$ sudo ./caServer.sh status

+ stat=0
+ case "$stat" in
+ echo 'Cassandra is running'
Cassandra is running
+ exit 0

It is still not working....

I have this warning in the network-protector.log:

2015-07-06 16:51:10.472 WARN  [http-bio-8443-exec-7] AccessFilter : SentinelServiceManager is unavailable at the moment - probably some OSGi dependencies are not resolved

and the Error Logs in the Support Logs (Printscreen)

Best regards,

[Joetel]

Hi Payton,

The errors about the token from the support log can sometimes be caused by having a browser session open on the controller for so long a time that the token expires. Assuming that you know where were they all come from, could you close all browser sessions to the controller and see if that makes those support log messages go away? Thanks.

Best regards,

Wouter

[Payton]

Hy Wouter

Yes, you are right. I closed all Browser Session and the Log message went away.

But I have still the Problem with the HP Network Protector app.

I have already reinstall the sdn controller with the app.

Do you have any ideas?

[Payton]

I reinstall the controller and now I can access the Network Protector App.

I can't find some devices in the Network Protector App. The Switches are correkt implement under the OpenFlow Topology.

Do you have any ideas?

There are no informationen about it in the sdn & switch log.

[EricAtHP]

Usually when the controller sees a switch and Protector doesn't, it is related to SNMP. Double check that the SNMP credentials entered into Protector match the switch configuration.