V3000 Continues to lock up

[helper_1]

I have posted about this customer previously.  This customer has a V3000 system that was showing signs of corruption (ie. Voice mail ports with duplicate extension numbers, attempt to remove unused extensions and system would lock up, voice mail locks up, system locks up with no GUI). The issue that is still going on is that user will attempt to log into their voice mail and the call will terminate after entering their password then # (ie extension presses voice mail button or dials 500 **, asked to enter voice mail password, caller presses their password and #, system disconnects call immediately).  Also system stops responding to ping, GUI will not open and FTP will not connect.  With all of that, the AA and ACD greeting, all calling (internal as well as external) work properly. 

Action taken:

Database was rebuilt on new hard drive. System continue to have voice mail and GUI lock up.

Patch for 6_5_22 (P03) was installed.  System continue to have voice mail and GUI lock up.

Hard drive, hard drive tray and memory moved to another system.  System continue to have voice mail and GUI lock up.

Save core and "i" command were captured.

[helper_1]

Examples of NBossLog

1101:082111:0019 Call           W connect - Conference request but no conference address was reserved                                                     
1101:082200:0057 VoiceMail5     E IMAPNewMessage::deliver: Could not open socket connection - error 55. Delivery to local smtpd failed                    
1101:082252:0634 CallGroup      W Call Group Device :CFWDALL enabled on primary do not ring                                                               
1101:082303:0557 VoiceMail4     E IMAPNewMessage::deliver: Could not open socket connection - error 55. Delivery to local smtpd failed                    
1101:082317:0076 VoiceMail3     W mbox_login: connect failed                                                                                              
1101:082402:0576 VoiceMail5     E IMAPNewMessage::deliver: Could not open socket connection - error 55. Delivery to local smtpd failed                    
1101:082421:0211 VoiceMail3     E IMAPNewMessage::deliver: Could not open socket connection - error 55. Delivery to local smtpd failed                    
1101:082426:0634 VoiceMail0     W mbox_login: connect failed                                                                                              
1101:082429:0923 VoiceMail1     W mbox_login: connect failed                                                                                              
1101:082514:0711 VoiceMail5     E IMAPNewMessage::deliver: Could not open socket connection - error 55. Delivery to local smtpd failed                    
1101:082537:0326 Call           W connect - Conference request but no conference address was reserved                                                     
1101:082550:0749 Call           W connect - Conference request but no conference address was reserved                                     

1101:124644:0538 Call           W connect - Conference request but no conference address was reserved                                                     
1101:124733:0307 VoiceMail3     W mbox_login: connect failed                                                                                              
1101:124747:0384 CallGroup      W Call Group Device :CFWDALL enabled on primary do not ring                                                               
1101:124802:0346 MediaServer    E Task_00 - State 02 - Vail Session 00                                                                                    
1101:124802:0346 IntVM6400      E Ready                                                                                                                   
1101:124802:0346 IntVM6401      E Ready                                                                                                                   
1101:124802:0346 IntVM6402      E Ready                                                                                                                   
1101:124802:0346 IntVM6403      E Ready                                                                                                                   

1101:124802:0365 VAILSess39     E Idle                                                                                                                    
1101:124802:0365 VAILSess40     E Idle                                                                                                                    
1101:124802:0365 VAILSess0      E

** START OF VAILSess0 LOG DUMP **                                                                                     
1101:124802:0365 VAILSess0      E 1288637168 Caller ID name Tonia Boyette, number 1003                                                                    
1101:124802:0365 VAILSess0      E 1288637168 EV SetCallerId in ST Transfer: NEXT ST Transfer                                                              
1101:124802:0365 VAILSess0      E 1288637171 EV CallReleased in ST Transfer: NEXT ST ReleaseCall                                                          
1101:124802:0365 VAILSess0      E 1288637171 Unreserved Media Handle 0                                                                                    
1101:124802:0365 VAILSess0      E 1288637171 EV ReleaseCall in ST ReleaseCall: NEXT ST Idle                                                               
1101:124802:0365 VAILSess0      E 1288637172 Device 20 Reserved Media Session with handle 0                                                               
1101:124802:0365 VAILSess0      E 1288637172 EV Reserve in ST Idle: NEXT ST Reserved                                                                      
1101:124802:0365 VAILSess0      E 1288637174 IncomingCall: Remote MAC Addr = 00:e0:bb:23:52:24                                                            
1101:124802:0365 VAILSess0      E 1288637174 App AA, Calling , Called , FwdReason 6                                                                       

[helper_1]

tExcTask   11013f4      26eac444   0 PEND        11352f2 26eac3b4   3006b     0
tLogTask   1081284      26ea9b40   0 PEND        11352f2 26ea9aac  3d0001     0
tShell     111243c      2691b818   1 READY       1093bc8 2691b4ec  1c0001     0
LedDisplay 9ef610       26e66530   2 DELAY       108dd2e 26e664b0       0     5
tTelnetd   10adc14      26921848   2 PEND+S      108bddc 26921774       0     0
tWdbTask   10b6a80      2691c9d0   3 PEND        108bddc 2691c684       0     0
Disk 0     9ec360       26918f74   3 PEND        108bddc 26918eec       0     0
itimerd    956570       26378f0c   3 PEND        108bddc 26378e80       0     0
t1         956570       26376d50   3 PEND        108bddc 26376cd4       0     0
cleanupd   956570       26374bbc   3 PEND        108bddc 26374b4c       0     0
kmemd      956570       26372a28   3 PEND        108bddc 26372990       0     0
fsflushd   956570       26370894   3 PEND        108bddc 26370810  3d0002     0
pageoutd   956570       2636e700   3 PEND        108bddc 2636e684  3d0002     0
epid       956570       25edc4e0   3 PEND        108bddc 25edc374  3d0002     0
ElvisCodecTd4fbc4       236b66dc  38 PEND        108bddc 236b6680       0     0
nbxElvis   d4fa18       25c5fb14  39 PEND        108bddc 25c5fab0       0     0
ElvisAudioRd53790       236b8a28  39 PEND        11352f2 236b8958       0     0
ElvisReceivd53950       236b7894  39 PEND        11352f2 236b77c8       0     0
nbxH3errorHd23238       25c5ab60  40 PEND        108bddc 25c5aac0       0     0
tAioIoTask11108370      26e90fd0  50 PEND        108bddc 26e90f70       0     0
tAioIoTask01108370      26e89e3c  50 PEND        108bddc 26e89ddc       0     0
tNetTask   10a6d24      269441a4  50 PEND        108bddc 2694414c  73000a     0
nbxH3AudioRd28afc       25c55bac  50 PEND        108bddc 25c55b54       0     0
nbxH3readerd28f4c       25c00be8  50 PEND        108bddc 25c00b90       0     0
tH3Input   e27db8       24999654  50 PEND        108bddc 24999610       0     0
tH3LoSnd   e25d28       249984c0  50 PEND+T      108bddc 24998474  3d0004     1
tH3UDPRx   e2b5c4       2499732c  50 PEND        108bddc 24997178       0     0
TransportRe62170c       2476df30  50 PEND        11352f2 2476dd8c       0     0
tAioWait   11088b0      26e98164  51 PEND        108bddc 26e97e60       0     0
tFtpdTask  10c398       2692008c  55 PEND        108bddc 2691ffb8       0     0
time       62170c       24763f5c  55 PEND+T      108bddc 24763edc  3d0004     6
nbxCallProc111d14       24788004  60 PEND        108bddc 24787dd8  3d0002     0
debugger   62170c       24772f08  60 SUSPEND     108d2ee 24772ec0       0     0
VoiceMailRe62170c       24768f58  75 PEND        108bddc 24768960       0     0
nbxCCMDBserd33e68       2499112c  99 PEND        11352f2 24991084       0     0
nbxDownloadd4b370       25c94d68 100 PEND        11352f2 25c94c84  3d0002     0
nbxCCMserved33818       24996104 100 PEND        108bddc 2499601c       0     0
nbxCCMVTLsed33818       2498e888 100 PEND        108bddc 2498e7a0  3d0001     0
nbxVAppIL  ede9d8       24983b58 100 PEND        108bddc 2498372c       0     0
nbxMediaT00e18e24       2497dcb0 100 PEND        11352f2 2497dbe0  3d0002     0
nbxMediaT01e18e24       249693a4 100 PEND        11352f2 249692d4  3d0002     0
nbxMediaT02e18e24       24954a98 100 PEND        11352f2 249549c8  3d0002     0
nbxMediaT03e18e24       2494018c 100 PEND        11352f2 249400bc  3d0002     0
nbxMediaT04e18e24       2492b880 100 PEND        11352f2 2492b7b0  3d0002     0
nbxMediaT05e18e24       24916f74 100 PEND        11352f2 24916ea4  3d0002     0
nbxMediaT06e18e24       24902668 100 PEND        11352f2 24902598  3d0002     0
nbxMediaT07e18e24       248edd5c 100 PEND        11352f2 248edc8c  3d0002     0
nbxMediaT08e18e24       248d9450 100 PEND        11352f2 248d9380  3d0002     0
nbxMediaT09e18e24       248c4b44 100 PEND        11352f2 248c4a74  3d0002     0
nbxMediaT10e18e24       248b0238 100 PEND        11352f2 248b0168  3d0002     0
nbxMediaT11e18e24       2489b92c 100 PEND        11352f2 2489b85c  3d0002     0
nbxMediaT12e18e24       24887020 100 PEND        11352f2 24886f50  3d0002     0
nbxMediaT13e18e24       24872714 100 PEND        11352f2 24872644  3d0002     0
nbxMediaT14e18e24       2485de08 100 PEND        11352f2 2485dd38       0     0
nbxMediaT15e18e24       248494fc 100 PEND        11352f2 2484942c       0     0
nbxMediaT16e18e24       24834bf0 100 PEND        11352f2 24834b20       0     0
nbxMediaT17e18e24       248202e4 100 PEND        11352f2 24820214       0     0
nbxMediaT18e18e24       2480b9d8 100 PEND        11352f2 2480b908       0     0
nbxMediaT19e18e24       247f70cc 100 PEND        11352f2 247f6ffc       0     0
nbxInetD   d6b5a8       230affc4 100 PEND        108bddc 230afbf4  2b0001     0
nbxVMAA0   d6d4a8       2310eb98 100 PEND        11352f2 230fad70      37     0
nbxVMAA1   d6d4a8       2309ae68 100 PEND        11352f2 23081988  3d0002     0
nbxVMAA2   d6d4a8       2306ae38 100 PEND        11352f2 23055178  3d0004     0
nbxVMAA3   d6d4a8       2303a958 100 PEND        11352f2 2302a578  3d0002     0
nbxVMAA4   d6d4a8       22ff9180 100 PEND        11352f2 22fdfcc8      37     0
nbxVMAA5   d6d4a8       22fcad98 100 PEND        11352f2 22fb34c8  3d0002     0
nbxVMAA6   d6d4a8       22f9a8b8 100 PEND        11352f2 22f87ac0  3d0002     0
nbxVMAA7   d6d4a8       22f6ad38 100 PEND        11352f2 22eafc28  3d0004     0
nbxVMAA8   d6d4a8       22e9acf8 100 PEND        11352f2 22e81818  3d0002     0
nbxVMAA9   d6d4a8       22e6ab20 100 PEND        11352f2 22e55e10      37     0

[helper_1]

nbxVMAA10  d6d4a8       22e25820 100 PEND        11352f2 22e2a588  3d0004     0
nbxVMAA11  d6d4a8       22e0a648 100 PEND        11352f2 22ddfb58  2b0001     0
nbxVMAA12  d6d4a8       22dcac30 100 PEND        11352f2 22db4160  2b0001     0
nbxVMAA13  d6d4a8       22d9ac18 100 PEND        11352f2 22d88758  3d0002     0
nbxVMAA14  d6d4a8       22d6a2b8 100 PEND        11352f2 22d3fab8       0     0
nbxVMAA15  d6d4a8       22d2ab90 100 PEND        11352f2 22d124b0       0     0
nbxVMAA16  d6d4a8       22cfa878 100 PEND        11352f2 22ce6aa8       0     0
nbxVMAA17  d6d4a8       22cb64b8 100 PEND        11352f2 22c9fa18       0     0
nbxVMAA18  d6d4a8       22c8aaf0 100 PEND        11352f2 22c70800       0     0
nbxVMAA19  d6d4a8       22c5aac0 100 PEND        11352f2 22c44df8       0     0
nbxBitSended6d4a8       22c2a5e0 100 PEND+T      11352f2 22c04918       2 25448
nbxNotMgr  a8a4d8       236bb96c 100 PEND        11352f2 236bb8d0       0     0
nbxYAVA    a7dab8       22bfabf0 100 PEND        11352f2 22bfaa44       0     0
OutDialT0  a8e15c       22bf63ac 100 PEND        11352f2 22bf6310       0     0
OutDialT1  a8e15c       22bdbb18 100 PEND        11352f2 22bdba7c       0     0
OutDialT2  a8e15c       22bbfab8 100 PEND        11352f2 22bbfa1c       0     0
OutDialT3  a8e15c       22bb38d4 100 PEND        11352f2 22bb3838       0     0
OutDialT4  a8e15c       22b976e0 100 PEND        11352f2 22b97644       0     0
OutDialT5  a8e15c       22b7b4ec 100 PEND        11352f2 22b7b450       0     0
OutDialT6  a8e15c       22b6f308 100 PEND        11352f2 22b6f26c       0     0
OutDialT7  a8e15c       22b53114 100 PEND        11352f2 22b53078       0     0
OutDialT8  a8e15c       22b36f20 100 PEND        11352f2 22b36e84       0     0
OutDialT9  a8e15c       22b2ad3c 100 PEND        11352f2 22b2aca0       0     0
OutDialT10 a8e15c       22b0eb48 100 PEND        11352f2 22b0eaac       0     0
OutDialT11 a8e15c       22af2954 100 PEND        11352f2 22af28b8       0     0
OutDialT12 a8e15c       22ae6770 100 PEND        11352f2 22ae66d4       0     0
OutDialT13 a8e15c       22aca57c 100 PEND        11352f2 22aca4e0       0     0
OutDialT14 a8e15c       22aae388 100 PEND        11352f2 22aae2ec       0     0
OutDialT15 a8e15c       22aa21a4 100 PEND        11352f2 22aa2108       0     0
OutDialT16 a8e15c       22a85fb0 100 PEND        11352f2 22a85f14       0     0
OutDialT17 a8e15c       22a69dbc 100 PEND        11352f2 22a69d20       0     0
OutDialT18 a8e15c       22a5dbd8 100 PEND        11352f2 22a5db3c       0     0
OutDialT19 a8e15c       22a419e4 100 PEND        11352f2 22a41948       0     0
INetDlvry0 a99eb0       22a257a8 100 PEND        11352f2 22a256f4       0     0
TCPServerT a9ee88       22a1b2ec 100 PEND        108bddc 22a1b0a4  2b0001     0
DiskLogger 9ef610       26916de0 128 PEND        108bddc 26916d6c       0     0
Performance9ef610       268ebbcc 128 PEND        108bddc 268ebb58       0     0
nbxLogger  e01a44       26826fc4 200 PEND+T      108bddc 26826a4c  3d0002 66875
nbxLogRemote068f4       2678ded0 200 PEND        11352f2 2678ddfc       0     0
rdcTask    e2f9b8       236154a8 200 PEND        11352f2 236153e8       0     0
nbxDiagCLISe2dc00       25bfb8a0 200 PEND        108bddc 25bfb798       0     0
tEmWeb     647110       247e3690 240 PEND+T      108bddc 247e3230       0   856
cpuUsageTas9ead54       26ffee48 255 PEND        108bddc 26ffedfc       0     0
nbxACDRTS  cb00e8       24988b30 255 PEND+T      108bddc 249883a4  3d0004   368
value = 0 = 0x0

[merlin215]

 there are a couple of normal tasks in suspension but nothing service affecting .  No issues there . what bothers me is the socket errors and unable to send smtpd  errors . I would open a case for this ASAP if you have a network trace and other info  .  A network trace will be needed as well as an nbxSaveCore . when you open the case , please mention this to the folks and tell them what you have collected .

Some other questions :

NBX Type ?

6.5.22P03

how many tels ?

how many outside lines and types ? isdn , analog , digital

Looks like a vmail port problem .

more Info definately will be necessary .

[bosoxfan]

I see IMAP errors. Check your IMAP users. Is it possible that someone misconfigured their PC with IMAP and is using the NBX as a mail server? Remember it is only the Incoming Mail server that has the NBX, Outgoing Mail server on the IMAP settings on the PC should be the regular mail server.

[helper_1]

V3000A

6_5_22_P03

57 - 3102A

2 - 23 channel PRI's

20 - voice mail ports

[helper_1]

According to the customer, no one is setup for IMAP.  They are running call assistant with the ACD group.  The call assistant is running on all members (15) of the ACD group.

[merlin215]

It seems to be trying to email out .  Thats the strange thing .

[bosoxfan]

hi. it is quite strange that you are getting IMAP errors but no user using IMAP.

You had stated that if a user logs in and puts in their password, then the system crashes.

Can you delete that user and phone and rediscover??? that would clear their mailbox if the mailbox is

corrupted. .. or is it more than one user?