IMC 7.2 to 7.3 upgrade

[pattap]

Hi All

Upgrading from iMC PLAT 7.2 (E0403) to 7.3 (E0506) and getting the below, anyone has come across this error? Worth mentioning that the file mentioned in the log doesn't exist (C:\Program Files\iMC\client\web\apps\rpt\WEB-INF\lib\imcweb_plat.jar)

log is the part of dmslog.txt file 

java.io.FileNotFoundException: C:\Program Files\iMC\client\web\apps\rpt\WEB-INF\lib\imcweb_plat.jar (Access is denied)
	at java.io.FileOutputStream.open(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:145)
	at com.h3c.imc.deploy.FileUtil.copy(FileUtil.java:111)
	at com.h3c.imc.deploy.tasks.FileCopyTask.executeTask(FileCopyTask.java:102)
	at com.h3c.imc.deploy.tasks.SimpleTask$a.a(SimpleTask.java:66)
	at com.h3c.imc.deploy.tasks.SimpleTask$a.doInBackground(SimpleTask.java:56)
	at com.h3c.imc.deploy.tasks.f.call(SwingWorker.java:294)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at com.h3c.imc.deploy.tasks.SwingWorker.run(SwingWorker.java:333)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:662)

in deployment agent getting the below 

 

[parnassus]

It looks like someone have been already hitted by a very similar issue and solved, see here.

[pattap]

Also in deploylog file just found this :

Timestamp: 2019-05-20 16:26:28
Component ID: iMC-REPORT
Name: iMC Platform - Report Management
Version: iMC PLAT 7.3 (E0506)
InnerVersion: V700R001B06D018
Address: 127.0.0.1
Operation: Update completed
Result: failed
Fail reason: Performing deployment or upgrade on the master server failed. For details, see the log on the master server.

[pattap]

Anyone? 

[NeilR]

I did not have this problem, so can't tell you much other than I upgraded incrementally, as they were released.

So I went from 7. 2 E0403P10 to 7.3 E504 and then on from there as patches were released.

Sometimes there are changes to the DB schema that are assumed to be in place from prior updates in order for the current one to succeed. 

The release notes also have a section on which predecessors will work for the current release.

So don't know if you tried going to the immediate next major release E504. and then forward from there.

[pattap]

I've tried from 7.2(403) to 7.2(403P04) getting the same error, the only difference is that I am able to start IMC but half of components are not deployed

[NeilR]

The file imcweb_plat.jar mentioned in the log does exist in my current configuration. Looks like from the post Parnassus referenced putting it back might work,

Might be able to find it in the  install zip files? Or worst case install a fresh copy and find the file and copy it.

[pattap]

I've tried that, I copied one from previous version - desperate for solutions.

I also tried looking for in in the zip file but no luck

[jguse]

Hello,

A few questions:

- Did you check for the SSL Cipher issue mentioned in the initial reply? IMC 7.3 E0605 onwards includes a new MSKBsC.exe that can automatically remove the affected ciphers. Just run it on the IMC and DB server on CMD and follow the output.

- Do you use a local or a remote DB?

- Have you restored the DB using any other tools than IMC's dbman (Deployment Monitoring Agent 'Backup' and 'Restore' buttons)?

There were some issues in the past where the Report Management failed because of non-dbman restores of the DB (eg. via SSMS), causing the 'report' user's mappings/schemas to be lost or incorrect.

I have an embedded DB lab system that was upgraded successfully over the years from 7.1 E0302 -> 7.2 E0403 -> 7.3 E0506 and onwards without any such issues, so I'd suspect that it is a DB- or deployment-related issue.

[pattap]

Hi Justin 

Just an update, not a good one. I'm not able to update to any patch within 7.2 from 7.2 either, same error

1. yes I have ran MSKBsC.exe, can I based on the below assume there's no issues detected?

2. Local data base - IMC and DB on the same server 

3. No, DB has been restored using IMC deployment tool 

Is there a place where more specific logs are placed?  I'm really stuck with this 

Thanks 

[jguse]

Hello,

When an upgrade fails, the relevant fatal/error log entry is normally found in iMC\deploy\dmalog.txt. I see you have already posted an entry from dmslog, but please check the dmalog as well. If there was a DB issue, the error there will usually include a reference to a log such as Windows\iMC-Reserved\install_x.txt or dbresult_x.txt for more details. You can then check that iMC-Reserved\install_ and dbresult_ for exact details on the failure.

[pattap]

Hi  Justin

 C:\Windows\iMC-Reserved\log\install_20190611131022.txt

contains 'SQLServer 2000 is not Installed' as below. Not sure if I need to worry about it. It's a bit strange as db on our old IMC was built on SQL2012 so is the new one

2019-06-11 13:10:35 [INFO ] [SwingWorker-pool-1-thread-1] [com.h3c.imc.deploy.install.InstallationComponentInfo::b(596)] Can not find java-monitor configurations in deploy configs, but found in update configs
2019-06-11 13:10:36 [INFO ] [SwingWorker-pool-1-thread-1] [com.h3c.imc.deploy.install.InstallationComponentInfo::b(596)] Can not find java-monitor configurations in deploy configs, but found in update configs
2019-06-11 13:10:39 [INFO ] [SwingWorker-pool-1-thread-1] [com.h3c.imc.deploy.dma.DeployManager::a(791)] Component deploy state changed
2019-06-11 13:10:39 [INFO ] [SwingWorker-pool-1-thread-1] [com.h3c.imc.deploy.install.wizard.InstallFolderChoosePanelDescriptor::a(447)] will update to big version : 7.2
2019-06-11 13:10:39 [INFO ] [AWT-EventQueue-0] [com.h3c.imc.deploy.dma.wizard.DatabaseSelectPanelDescriptor::isSQLServer2000Installed(375)] SQLServer 2000 is not Installed

There was no dbresult_xx file dated as the file above but I have looked at some other files and found this

........omitted.............
(1 row affected)
Msg 547, Level 16, State 1, Server IMC-02, Line 565
The INSERT statement conflicted with the FOREIGN KEY constraint
"fk_tbl_enterprise_refrence". The conflict occurred in database "monitor_db",
table "imc_monitor.tbl_trap_enterprise_id", column 'enterprise_id'.
The statement has been terminated.
Msg 547, Level 16, State 1, Server IMC-02, Line 566
The INSERT statement conflicted with the FOREIGN KEY constraint
"fk_tbl_enterprise_refrence". The conflict occurred in database "monitor_db",
table "imc_monitor.tbl_trap_enterprise_id", column 'enterprise_id'.
The statement has been terminated.
(1 row affected)
Msg 547, Level 16, State 1, Server IMC-02, Line 568
The INSERT statement conflicted with the FOREIGN KEY constraint
"fk_tbl_enterprise_refrence". The conflict occurred in database "monitor_db",
table "imc_monitor.tbl_trap_enterprise_id", column 'enterprise_id'.
The statement has been terminated.
Msg 547, Level 16, State 1, Server IMC-02, Line 569
The INSERT statement conflicted with the FOREIGN KEY constraint
"fk_tbl_enterprise_refrence". The conflict occurred in database "monitor_db",
table "imc_monitor.tbl_trap_enterprise_id", column 'enterprise_id'.
The statement has been terminated.
(1 row affected)
........omitted.............

[jguse]

Hello,

The log will show "xxx is not Installed' for every database version that you do not have installed. It's just checking through possible versions until it finds the correct one, and reports something like "SQL Server 2012 is installed."

Do you only have that one install_ log and no dbresult_ logs from that date when you upgraded? If a few foreign key constraints are the only issue, that could be fixable with sqlcmd or Management Studio scripts to remove affected constraints, but I can't recommend that without direct assistance from support.

Did you run the MSKBsC.exe before or after your upgrade attempt failed?

It needs to be run before performing any upgrade,  even before restoring the old database, as the SSL Cipher mismatch issue causes DB connectivity to spontaneously interrupt. If you tried to start it again after the upgrade was interrupted the first time, it will fail the second time onward with errors like 'Foreign key constraint..." because the IMC install/upgrade scripts will try to re-insert values in the database that are already restricted from changes by foreign key constraints.

If that's the case, you would need to restart the upgrade process from the original version again, making sure the ciphers were removed beforehand. I'd recommend running MSKBsC.exe that is included in the E0605 version's tools folder, as that is the new version that automatically removes them for you. It needs to be run once on the remote DB as well.

Hope that helps...

[pattap]

Yes install_ log was the only file from the time of upgrade.

MSKBsC.exe was ran before and after the upgrade process, I've ran environment check as well, which for some reason stopped working now.

I have started from the scratch, fresh mySql and IMC. Very similar, I get ok to 7.2 but any patches from there fail.

This time also seeing this in C:\Windows\iMC-Reserved\log\installxxxxx,

2019-06-16 18:28:07 [WARN ] [Thread-4] [com.h3c.imc.deploy.DMALauncher::tryToSendBatchOperCommand(382)] Send "BATCHOPER" command error
java.net.ConnectException: Connection refused: connect
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
	at java.net.Socket.connect(Socket.java:529)
	at java.net.Socket.connect(Socket.java:478)
	at java.net.Socket.<init>(Socket.java:375)
	at java.net.Socket.<init>(Socket.java:189)
	at com.h3c.imc.deploy.DMALauncher.tryToSendBatchOperCommand(DMALauncher.java:371)
	at com.h3c.imc.deploy.install.wizard.FinishPanelDescriptor.aboutToHidePanel(FinishPanelDescriptor.java:111)
	at com.h3c.imc.deploy.wizard.Wizard.close(Wizard.java:647)
	at com.h3c.imc.deploy.install.wizard.FinishPanelDescriptor.displayingPanel(FinishPanelDescriptor.java:87)
	at com.h3c.imc.deploy.wizard.Wizard.setCurrentPanel(Wizard.java:546)
	at com.h3c.imc.deploy.install.wizard.InstallingPanelDescriptor$c.run(InstallingPanelDescriptor.java:302)

in addition same in dmslog contain the same error as previously

2019-06-16 19:24:34 [ERROR] [SwingWorker-pool-6-thread-8] [com.h3c.imc.deploy.FileUtil::copy(125)] 
java.io.FileNotFoundException: C:\Program Files\iMC\client\web\apps\rpt\WEB-INF\lib\imcweb_plat.jar (Access is denied)

Regarding remote DB, I'm having IMC and DB on the same server

I can start and connect to IMC but of course reporting and asset management are not working- quick update asset management seem to be working

[jguse]

Hello,

That's odd, you have completely re-installed again and the issue re-appears?

Just a thought since you mention "I've tried that, I copied one from previous version - desperate for solutions." about the .jar file... can you please check your directory:

C:\Program Files\iMC\client\web\apps\rpt\WEB-INF\lib\

There should only be one single imcweb_plat.jar file in here. Make sure this directory does NOT contain any backup copies of this file, even if it was renamed. For example if you had replaced it during troubleshooting. If you make a backup of imcweb_plat.jar before replacing it, it must be moved somewhere out of this directory, or IMC will load it anyway and that will cause issues if there are two separate copies trying to load.

I found one old issue around IMC 7.2 times with the imcweb_plat.jar error, and it turned out they had replaced imcweb_plat.jar during troubleshooting with an incorrect one from older IMC version, where that also triggered the error.

Remember to restart IMC after you make any changes to this directory.

If you still see that error, can you upload the imcweb_plat.jar that your current installation is using to an FTP and PM me with the link, along with the exact IMC edition and version (eg. 7.2 E0403 Standard) and DB (you mention MySQL and MSSQL) you are running now? I want to compare the file to a matching installation in the lab.

[pattap]

Yes fresh install of IMC and SQL

Interesting is the fact that imcweb_plat.jar does not exist in 

C:\Program Files\iMC\client\web\apps\rpt\WEB-INF\lib\

I have copied it over from old version and tried  upgrading, once it failed the copied version wasn't there anymore

I've ran search in installation directory of IMC and the only locations containing the file are:

C:\Program Files\iMC\client\web\apps\gam\WEB-INF\lib
C:\Program Files\iMC\client\web\apps\rptviewer\WEB-INF\lib
C:\Program Files\iMC\client\web\apps\imc\WEB-INF\lib

I have uploaded them, link on its way to your inbox, they are E0403P10 though, if needed I'm happy to bring it down to 7.2E0403

ok turns out I'm not able to send messages anymore - limit reached, sent two messages since restringing here?! 

Anyway, files uploaded to https://gofile.io/?c=r8YuRK read notes to figure which is from which location 

[jguse]

Hello,

Thanks for sharing the imcweb_plat.jar files. It took me a while to find and restore an older 7.2 E0403P10 system, but I eventually managed. Comparing the .jar files, they are all exactly the same files as in my installation.

Do you have other .jar files (there are more than a dozen) in the web\apps\rpt\WEB-INF\lib directory? Only imcweb_plat.jar is missing, right?

Either way I've shared the imcweb_plat.jar for rpt on FTP, and also uploaded an archived copy of the entire "rpt\WEB-INF" directory to this FTP: ftp://esfrm321:q*ZY6mw.@ftp.ext.hpe.com

Please try placing just the imcweb_plat.jar first and see if that works. If you end up missing the other .jar files or other files based on the WEB-INF archive I sent, feel free to put them in place as well, but remember to take a backup of the whole directory first (placed somewhere outside IMC directories).

[pattap]

Correct only Only imcweb_plat.jar is missing

your ftp is asking for user/pass

[jguse]

Sorry I didn't mention them separately. The username & password were already predefined in the link: esfrm321 and q*ZY6mw.

EDIT: Fixed the password.

[pattap]

My bad | haven't spotted it.

no luck with new files, however I've managed to upgrade to 7.3(605)... but I'm still not able to deploy imc platform - network asset management and report management components. The below is what I'm seeing in dmslog

[ERROR] [pool-4-thread-1] [com.h3c.imc.deploy.Util::g(2962)] java.io.FileNotFoundException: components\common\deploy\custom\custom.properties (The system cannot find the path specified)

This exactly error would show over 15 times  

[jguse]

Hello again,

I don't think that would be the relevant error here, as the lack of "custom.properties" generates an error regularly that just means you are not using this file. It is only used if you are defining some custom Batch Operations in IMC (see also the Customizing IMC chapter in the Admin Guide).

Since you showed that from dmslog, can you check the dmalog for errors as well when you try to deploy these components? Remember to also check the Windows\iMC-Reserved\ folder for any new install or dbresult logs that could indicate what is going wrong.

If you are still stuck with this, I'd suggest gathering a Data Collection (System Configuration > Data Collection) and opening a support case so we can have a proper look at the system.