IMC Data Base Change

[idareval]

I have a IMC currently installed and I need to change the current database to another that is installed on another server. how I can do this?

Thanks

[LindsayHill]

Are you looking to dump the DBs from one server, and restore them to another server?

I haven't tried it, but you could probably shut down iMC, migrate all the DBs, and set up all the users. Make sure you give the users the same password.

Then edit C:\Program Files\iMC\deploy\conf\dma.conf, and C:\Program Files\iMC\common\conf\server-addr.xml

In those files, change all the references from your old server IP/hostname to the new name/IP.

Then restart IMC.

If you are creating the new users in the new DB, and you don't know the passwords used for each of the user accounts associated with each DB, use C:\Program Files\iMC\deploy\pwdmgr.bat - it will tell you what the passwords are.

I don't know for sure if this will work, but it would be straightforward enough to test. Since you will still have your existing DB in place, you can easily roll back.

[BlakkheimGW]

Hey there,

Sorry to resurrect this post, but I have exactly the same question. I'm trying to move the database to another server. Everything is working fine in the IMC interface : I've retrieved my devices, performance indexes and alarms etc... BUT in the "Deployment Monitoring Agent", "Environment" tab, I have an error message regarding the database. It says "Incorrect database password". When I click the "Change password" button, the "Database server address" field is greyed out and it displays the old server address.. I've captured network traffic and IMC really tries to contact the old server.

And if I try to deploy a component via the "Deploy" tab, it fails with a database error because it also tries to contact the old server.

I've tried to grep any remaining occurence of the old address in the whole IMC directory, without luck. Any idea ?

Lindsay, in your previous posts, you are mentionning the dma.conf file but I can't find any occurence of the database address in that file..

Thanks !

[LindsayHill]

What do you have in deploy\conf\dbconfig.xml ?

[BlakkheimGW]

Hello Lindsay,

My dbconfig.xml looks like this :

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<dbconfigs>
  <item compId="iMC-PLAT" dbAddress="NEWIP" dbName="config_db" dbType="MySQL" dbUserName="imc_config" forceDropWhenRemove="false"/>
<item compId="iMC-NME-FAULT" dbAddress="NEWIP" dbName="monitor_db" dbType="MySQL" dbUserName="imc_monitor" forceDropWhenRemove="false"/>
<item compId="iMC-ICC" dbAddress="NEWIP" dbName="icc_db" dbType="MySQL" dbUserName="imc_icc" forceDropWhenRemove="false"/>
<item compId="iMC-REPORT" dbAddress="NEWIP}|{NEWIP" dbName="report_db}|{reportplat_db" dbType="MySQL" dbUserName="report}|{reportplat" forceDropWhenRemove="false"/>
<item compId="iMC-NME-PERF" dbAddress="NEWIP" dbName="perf_db" dbType="MySQL" dbUserName="imc_perf" forceDropWhenRemove="false"/>
<item compId="iMC-ACLM" dbAddress="NEWIP" dbName="aclm_db" dbType="MySQL" dbUserName="imc_aclm" forceDropWhenRemove="false"/>
<item compId="iMC-NETASSET" dbAddress="NEWIP" dbName="invent_db" dbType="MySQL" dbUserName="imc_inventory" forceDropWhenRemove="false"/>
<item compId="iMC-SYSLOG" dbAddress="NEWIP" dbName="syslog_db" dbType="MySQL" dbUserName="imc_syslog" forceDropWhenRemove="false"/>
<item compId="iMC-VLAN" dbAddress="NEWIP" dbName="vlanm_db" dbType="MySQL" dbUserName="imc_vlan" forceDropWhenRemove="false"/>
<item compId="iMC-VNM" dbAddress="NEWIP" dbName="vnm_db" dbType="MySQL" dbUserName="imc_vnm" forceDropWhenRemove="false"/>
</dbconfigs>

All occurences I've found in every file of the IMC directory have been replaced..

[LindsayHill]

Hmm, that file looks correct. I'll have a look around, see if I can figure out which other files you might need to check.

[kortemi]

Hi,

I'm struggling with this same problem:

Had to move the databases to another SQL Server, to a named instance (in the old server it was on the default instance).

I've edited all of the IMC config files I can think of (list below) with the new SQL server details. All other IMC processes start OK, except jserver, which just says "Failed to connect to database".

I am stumped, anyone for help?

Here's the list of files that I've edited:

FOLDER:  ..\IMC\common\conf:
server-addr.xml

FOLDER: ..\IMC\deploy\conf
component-env.xml
dbconfig..xml

FOLDER: ..\IMC\dbman\etc
dbman.conf

FOLDER: ..\IMC\client\web\apps\imc\reports
iMCCRconfig.xml

FOLDER: ..\IMC\client\web\apps\imc\reportsv2
dataSource.xml

FOLDER: ..\IMC\client\web\apps\rptviewer\WEB-INF\repository\data
<serverip>.xml
127.0.0.1.xml

Since the databases now exist in a named instance, I've entered the server addresses in the following format. Is this correct, or am I doing something wrong here? (example from server-addr.xml

<db-config address="<dbserverIP>\IMC" dbname="aclm_db" password="<pwhash>" type="SQLServer" username="imc_aclm"/>

In component-env.xml I've configured the following:

<component id="iMC-PLAT">
    <var name="DATABASE_PORT" value="1433"/>
    <var name="DATABASE_ADDRESS" value="1<dbserverIP>"/>
    <var name="DATABASE_TYPE" value="SQLServer"/>
    <var name="DATABASE_NAME" value="config_db"/>
    <var name="DATABASE_PASSWORD" value="<pwhash>"/>
    <var name="DEPLOY_IP" value="127.0.0.1"/>
    <var name="JDBC_INSTANCE_NAME" value=""/>
    <var name="DATABASE_INSTANCE" value="IMC"/>
    <var name="DATABASE_USER_NAME" value="imc_config"/>
    <var name="INSTANCE_NAME" value=""/>
    <var name="VERSION" value="7.2-E0403"/>
    <var name="INNER_VERSION" value="V700R001B06D008"/>
  </component>

Have I correctly understood the entries DATABASE_ADDRESS and DATABASE_INSTANCE?

In dbconfig.xml I've the following for each table:
<item compId="iMC-ACLM" dbAddress="<sbserverIP\IMC" dbName="aclm_db" dbType="SQLServer" dbUserName="imc_aclm" forceDropWhenRemove="false"/>

And in dbman.xonf I've got the following:

# DBMAN Configurations

Language = en
# Common Configurations
EnableDbman = 1
BackHoseIp =
PrimaryHost = 1
MaxLogSize = 10485760
FileReserveTime = 7
CommandPort = 2810
SessionTimeout = 300
FaultIp = 127.0.0.1
IfTransToHost  = 0

# Primary Common Configurations
BackupAtOnce = 0
BackupTime = 4
BackupTimeMinute = 0
AfterFTPSave = 2
# iMC server Configurations
ServerCount = 1

iMCIP1 = 127.0.0.1
DBaseIP1 = <dbserverIP>
DBType1 = 1
DBInstance1 = IMC
PrimaryDbSaUserName1 = sa
PrimaryDbSaPassword1 = <pwhash>
PrimaryDbPort1 = 1433
DBCount1 = 10

# Primary 1 Configurations
BackupPath1_127.0.0.1 = D:\IMCBackup
BackupPath1_10.158.90.138 = F:\SQL_BackUp\IMC

PrimaryDbInst1_1 = <dbserverIP>\IMC@aclm_db
PrimaryDbUser1_1 = imc_aclm
PrimaryDbPassword1_1 = <pwhash>
PrimaryDbInst1_1_iMC-ACLM = 7.2.E0403
ifbackup1_1 = 1
iftrans1_1 = 0

Is there something I've got wrong in the above examples?

Please help, this is driving me round the bend...

[LindsayHill]

I suspect that you need to change "INSTANCE_NAME", not "DATABASE_INSTANCE".

I also don't think that you can specify the address in that format.

What you could do as an experiment is to set up a new named instance, and a new IMC install on a different server (just install with the trial license). Then you can see what format IMC uses in those files, and change your production config as needed.

[LindsayHill]

Oh and the other thing you could look at is doing something like doing a backup & restore, but restoring to the new DB location.

[JasonL1]

I have done this. It is, from an ultra high level, copy, paste, redirect, done. I copied the databases and moved them to a fresh SQL VM, installed IMC (the same version that you are using), copied the databases over as a backup then, using IMC, restored the databases. Once the database move was done, I redirected the main IMC server to the new DB server within its config file.

I have attached the document that was sent to me by the HP Engineer, I followed most of the instructions but, I skipped making two new servers and edited the file it has you copy. That was done simply to save time and resources.

As always, make sure you have a recent backup for your server before you make these changes. You could easily make your day worse.

[JasonL1]

---

@LindsayHill wrote:

Are you looking to dump the DBs from one server, and restore them to another server?

I haven't tried it, but you could probably shut down iMC, migrate all the DBs, and set up all the users. Make sure you give the users the same password.

Then edit C:\Program Files\iMC\deploy\conf\dma.conf, and C:\Program Files\iMC\common\conf\server-addr.xml

In those files, change all the references from your old server IP/hostname to the new name/IP.

Then restart IMC.

If you are creating the new users in the new DB, and you don't know the passwords used for each of the user accounts associated with each DB, use C:\Program Files\iMC\deploy\pwdmgr.bat - it will tell you what the passwords are.

I don't know for sure if this will work, but it would be straightforward enough to test. Since you will still have your existing DB in place, you can easily roll back.

---

This is pretty much dead on, how the process works.

[kortemi]

Thanks, Lindsay

Good suggestions, but alas, they did not solve the problem.

First I tried the change the name of the instance to the tag "JDBC_INSTANCE_NAME". That did not do it.

Then I tried your ("very good, why did I not think of this myself") suggestion about doing a trial install. Well, that took me all day (first, install yet another SQL instance, then install the SQL Management tools, then try the IMC installation). But, despite several (a lot) tries, I was unable to get past the first screen of the IMC Custom install - database connectivity. Selecting SQL Server, the Instance name, the sa account and pw, server IP, and port 1433 (and configuring the SQL server instance according to the HPE Intelligent Management Center v7.2 SQL Server 2012 Installation and Configuration Guide), the IMC installation would give me the error "Unable to connect to the database".

I am beginning to suspect that there is a serious flaw in IMC code, and that it's just simply unable to connect to a named instance....

(and yes, I had actually done the backup and restore of the databases, setting up the IMC db user accounts and their pw's to what they were, etc).

So, now I thought to try one more thing, setting up an SQL 2014 Express on the actual IMC server itself, maybe that would do it - at least then I'll be able to set it up on the default instance...

[JasonL1]

Kortemi, 

Are you using the same flavour of SQL and IMC on both sides?

[kortemi]

Yep, all are the same. In fact, the new test SQL instance was on the same server, just an additional instance.

[LindsayHill]

---

@kortemi wrote:

First I tried the change the name of the instance to the tag "JDBC_INSTANCE_NAME". That did not do it.

I had been thinking of this line:

    <var name="INSTANCE_NAME" value=""/>

Not this one:

    <var name="JDBC_INSTANCE_NAME" value=""/>

But it was just speculation, I don't know for sure.

I am beginning to suspect that there is a serious flaw in IMC code, and that it's just simply unable to connect to a named instance.....

---

According to the 7.2 manuals, IMC should be able to use a named instance, but I can't confirm it as I haven't tested it myself. These days I mainly work with iMC on Linux with MySQL, as it's easier for me to automate.

[kortemi]

Ok, thanks, Lindsay. I'll try that one then (missed it somehow, must be tired or something)

[Pack3tL0ss]

There is currently no supported 'easy' way to migrate IMC to a new DB, DB port, instance etc.

*The below is from memory, is certainly not in exact order... don't use it as a cook-book*

There is a process to do so.  It involves configuring a Temporary IMC server (doesn't need licensing, it'll run in eval mode).  The Temp IMC should be installed to match the Real IMC (same version, same modules, same patch levels), but it is installed against the new DB.  Database is backed up from Real IMC and restored on Temp IMC.  Some key files are coppied from Temp IMC onto Real IMC.  Temp can be de-commisioned and Real IMC should be pointing to new DB.

The reason for this is that IMC creates a file during the install: c:\windows\imc-reserved (or something very close to that - from memory).  That file includes a bunch of information relating to the database (IP, instance, port, credentials, etc).  The file is hashed so you can't manually edit the file.  It is used during upgrades or the installation of a new module.  If you've moved the database it's possible you have an IMC system that seems to function correctly, but when you go to upgrade it will fail as that process references that hashed file.  I believe it's also referenced in other places, the environments tab in the DMA is likely one of them.

If you've migrated and find yourself in a pickle, you can either role everything back to the backup you've hopefully taken so it's pointing to the original database.  Then follow the process to create the temp IMC etc.  OR you can contact support, they have a tool that allows them to de-crypt, edit, then re-hash that file which hopefully would rectify the issue.

I'm hopeful that we will have a migration utility that scripts this and removes the need for a temporary server, but no commitment yet.

Hope that helps,
PL

[kortemi]

Thanks, PL.

This is getting interesting. I thought I'd give Jason's document a try (it's almost the same as another document I've seen, but with some additional info), and so I tried to installa a TEMP IMC installation on the SQL Server where I want to host the databases.

In this document the description reads:

How to change iMC local database to remote database server：

Scene: A customer install iMC and database on server A （IP：192.168.1.114）, now he want to only move database to server B（IP：192.168.1.133）.

Version: IMC 7.0

Restriction: Install IMC platform and all modules in same server

And the process is to

1. stop IMC on Server A
2. install SQL on Server B
3. install IMC on server C (a new TEMP server), pointing to the SQL on server B
4. After IMC installation, copy 2 files from Server C to Server A (server-addr.xml and instinfo.txt)
5. Start IMC on server A
6. Shut down Server C and remove SQL on Server A

BUT, when I tried to install IMC on our SQL Server, the installation (selected Custom) fails on the Checking Database Connectivity - even on Localhost.  AND IT FAILS on both named instance and default instance with the message "Connecting to the database failed. Make sure that the database is running normally and you have input correct login information".

This is getting curious. I'm trying to install ICM v7.2 E0403, so I'm starting to suspect that this version of IMC setup is broken.

So, now I'm considering maybe dropping back to v7.1  for the TEMP, just to see if that would solve the problem - and enable me to follow the process in Jason's document.

[kortemi]

Right, there goes that theory....

Tried with 7.1 E0303, and exactly the same result "Connecting to the database failed".... AARGH!

And this is on the SQL Server itself....

I've edited the SQL TCP/IP and Named Pipes settings, firewall is off (all), I even created a new 'sysadmin' user.... No luck.

And of course the setup does not log anything in the Windows Logs, and I cannot find any log files that the setup would be logging to...

[JasonL1]

I recieved the same error a few minutes ago when trying to install a few new modules after I upgraded to 7.2 here at work. I fought with it then remembered, I turned SQL services off on my main IMC server which, is where IMC was trying to create the new databases.

Now, I need to figure out how to make the installer allow me to change the database location from the IMC server to my database server.

[kortemi]

Thanks,  Pack3tL0ss.

Based on your post, I looked into the C:\Windows\iMC-reserved folder.

In there I found two files, instinfo.txt and a curious file called install.lock.
The install.lock file is 0 KB in size, and contains absolutely nothing.

The instinfo.txt file does state at the beginning "#Do NOT modify anything in this file or delete this file manually!", then an instdate entry, and a instdir entry, and a datadir entry, before a hashed instinfo entry.

I have, just in case this is the source of my problems, a support case with HPE, but I am sitll wondering about the install.lock file. Can't seem to finid any information on this file ahywhere (or have not had any luck so far anyway).

[Pack3tL0ss]

Installlock appears to be a kind of pid file.  If you start the installation process to add or upgrade a module it seems to change an attribute for that file (if you right click and go to properties --> details).  It has an attribute "N" when nothing is going on, and "A" when you've clicked the install button and have the installation process running.  So it's just used to tell the installer if the process is already running or not so it doesn't spawn multiple installation processes, which could likely bork things.

it's the installinfo.txt file that has the hash for the originally specified database location/instance/credentials/etc. 

For the database connectivity issues on the temp server.  IMC needs the SQL client components from the MS SQL install media installed on the IMC server, without it it won't connect to an external database.  I thought that when it couldn't find the client it automatically assumed you wanted the embeded database (MS SQL Express) installed co-resident with IMC and would start installing it, but they may have changed that.  At anyrate I'd verify the client components are installed, the required pieces are documented in the IMC SQL Server XXXX installation guide (One of the docs in the HPE IMC documentation)... 

Again for anyone else with issues if you've moved the IMC database at anytime after the original IMC installation, (and didn't go through the process of setting up a temporary IMC server), then the installinfo file is the source of the issue.  At the time of the original install (which would be the original database) that file is created and crucial information is placed there, but it's hashed so it can't be modified.  Support has a tool to update the file and hopefully rectify the situation.

[kortemi]

Thanks, Pack3tL0ss

The SQL Server management tools are installed, and I can connect to the restored (with SQL itself) databases so that's not an issue.

But, I'm beginniing to suspect that the root of the problem, as you state, lies indeed in the installinfo.txt file. I have a ticket open with HPE support, and their first suggestion was to install a temp IMC installation, pointing to the new DB. Unfortunately I still have the problem that during installation IMC says that it is unable to connect to the database (even when I install IMC on the db server)..

[kortemi]

My issues have been raised at HPE support to the highest level (levels 1 and 2 were unable to assist), but since I had had very little feedback from HPE, and feeling frustrated with the situation, I thought to try a yet another approach.

1) I removed the SQL Server Instance on the separate SQL Server.

2) In installed Microsoft SQL Server 2014 SP1 on the same server where IMC was installed, only default instance (no instances this time)

3) I uninstalled IMC (but left the data behind, when asked by the uninstall)

4) I tried to install IMC (7.2 E0403) again by running the install.bat from the folder windows\install

5) I then left Country/Region and Language as suggested by the installation program, and selected Custom as the Installation Type

6) After clicking on OK, I get presented with the Checking Database Connectivity dialog box. In it, I select the following:

 a. Database Type: Microsoft SQL Server

 b. Instance Name: Default Instance

 c. Superuser: sa

 d. Password: < pw>

 e. Database location: local host

 f. Database server address: <greyed out> 127.0.0.1

 g. Listening port: 1433

7) When I press OK, I get the same error message:  Error: Connecting to the database failed. Make sure that the database is running normally and you have input correct login information

 a)  This turned out to be a disabled sa account (I had chanded the pw, but that did not enable the account). Once I enabled the account, I was able to move forward with IMC installation.

8) However, IMC is now at a fresh install state.

 a) I then tried to do a restore of all database files from the IMC Deployment Monitoring Agent, and I got an error.

The dbman log files give some info on this:

At the end of the dbman_debug.log file there are the following lines:

2016-02-11 13:27:53 [ERROR] [checkDBFiles] Restore all database failed: Components is not match

2016-02-11 13:27:53 [ERROR] [checkDBFiles] Database: 127.0.0.1@config_db. Components: iMC-PLAT=7.2.E0403 iMC-GAM=7.2.E0403 iMC-EUPLAT=7.2.E0403 

2016-02-11 13:27:53 [ERROR] [checkDBFiles] DbFileConf: <oldIP>0@config_db_imc_config_db_20160108_122120_full.db. Components: iMC-PLAT=7.2.E0403 

2016-02-11 13:27:53 [INFO] [sendTrapAlert] Send trap success

2016-02-11 13:27:53 [ERROR] [ManualRestoreBase] Fail to check DB file.

2016-02-11 13:27:53 [ERROR] [response_err_code] errCode = -1

The logn states “Components is not match”, and this would indicate that the restore components are not the same as the previous install.

9) Then I thought to try to restore the databases one at a time, and now I managed to manually restore all databases, except for the config_db. This is the one that gives me the error I referred to.

So, it seems that the reinstall probably does not match (component-wise) the previous installation.  BUT, since it's nearly two months since my problems started, does anyone know, if there are any tools that I could use in order to figure out which components I am missing?

[kortemi]

Just to wrap this thing up (and maybe help whoever's looking for help in the future):

I managed, after all, to sort the problem out.

I noticed that there were two services that were by default undeployed (User Serservice Management and Guest Access Management). I deployed these, and then tried to restore the config_db, and this time it worked.

So, now our IMC is working on a new database.