IMC Data Base Change

[kortemi]

Thanks,  Pack3tL0ss.

Based on your post, I looked into the C:\Windows\iMC-reserved folder.

In there I found two files, instinfo.txt and a curious file called install.lock.
The install.lock file is 0 KB in size, and contains absolutely nothing.

The instinfo.txt file does state at the beginning "#Do NOT modify anything in this file or delete this file manually!", then an instdate entry, and a instdir entry, and a datadir entry, before a hashed instinfo entry.

I have, just in case this is the source of my problems, a support case with HPE, but I am sitll wondering about the install.lock file. Can't seem to finid any information on this file ahywhere (or have not had any luck so far anyway).

[Pack3tL0ss]

Installlock appears to be a kind of pid file.  If you start the installation process to add or upgrade a module it seems to change an attribute for that file (if you right click and go to properties --> details).  It has an attribute "N" when nothing is going on, and "A" when you've clicked the install button and have the installation process running.  So it's just used to tell the installer if the process is already running or not so it doesn't spawn multiple installation processes, which could likely bork things.

it's the installinfo.txt file that has the hash for the originally specified database location/instance/credentials/etc. 

For the database connectivity issues on the temp server.  IMC needs the SQL client components from the MS SQL install media installed on the IMC server, without it it won't connect to an external database.  I thought that when it couldn't find the client it automatically assumed you wanted the embeded database (MS SQL Express) installed co-resident with IMC and would start installing it, but they may have changed that.  At anyrate I'd verify the client components are installed, the required pieces are documented in the IMC SQL Server XXXX installation guide (One of the docs in the HPE IMC documentation)... 

Again for anyone else with issues if you've moved the IMC database at anytime after the original IMC installation, (and didn't go through the process of setting up a temporary IMC server), then the installinfo file is the source of the issue.  At the time of the original install (which would be the original database) that file is created and crucial information is placed there, but it's hashed so it can't be modified.  Support has a tool to update the file and hopefully rectify the situation.

[kortemi]

Thanks, Pack3tL0ss

The SQL Server management tools are installed, and I can connect to the restored (with SQL itself) databases so that's not an issue.

But, I'm beginniing to suspect that the root of the problem, as you state, lies indeed in the installinfo.txt file. I have a ticket open with HPE support, and their first suggestion was to install a temp IMC installation, pointing to the new DB. Unfortunately I still have the problem that during installation IMC says that it is unable to connect to the database (even when I install IMC on the db server)..

[kortemi]

My issues have been raised at HPE support to the highest level (levels 1 and 2 were unable to assist), but since I had had very little feedback from HPE, and feeling frustrated with the situation, I thought to try a yet another approach.

1) I removed the SQL Server Instance on the separate SQL Server.

2) In installed Microsoft SQL Server 2014 SP1 on the same server where IMC was installed, only default instance (no instances this time)

3) I uninstalled IMC (but left the data behind, when asked by the uninstall)

4) I tried to install IMC (7.2 E0403) again by running the install.bat from the folder windows\install

5) I then left Country/Region and Language as suggested by the installation program, and selected Custom as the Installation Type

6) After clicking on OK, I get presented with the Checking Database Connectivity dialog box. In it, I select the following:

 a. Database Type: Microsoft SQL Server

 b. Instance Name: Default Instance

 c. Superuser: sa

 d. Password: < pw>

 e. Database location: local host

 f. Database server address: <greyed out> 127.0.0.1

 g. Listening port: 1433

7) When I press OK, I get the same error message:  Error: Connecting to the database failed. Make sure that the database is running normally and you have input correct login information

 a)  This turned out to be a disabled sa account (I had chanded the pw, but that did not enable the account). Once I enabled the account, I was able to move forward with IMC installation.

8) However, IMC is now at a fresh install state.

 a) I then tried to do a restore of all database files from the IMC Deployment Monitoring Agent, and I got an error.

The dbman log files give some info on this:

At the end of the dbman_debug.log file there are the following lines:

2016-02-11 13:27:53 [ERROR] [checkDBFiles] Restore all database failed: Components is not match

2016-02-11 13:27:53 [ERROR] [checkDBFiles] Database: 127.0.0.1@config_db. Components: iMC-PLAT=7.2.E0403 iMC-GAM=7.2.E0403 iMC-EUPLAT=7.2.E0403 

2016-02-11 13:27:53 [ERROR] [checkDBFiles] DbFileConf: <oldIP>0@config_db_imc_config_db_20160108_122120_full.db. Components: iMC-PLAT=7.2.E0403 

2016-02-11 13:27:53 [INFO] [sendTrapAlert] Send trap success

2016-02-11 13:27:53 [ERROR] [ManualRestoreBase] Fail to check DB file.

2016-02-11 13:27:53 [ERROR] [response_err_code] errCode = -1

The logn states “Components is not match”, and this would indicate that the restore components are not the same as the previous install.

9) Then I thought to try to restore the databases one at a time, and now I managed to manually restore all databases, except for the config_db. This is the one that gives me the error I referred to.

So, it seems that the reinstall probably does not match (component-wise) the previous installation.  BUT, since it's nearly two months since my problems started, does anyone know, if there are any tools that I could use in order to figure out which components I am missing?

[kortemi]

Just to wrap this thing up (and maybe help whoever's looking for help in the future):

I managed, after all, to sort the problem out.

I noticed that there were two services that were by default undeployed (User Serservice Management and Guest Access Management). I deployed these, and then tried to restore the config_db, and this time it worked.

So, now our IMC is working on a new database.

[Jannie Hanekom]

In case it’s of any use to anyone else ever stumbling on this discussion thread (as I did):

Take backups.  Of the *entire* system – not just databases.  Before, during and after making changes.  I can’t stress that enough.

Named instances are tricky to deal with.  Although the documentation makes mention of the SQL Browser service being required in order to locate a named instance, I was unable to get it to connect to the named instance until I manually set the TCP port the named instance operates on, and configured the installer to point to that port rather than to 1433/tcp.  Useful tip: The “ERRORLOG” log on the SQL server in the \LOG directory of each instance is invaluable in troubleshooting which instance is being connected to. **EDIT** I also found what appears to be a bug in the reporting component when dealing with named instances.  More at the end of this post. **/EDIT**

The best way to migrate the database is to use the procedure provided by HPE (install a clean instance of iMC on a third server that points to the SQL server and get that working, then copy the correct config files across.)  Manually hacking at configuration files is incredibly tedious and error-prone.  It also has the challenge that you need to manually create the SQL Logins and set their passwords, or use the sp_help_revlogin method.  Using the installer and copying the config files across eliminates all of that.  See https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c05162581 and https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c05162582

As I don’t use the sa user for the Deployment Utility, I also had to manually create a “deploy” login on SQL and set its password in the Deployment Utility, so that one is able to see database statistics in the “Environment” tab of the utility

I did find that – in my case at least – copying over the two config files in the documentation was insufficient.  I used the list that “kortemi” provided: **EDIT** For the reporting component, copying across the files does not matter.  The "dataSource.xml" and "<serverip>.xml" files get dynamically generated on iMC startup from server-addr.xml. **/EDIT**

C:\Windows\imc-reserved\instinfo.txt
..\IMC\common\conf\server-addr.xml
..\IMC\deploy\conf\component-env.xml
..\IMC\deploy\conf\dbconfig.xml
..\IMC\dbman\etc\dbman.conf
..\IMC\client\web\apps\imc\reports\iMCCRconfig.xml (did not exist in my installation)
..\IMC\client\web\apps\imc\reportsv2\dataSource.xml
..\IMC\client\web\apps\rptviewer\WEB-INF\repository\data\<serverip>.xml

 _{* (at present, my Reporting capability still does not work.  I'm guessing I missed something somewhere.) **EDIT** Reporting fixed, see end of post **/EDIT**}

For what it’s worth, below are some extracts of how a named instance is configured in the configuration files.  It’s clear that you have ZERO chance of guessing the format correctly.  Installing a temporary copy of iMC is the only way to get these right.

..\IMC\common\conf\server-addr.xml

  <component address="127.0.0.1" id="iMC-ICC">
    <db-config address="MYDBSERVER" dbname="icc_db" instance="IMCDB" password="-105-61-35-7-31-18-210-210-181-198-178-166-177-162-102-142-126" port="1435" type="SQLServer" username="imc_icc"/>
  </component>

 ..\IMC\deploy\conf\component-env.xml 

  <component id="iMC-PLAT">
    <var name="DATABASE_PORT" value="1435"/>
    <var name="DATABASE_ADDRESS" value="MYDBSERVER"/>
    <var name="DATABASE_TYPE" value="SQLServer"/>
    <var name="DATABASE_NAME" value="config_db"/>
    <var name="DATABASE_PASSWORD" value="{CT}|LtT2SCrxTkCb3k7X9q45dA=="/>
    <var name="DEPLOY_IP" value="127.0.0.1"/>
    <var name="JDBC_INSTANCE_NAME" value=";instance=IMCDB"/>
    <var name="DATABASE_INSTANCE" value="MYDBSERVER,1435\IMCDB"/>
    <var name="DATABASE_USER_NAME" value="imc_config"/>
    <var name="INSTANCE_NAME" value="IMCDB"/>
    <var name="VERSION" value="7.3-E0506"/>
    <var name="INNER_VERSION" value="V700R001B06D018"/>
  </component>

 ..\IMC\deploy\conf\dbconfig.xml

<item compId="iMC-PLAT" dbAddress="MYDBSERVER" dbName="config_db" dbType="SQLServer" dbUserName="imc_config" forceDropWhenRemove="false"/>

**EDIT** Fixing Reporting after migrating the database to a named instance: (and probably on a fresh install too!)

After migration, I found my reporting feature did not work.  At first, I thought I needed to add a separate data source in iMC.  I tried that, but compared it to a known-good iMC that was installed with an external database from the start, and found that it wasn't necessary: the "127.0.0.1" data source was quite acceptable.

What I did find was that the "datasource.xml" and "127.0.0.1.xml" files in the report folder contained only the SQL Server name and port number, NOT the instance name, like so:

...<url>jdbc:sqlserver://MYDBSERVER:1435;databaseName=report_db</url>...

Editing this file directly is futile, as it's automatically overwritten again when you restart iMC for the change to take effect.  I eventually found that it's getting the details from server-addr.xml. However, in that file, the instance name was indeed specified, as it was for all the other DB's (see "instance=" parameter.)  It seems the parser that builds the "datasource.xml" and "127.0.0.1.xml" ignores this parameter...

  <component address="127.0.0.1" id="iMC-REPORT">
    <db-config address="MYDBSERVER" dbname="reportplat_db" instance="IMCDB" password="-105-61-35-5-31-250-237-242-184-229-183-156-183-151-132-142-126" port="1435" type="SQLServer" username="reportplat"/>
    <db-config address="MYDBSERVER" dbname="report_db" instance="IMCDB" password="-105-61-35-38-247-15-18-245-240-223-210-196-159-153-141-115-95-83-79-61-62-46" port="1435" type="SQLServer" username="report"/>
  </component>

I then changed the server-addr.xml file to read as follows:

  <component address="127.0.0.1" id="iMC-REPORT">
    <db-config address="MYDBSERVER\IMCDB" dbname="reportplat_db" instance="IMCDB" password="-105-61-35-5-31-250-237-242-184-229-183-156-183-151-132-142-126" port="1435" type="SQLServer" username="reportplat"/>
    <db-config address="MYDBSERVER\IMCDB" dbname="report_db" instance="IMCDB" password="-105-61-35-38-247-15-18-245-240-223-210-196-159-153-141-115-95-83-79-61-62-46" port="1435" type="SQLServer" username="report"/>
  </component>

When I started iMC again, the "datasource.xml" and "127.0.0.1.xml" files were built "correctly", and my reporting started working again:

...<url>jdbc:sqlserver://MYDBSERVER\IMCDB:1435;databaseName=report_db</url>...