iSPI Traffic 9.1 ETL failure results in mass file build up

NNMi Advanced 9.1 in 2 node app failover configuration.

Traffic Extension and Leaf Collector are installed on the NNMi management servers.

The Master Collector is installed on a dedicated NPS.

Netflow is v5.



I am having an issue where the following iSPI Traffic ETL jobs fail, sporadically:


* ETL.Interface_Traffic

* ETL.Interface_Traffic_1_min

* ETL.Interface_Traffic_15_min


This causes a build up of Interface_Traffic_*.gz files in the \\.....\perfSpi\datafiles\metric\final directory. If left unchecked over the weekend we're talking hundreds of thousands of files. The current file count is around 322,000.


When I restart the NNM iSPI Performance ETL Server service on the NPS the ETL jobs will restart and the file count in the above directory will start to decrease, however, it seems the bulk load only processes 12 files per run. I did a little math and at the current rate of load it would take over 20 days to process all these files.


I need to know if there is a way to increase the number of files processed per ETL execution or whether there's any other way to get these files processed quicker.


Also if anyone has had any experience dealing with iSPI Traffic ETL jobs failing from time to time and have found a resolution I'd be happy to hear about it.


I have iSPI metric installed and it runs flawlessly... anyone else experiencing issues with iSPI Traffic 9.1 stability?