Client Support
Showing results for 
Search instead for 
Do you mean 

Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Frequent Advisor

Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Hi, my environment:

Hp TRIM 7.1.1828 Win 7, MS Office 2010 ,Server 2008R2

 

What to do when Scheduled Event Triggers are accumulating at the rate faster than they are processed?
E.g. hourly, we create 1200 events and they are being processed at approx 50 per hour.
Based on this, we never going to decrease the load.

When there is not so many events accumulated, the rate is faster...Now, it looks like that there is no solution .. 400.000 event triggers are there now and constantly increasing.

The re-start of the server does not help.

Any advice on this? It is kind of urgent.

 

Is this issue exist/fixed in HP TRIM 7.3.2.5511?

Thanks

 

 

20 REPLIES
Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

This was improved in later versions of 7.2.x, and in 7.3.x. KM1420410

 

Also, there is an excellent knowledge document regarding this, KM765264.

 

I have removed all the existing events and manually recalculated specific schedules before, but it is time consuming to identify what needs to be recalculated.

Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Thanks,

so there is no solution for my problem with the events?

Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

What's actually happening with TRIM? Is someone performing a large migration, or moving classifications around?

 

These events are kicked off because record objects are being created or moving within the system. If the rate at which they are being generated has ramped up, that indicates abnormal activity.

 

If a large number of records are being imported, then it is worth simply turning off the event processing for these triggers for the duration of the import.

Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Yes, large data migration. I recevied a few suggestions what to do "next time" but not how to fix the current issue .

HPE Expert

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

[ Edited ]

Has the large data migration already finished? If that's what triggered the large backlog of events, perhaps if you're patient the system will eventually start catching up. It's the speed at which the database engine can process the fairly complex queries that determines how quickly they process. Perhaps ask your DBA if there's any way they can throw extra resources at it until the backlog's processed?

 

There are ways to intervene and remove the backlog from the queue, with a view to re-processing them manually (as outlined in the knowledge document mentioned in an earlier post), so if you're particularly impatient you may wish to log a support case with HP Support asking for assistance.

 

One of the good things about scheduled event triggers is they're the sort of thing that only ought to be up to date when the records people decide to do some scheduled disposal activities. Sometimes this is quite infrequently (e.g. annually) so you may want to speak with them.

 

The other thing you always need to monitor is whether people are putting a large number of contents into containers (300+ is considered large). The TRIM documentation recommends not containing more than 300 documents in any one container, and the automated part rules feature is provided to allow you to configure TRIM to avoid this. Perhaps some of the imported records fall into this category, so it may not be the sheer volume of them that's the cause.

 

Here’s a script your DBA can run to identify containers in your database with more than 300 contents:

 

select b.title,
            b.fullrecordid,
            count(a.uri) as #Documents
from TSRECORD a, TSRECORD b
where a.rccontaineruri=b.uri
group by b.title,
            b.fullrecordid         
having (count(*) > 300)
order by 3 desc


Neil

Note: Any posts I make on this forum are my own personal opinion and (unless explicitly stated) do not constitute a formal commitment on behalf of HPE.

(Please state the version of CM you're using in all posts. At any given time, HPE are supporting approx. 30+ released versions)

HPE Software Support Online (SSO): https://softwaresupport.hpe.com/
Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Hi,

the data migration is not finished and also much more data migratrion is about to start.

Events are still building up.

Somebody suggested that more resources could be added...Not sure how that will reduce the number of events... They are already there.

I sugested the removal of the known events and adding them later...when safe...

 

The tool is implemented to monitor TRIM files with more than 999 records and there are a few hundreds of these...some old and some still active ( documents are added daily).

 

Is this really a "catch 22" situation?

 

Local HP rep was contacted as well...

Will let you know how this will be solved (next week I hope).

Thanks

 

Honored Contributor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Complete your import, clear up folders with more than 300-400 items (new parts), purge all of the entries for Scheduled Event Triggers, and then recalculate triggers.

 

Your build is just a problematic build (as with all 7.1.x) with regard to SET's. 

Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

It is not so easy to create new parts as some business applications using the same file for the last 3-4 years...

 

Not sure how to purge all entries for triggers (only related to my import?) ...

 

Then, new imports will do the same thing? Create so many event triggers ....

 

Regards

 

Honored Contributor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

I use a utility that does all this automatically, because it is a big problem. If you don't parse out the folders with lots of content then the issue will keep coming up each time you add a single document to that massive folder.
Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

 

800.000 events and raising.

 

2 questions:

1. A test was done: The corporate file where the records are migrating to is wihout the retention schedules, so how many triggers events are created per, let say 1,000 records. The answer is = 2,000 events. The migration was done not through the main central server, but through some other one.

 

The question is: Are these events, although processed by the other server, ever end up with the main central server or not at all, e.g. these 1000 records imported that created 2,000 events through the other server were processed by that server and nothing eventswise was added to the main server? We are trying to use the other server without influencing the main server, e.g. not to add these events on the 800,000 events to become 802,000 events...

 

2. Would a drastic rebuild of a main (central) server help processing the outstanding events?... (We are staying for now on the same TRIM Version, but that new server has 12 time better CPU and memory power)

 

New version of TRIM will be implemented in January 2014, but we can not wait for so long with these events problems

 

Thanks

 

 

 

 

 

Honored Contributor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

1.  That is expected.

2. Possibly.  The issue is more in the configuration of your record types, retention schedules, and triggers.  Throwing more resources may help, but if you don't solve the underlying problem then it's not going to go away.

 

Use this link for more information:  http://support.openview.hp.com/selfsolve/document/KM765264?searchIdentifier=31e035dc%3a13f88a33b01%3a393b&resultType=document&documentURL=KM765264&resultsURL=%2fselfsolve%2fdocuments&allowReturn=true&searchString=trigger&searchKey=2013-06-29+05%3a41%3a46

Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

I am sorry, what is expected? For these trigger events to actually be added to the main server? To increase the number to 802,000 events?

 

Why then we are doing the processing via different server from some other office e.g.? Or from virtual server that is on the same server as the main one, physical one..

 

I am sure that  configuration of our record types, retention schedules, and triggers is fine... You see, the files were the records are imported to, do not have retention schedule now....

 

2. I can't open the document as my account is for the forum only... any chance someone posting it somewhere....? I want to analyse these underlying problems ... if possible...

 

So, new stronger server cannot help the actual processing?

 

Regards

 

 

Honored Contributor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

1.  You last wrote: "what is expected".  What's expected is that you wrote previously.  You imported 1,000 records and 2,000 events got queued.  The documentation explains why this is the case.

 

2.  I can't explain why you're "processing via different server from some other other" as I don't even know what that means.  These are events and events are usually processed on a different box from the workgroup server.

 

I've uploaded the document I previously linked.  This provides a very good explanation as to the underlying problem and how to work around it (without upgrading).

 

And unfortunately your configuration is not fine.  You've got containers with lots of contents.  That is what I mean.  The fact they don't have a schedule doesn't make a difference.

Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Thanks for the document, I am analysing it...

 

What I meant to say is this:

 

I am trying to avoid adding the events to the main server ... Does not matter if another 30,000 events ( from 15,000 records) are added to some other virtual server that is on the same physical server where the main server is.

 

In regards to the number of documents per a container, the intention was to have 8,000 documents per container which does not have the retention...but as you said before, obviously, the events are created anyway.

 

Thanks, I am going to analyse the document now.

Cheers

 

Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Just to add, in my testing environment, the automated part rules were not functioning quite right...as I was able to import records over 300 items... ..Only when I stopped the import (using TRIMPORT.EXE) , and after I hit F6 , the container without volumes become volume 1 (there was also Vol 2 created) and just then when I tried to add mor erecords there, the warning popped up saying that container is closed..

 

Regards

 

 

Honored Contributor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Having 8,000 documents in a container will cause the scheduled event triggers to bog down.  The documentation explains that you'll need to ensure the folder has a trigger based on date closed (or something equivalent).  The problem here is that TRIM is trying to determine if the folder needs to have it's destruction date changed, regardless of whether it does or not.  Each time you add a new document it's queueing up mor events, and when that event is processed it's going to evaluate every record in that container.  So the more you add the worse it gets.

 

As for the automated part rule, that is also event based.  So eventually the event server sees there's too many things in the original container and then creates a new part.  The creation of  a new part automatically closes the previous part.  There is a system setting which defines whether you can continue to add to an existing part (Tools->HP TRIM Administration->System Options->When Placing Records in a Closed Container.  Note that if you continue to add to an existing part after the automated part rule has been executed, it will not move items off into the other part for you later.

Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

We deleted 1 Million events, not sure if we ever going to use the triggers re-calculations.

Can anyone please explain how the "events engine" - triggers calculations are improved in HP TRIM 7.3.3 , e.g. still the same number of events is created per 1 record save / modification or this is improved and how?

Regards

 

Honored Contributor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

The "optimize management of contained records to allow for very large folder size" option has improved this somewhat, but the create of events per save/modification will still be there.... depending on your configuration.

Search for "Large Folders" in the TRIM help for a detailed explanation.
Frequent Advisor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

Thanks,

I would also need information if anyone has done large imports of data these days in HP TRIM 7.3.3., and how events behavied there.... To think whether we enable them or not.

Regards

 

Honored Contributor

Re: Scheduled Event Triggers issue - Big problem, 7.1.1828 - Urgent

It's better to just turn them off and do one big recalc. At least that's how I do it.
//Add this to "OnDomLoad" event