pstat header files out of date??

[Phillip Bell]

Hi Guys,
My company has a little performance agent that collects all the performance stats we think we want and stores them in a file which another program reads and produces reports later. Which is great.

We recently installed the large patch bundle released in june/july. The format of the statistics changed yet the header files remained the same and dont match the data returned.

So i'm trying to find out if there is a later version of the header files than the ones I have.

The header files i'm looking at are in the /usr/include/sys/pstat folder. Specifically i'm interested in pm_pstat_body.h

The modified date for most of my files is 14/Nov/2000
The modified date for my pm_pstat_body.h is 06/Sept/2004

I've attached a zip file with all my header files in it if you fancy doing a comparison.

Any help or just modified dates of your files appreciated!

Cheers
Phil

[Phillip Bell]

Sorry forgot to mention i'm on 11.11.

Cheers
Phil

[Zeev Schultz]

I didn't see particular patch for pstat header files. Some do change to this or that header but I don't know what kind of change.

Anyway, the fileset is ProgSupport.C-INC and you can list in /var/adm/sw/swinstall.log and see what patch touched this fileset.

I checked on hpux 11.23 that I have and all pset headers are dated Mar 26 2004. On 11i in my case all are dated Nov 14 2004.Sizes are different and I can see line here or there for 11.23 files.

I still wonder why do you need this info for... :)

[Phillip Bell]

I dont suppose you can point me to somewhere where I gan download the latest ProgSupport.C-INC fileset (Assuming it's compatible with 11.11?). :-)

Or could you supply a copy of your pstat header files so I can look at them and see if they match the data we got?

The reason we are after this information? I presume you mean why we want the headers? We have "extra" data returned from the OS. We are not sure what it is as it's not defined in our headers... Which means that our program trying to analyse the data gets a little "confused"!

Cheers! :-)
Phil

[Zeev Schultz]

It's not downloadable as its a part of ProgSupport.C-INC - meaning all system headers in /usr/include/sys etc.

Basically there are only 3 patches in HPUX 11i that mess with /usr/include/sys/pstat directory. And those are (with files they change) :

PHKL_28244 (/usr/include/sys/pstat/crash_pstat_body.h)

PHKL_24583
(/usr/include/sys/pstat/pset_pstat_body.h)
PHKL_24582
(/usr/include/sys/pstat/global_pstat_body.h)

none of them dates to Sep 2004. So look somewhere else. I'd start with checking "/var/adm/sw/swinstall.log" to see what touched ProgSupport.C-INC ...

[Don Morris_1]

There's no difference between your pm_pstat_body.h and the 11.11 LR version of the file -- I can't say why the timestamp has changed (the patch install did a touch?)

Could you post what pstat function is returning extra information on you. Knowing what structure has a size discrepency between user and kernel would help.

Thanks.

[Phillip Bell]

Thanks for the help guys it's appreciated!

In the pm_pstat_body.h file the last field is pst_pset_id. But the data returned to us has more data beyond that. So we were hoping to find a header file that would explain what the data beyond pst_pset_id is.

We also have a similar problem in the processor data but i'm going to check out PHKL_24582 to see if that explains whats going on in that area.

Cheers
Phil

[Don Morris_1]

Ok, I have to confess I don't get it. There's nothing defined in the struct past that point (user or any 11.11 kernel I can see) -- which means that if your app thinks it gets more data, then it must be reading past the end of the structure. What's after the structure depends on where it was allocated, but it is more likely than not something else on your heap.

Can you post the code where your app does the pstat call and then where it dereferences the struct? Because it sure seems as if it isn't using sizeof() as it should or just dereferencing the fields directly.

[Phillip Bell]

Ok thanks again guys. I managed to get this out of the guy who wrote the program:

"The call to the dynamic information indicates the record size for the pst_status record is of a certain size (as provided by the operating system). The size of the supplied header for pst_status record is 172 characters smaller than indicated by the value provided by the operating system. For the processor record, the size provided by the operating system is 12 characters larger than the corresponding header."

SO I may have been inaccurate about what we're struggleing with. Sorry about that. Anyway if you have any ideas on the above quote i'd love to hear them.

Cheers
Phil
(If your wondering why the developer (in US) isn't pasting these questions...the HP-UX system happens to be in my office (in UK) and therefore it's me running around finding him information. Damned office politics! ;-) )

[Don Morris_1]

Are the callers the same application? How is he calculating the "supplied header size"?

My take on what's going on:

pst_static_size is the field in the pstat static structure I assume he's referring to. That value gets set to the expected size of a pst_status structure *for that application*. A 32-bit (narrow mode pstat) pstat call will get a smaller size than a 64-bit one. So if he's getting the value from a 64-bit application and then doing sizeof() to the structure in a 32-bit application, he's going to see definate size differences.

If this is in the same app, then pst_static_size should agree with sizeof(struct pst_status). Make sure that all .o's in the app are compiled consistently with regard to -D_PSTAT64 [either use it for all or none.. if you mix and match then you get this kind of mess..].

[Phillip Bell]

Thanks guys I think you've been a great help. :-).

I'm going home! :)

[Phillip Bell]

Hi Guys,
I got an e-mail from the developer overnight trying to answer some of your questions the contents are below. If you have any comment on this let me know. Thanks again.
Phil

1. There is only one application, the one collecting the information. This is compiled on the collecting machine with the headers and settings defaulting to the system. The object is not shipped to the system, only source. The application must be compiled before it can be used, therefore is will always use system consistent settings. (The application does not use any settings just for this reason. It will always use the system defaults).

2. The call is indeed to retrieve the static values. I use nine of these type of records in the application. Of these nine records, seven of them match in size with the headers (structs) that I have. The other two are processor and status. The static all indicates the processor record is 12 bytes larger than what I see, and the status record is 172 bytes larger than what I see.

3. The data in the processor and status records is correctly aligned for what data is there. That is to say, if there was 64 bit fields, they would not align properly, yet I am getting correct intermediate values.

4. If this was a 32bit vs 64bit issue, the seven other structures would surely be different as they use the same defines as the processor and status records.

Out of nine structures that I use, seven of the are consistent with the previous release. The other two have changed in size from the previous release.

[Stephen Keane]

This might be your problem (from /usr/include/sys/pstat/global_pstat_body.h) ...

The application will experience binary compatibility problem under the following condition:

Application is compiled with the new fields of the pstat data structure, and it runs on the kernel which has not been patched and therefore does not have the additional fields.

When the application requests more information than what the kernel can provide, it will fail. pstat() cannot return any valid information beyond what kernel data structure is kept and maintained.

To ensure binary compatibility, the "_ICOD_BASE_INFO" macro has been added to the pstat data structure.

The macro will condition the new pstat data structure. So, when the application writers need to access the new pstat data structure on the new kernel, they should compile their applications with the "_ICOD_BASE_INFO" macro.

With this in mind, the pst_processor struct has 3 additional fields when _IOCD_BASE_INFO is defined. These are 4 byte fields and 4 x 3 = 12 bytes (as you've experienced. I don't know which structure you are referring to when you say "status", but I expect a similar situation is occurring.

The answer then is for the developer to understand the macro as described above, or for you to patch your kernel to the same level as theirs.

[Phillip Bell]

Thanks Stephen. That sounds like a good lead.
I'll let you know if defining that macro works.
:-)

[Phillip Bell]

Ok it looks like we got somewhere with the Processor structure size.

I got this back from the developer.

"This is fine for the processor record. Those fields are present and there is an option to included. This structure I do not have a problem with as the data is mostly there and intact. I can correct this record is required, but is not an issue.

The problem I have is with the status structure. The structure definition I have is 172 bytes smaller than what the operating system says is there. I need to know what these 172 bytes are as this is a ton of data I cannot see."

So he's still having trouble with the size of the Status structure. Before the large patch bundle when we asked the OS the size of the structure it reported it to be 596 bytes. Which matches the structure defined in the header. After the update when we ask the size of the structure the OS says it's 768 bytes, but the header (unchanged) still says it should be 596 bytes. Thats our 172 byte difference. While we are happily collecting and storing this data in the file we are unable to interpret it.

I'm starting to get the impression not many people go and look at this 'pst_status' structure... ;-)

Phil

[Stephen Keane]

The size of the pst_status (actually __pst_status) structure is 768 bytes. The header file you posted agrees with this. When you say the OS reported the size as 596 bytes, do you mean using sizeof(struct pst_status) ? Can you get your developer to run a quick C program reporting sizeof() the following and see if it agrees

sizeof(pst_status) is 768 bytes
sizeof(struct __psdev) is 8 bytes
sizeof(struct __psfileid) is 16 bytes
sizeof(struct pstsigset) is 32 bytes
sizeof(struct __pstcycles) is 8 bytes
sizeof(struct __psttime) is 8 bytes
sizeof(struct __pst_fid) is 64 bytes

Also what is PS_FID_SZ defined as in /usr/include/sys/pstat/fid_pstat_body.h?

[Phillip Bell]

Hi Stephen,
Thanks for helping out. :-)

It looks like everybody is in agreement that the size of the pst_status structure is currently 768 bytes.

However the developer thinks the header is saying the structure should be 596 bytes (Which is what he says we used to get before the patch bundle). And you say the header I posted says the structure is 768 bytes. So i'll get back to the developer and make sure he takes a good close look at the header again to see what he's missed.

Thanks Again.
Phil

p.s.
This is the line from the code I think is doing the sizeof() for pst_status.

proc_table[i].status = calloc (1, sizeof (struct pst_status));

[Stephen Keane]

Unless your developer is talking about the field pst_status_size in the pst_static struct. This is supposed to return the size of the pst_status data and should agree with the structure definition in the header file.

You could try compiling and running the attached C code. You can complile it with the built in C compiler

cc .c -otest

[Stephen Keane · Accepted solution]

That should have read

cc test.c -otest

sorry about that.

[Phillip Bell]

Ah ha! This is the output from your program. :-)

pst_static.pst_status_size = 768
sizeof (struct pst_status) is 768 bytes

I think this means that the OS and the header are both saying 768 bytes. And so long as thats using the header I attached (and thats the only file with that name on the system), then it points to the developer missinterpreting the header?

Thanks again! :-)

[Phillip Bell]

Thanks Stephen!
The developer finally admitted the possibility his code might not be perfect. And guess what, found a problem which he fixed.

Thanks!
Phil