topic Re: write back caching and data loss in Operating System - HP-UX

write back caching and data loss

Scotty HD — Sat, 12 Jun 2010 15:46:02 GMT

i read about write back data caching and have question

is HP-UX write back data caching ?

when data in cache dirty, if system goes down. is it data loss?
is there some way to get that data back. like some utility that
read the system core dump and rebuild the cache or something like that.

Scotty

Re: write back caching and data loss

Bill Hassell — Sat, 12 Jun 2010 16:54:24 GMT

HP-UX has disk data in several places, not just the buffer cache, which is primarily filesystem data and directory values. There can be memory mapped files which are also memory resident as well as possible page-in and page-out activities plus all the programs with local data and shared memory segments.

So if the system goes down, memory is never saved unless the system performs a TOC or has a system crash. Then a memory dump is taken with the contents sent to the dump area. However, today's RAM can be enormously large (32 GB to 256 GB) so not every part of RAM is saved. With crashconf, the selected memory sections can be included or excluded.

Although there is no utility to "rebuild the cache" since there would be no way to start all the programs in a way that they could synchronize and use the data.

The first step towards reliability is power and air conditioning. A simple UPS for a server is not enough -- the data center needs a big UPS with a backup generator. Or the big UPS comes with software to force an orderly shutdown of every system *and* disk arrays. Similarly, the air conditioning must also provide an orderly of all servers when the temperature exceed 95F or higher.

Programs and databases must be designed for crash recovery -- it will happen. Crash prevention includes patching the OS at least twice a year and the applications and database engine as required.

Re: write back caching and data loss

Dennis Handly — Sat, 12 Jun 2010 21:48:45 GMT

>I read about write back data caching and have question

Are you talking about the cache in disk controllers or what?

There are CPU caches. The file cache in RAM and various caches in array controllers.

Re: write back caching and data loss

James R. Ferguson — Sat, 12 Jun 2010 22:07:40 GMT

Hi Scotty:

When using the JFS (VxFS) filesystem, various mount options control data integrity across a crash. You can ramp up performance at the expense of some data loss (e.g. mounting with 'tmplog') or chose to record data and metadata changes before returning to the calling program with the 'log' mount option.

See the manpages for 'mount_vxfs' for more information.

Regards!

...JRF...

Re: write back caching and data loss

Scotty HD — Sun, 13 Jun 2010 05:09:35 GMT

Dennis Handly>>
i was referring to data cache which termed as buffer cache in hp-ux.

Bill Hassell>>
did you mean,
if system crashed due to some bug in some component and lets say there is
dirty data in cache. then that data will be lost.
the program/application should be written such that it would deal with this
scenario ?

Scotty

Re: write back caching and data loss

Dennis Handly — Sun, 13 Jun 2010 06:56:26 GMT

>termed as buffer cache in hp-ux.

On 11.31, it is now called the Unified File Cache.

>the program/application should be written such that it would deal with this scenario?

Are you using a database? They may have their own transaction managers to handle this.

Re: write back caching and data loss

Bill Hassell — Sun, 13 Jun 2010 10:41:17 GMT

> if system crashed due to some bug in some component and lets say there is
dirty data in cache. then that data will be lost.

Correct.

> the program/application should be written such that it would deal with this
scenario ?

Yes.

When any computer crashes, everything you know about the OS is gone. In HP-UX, if there is an OS crash, selected portions of memory are written to the crash dump area on disk (assuming that the disk is working). Then the system boots up and you start your application again. Assuming that you knew enough about the HP-UX kernel and memory mapping (and adb and other tools), if you had a copy of the data, what would you do with it? Your program has no way to know where it was when the crash occurred.

Your programs should be written to recover from a crash, that is, keep transaction and rollback logs. Given that a well-patched HP-UX system very seldom crashes, I would be more concerned about mirroring your vg00 disks, taking regular Ignite backups and rehearsing a hard disk recovery.

Re: write back caching and data loss

Scotty HD — Mon, 14 Jun 2010 12:07:16 GMT

Bill Hassell>>
well written reply. easy to understand.

if talk about writeback advantage.
does anybody have statistic as to how much percent faster is writeback
compared to say writethrough data caching.

Scotty

Re: write back caching and data loss

Bill Hassell — Mon, 14 Jun 2010 13:56:26 GMT

> does anybody have statistic as to how much percent faster is writeback compared to say writethrough data caching.

I think you'll have to explain your terminology. These terms are normally applied to the electrical design of a CPU chip or to the design of a data cache such as the HP-UX buffer cache (Unified File Cache for 11.31). You can learn all about it here:

http://h20338.www2.hp.com/enterprise/w1/en/os/hpux11i-kod-unified-file-cache.html

Re: write back caching and data loss

Scotty HD — Thu, 17 Jun 2010 13:12:50 GMT

Bill Hassell>>
by cache i refer to buffer cache (data cache)

write through
Application does write IO
- data written to cache
- data written to disk
- IO completes

Write back
Application does write IO
- data written to cache
- IO completes
- data written to disk based on write back policy

write back give better performance compared to write through.
i wanted to know if anybody has some percentage numbers.

Scotty

Re: write back caching and data loss

Scotty HD — Thu, 24 Jun 2010 14:19:31 GMT

experts,
any idea on this ?

Scotty

Re: write back caching and data loss

Scotty HD — Fri, 09 Jul 2010 13:34:31 GMT

no replies.
nobody came accoss the statistics for these?

Scotty

Re: write back caching and data loss

Dennis Handly — Fri, 09 Jul 2010 17:49:48 GMT

>nobody came across the statistics for these?

I'm not sure what you would do with these statistics?
What you are asking for is HP-UX design considerations.

Re: write back caching and data loss

Scotty HD — Fri, 30 Jul 2010 14:42:37 GMT

closing this thread

Scotty