ODBC connection

[Riverhawk]

We have a non-VMS webserver with the Db running on VMS.
Recently the site went down and after checking everything else the problem was resolved by rebooting the VMS server. Can anyone shed any light as to what service or issue was possibly resolved with a reboot.

We currently are waiting for approval to install the lastest/last version of DECevent so we don't really have access to all of the logs that might help us.

We are very inexperienced with VMS.

We are running Openvms 7.2-2.

Please let me know if you need any othe information. I'm just trying to get an idea of what the problem might be as this has happened a couple of times in the last month.

Thanks in advance for your help.

[marsh_1]

hi,

what database/version is running on the vms server and what is the webserver and platform.

[Hoff]

There is an infinite number of potential reasons for whatever happened here.

A similar outage could easily be triggered by hardware errors or by software problems or network problems or web server problems or database problems or, well, anything? This is a huge area. I could see a wedged network switch port, for instance, causing this.

If you're running production and are left to reboot servers as your solution to what looks to be a critical outage, you'll want to call in formal help either per-call or as part of a support service offering.

Or to migrating the databases to a platform you're more comfortable in maintaining and operating.

If this happens again, investigating the state and potentially forcing a system bugcheck and writing a crashdump from the system console would be minimally necessary. The crashdump will at potentially some idea of the state of the system when the problem arises. But that may or may not isolate this.

Your organization need an escalation and support channel here that's rather better and more direct and more immediate than ITRC discussion forums.

Stephen Hoffman
HoffmanLabs LLC

[Riverhawk]

Thanks Hoff your a DICK!!

[Jan van den Ende]

Uhm,

my (non-native) English may be less than perfect, but I read

>>>
Thanks Hoff your a DICK!!
<<<

as not very friendly. (and PLEASE correct me if I am wrong!!!)

Fact is, maybe Steve can be somewhat candid at times, he is VERY WELL in VMS and in (not necessarily VMS-friendly) broader aspects of IT in general.

Even (or maybe, JUST) when his comments contain (some or more) criticism, they are worth reading, and then studying again.

I may not always share his views, but more often than not I have had to review MY take (sometimes MUCH later.

Bottom line: swallow your pride, and take his comments for their content.

Really, he DOES know what he is talking/writing about!

And apologies if my take on this was wrong. That would imply my English needs polishing.

Proost.

Have one on me.

jpe

[Andy Bustamante]

Also what TCPIP software is running on the Alphaserver? If you have HP TCPIP, what patch level? What database software and what are the licensing restraints?

Availability Manager http://h71000.www7.hp.com/openvms/products/availman/ is one tool which may provide information on what your system and running software are doing.

You may also consider forcing a crash dump. HP may or may not review that information since VMS 7.2-2 is no longer offically supported. There are other other sources, some of them to be found here.

Consider moving to a supported version of OpenVMS as well. Staying on an unsupported operating system will generate costs down the line. Some of these may be in terms of performance or additional requirements for troubleshooting.

Andy Bustamante

[Hein van den Heuvel]

>> Can anyone shed any light as to what service or issue was possibly resolved with a reboot.

>> We currently are waiting for approval to install the lastest/last version of DECevent so we don't really have access to all of the logs that might help us.

That reminds me of a joke our friend Steve Lieman mentioned last night at the OpenVMS reunion for situations just like this!

"there is the drunk who staggers from the bar to his car, only to realize that heâ s dropped his keys somewhere. A friend comes across him two hours later, on his hands and knees under the lamppost on the corner. â Why are you still looking here?â the friend asks. â You must have dropped them nearer to the car.â The drunk responds that itâ s too dark to see the keys on the ground near the car, which is why heâ s still looking under the lamppost, where itâ s easier to see. "

In that light... (sic)
What makes you think DECevent might help?

Anyway...

When ODBC access failed, was the system running at all?
Could folks log in?
Could you look aroudn with basic commands like SHOW SYSTEM and SHOW MEMORY?
Did you check the playbooks to see what that output should look like when the system is healthy? Did you check OPERATOR.LOG or ACCOUNTING data around the time the problem occurred?

OpenVMS does not natively provide ODBC.
Folks typically have bought a tool like ConnX or Attinutity or ... to provide that access.

The tool, its logs, and/or its support folks are the best starting points for to better understand, and resolve, this problem.

>> We are very inexperienced with VMS.

Fix that !

>> We are running Openvms 7.2-2.

Fix that !

Best regards,
Hein.

[Hoff]

Regardless of the chosen database platform, a hard reboot is going to leave you subject to and vulnerable to subsequent similar incidents. I too have been forced into the "get the server back online NOW!" production case. Having or saving off some state or some potential clues around the hang/wedge/crash/whatever can help isolate the trigger off-line.

The IT bosses get REALLY mad when it happens AGAIN, in my experience. And it usually will. And the IT boss WILL yell. This why I try to get some diagnostics or dumps or such I can look at after the servers are booted and rolling again. This if I can't troubleshoot when the box is wedged. I have the platform-specific forced-crash sequence available on the console of all the production boxes I manage. Because I know I can be in a hurry when I'm working to get a box back online.

If you're running a production server box as is typical with installed OpenVMS environments, then you're going to have some choices. That can involve learning the platform and its arcana, or around getting help either as support or training (which can help shift some of the risk), or a support and escalation channel, or you're going to be migrating to a platform or an out-source provider that you're more comfortable with supporting.

I deal with databases and networks on on various OpenVMS and non-OpenVMS platforms. As the platform choices go, OpenVMS isn't simple, and it isn't obvious, and there are inherent costs involved in any of these sorts of production cases; that's typical for most (all?) large and complex and enterprise boxes and environments. (I also manage Linux and Unix boxes and databases, and I'm not going to call that area simple, either.) This is life in IT with an installed base of random boxes.

Why look at migrating? IT organizations everywhere are looking for cost savings (and who else isn't?) that usually means narrowing the choices of platforms and infrastructures and vendors and databases and targeting employee skills and training, and other such costs. If you understand the box, you're better able to manage it. And having multiple random boxes adds cost.

The ITRC forums just isn't a good support and escalation channel. You're pointing to ODBC and to DECevent and to an outage; that's a huge territory and rich with potential bugs and errors, and (without the dump file or other such clues as mentioned earlier) there's a good chance this is a serious investment in troubleshooting time here leading up to what may well be a guess. And another outage. Support is also a way to get somebody to quickly log in and look and then troubleshoot and restart the box.

As for your crash here, catch a crashdump when it happens again. And dig around for log files and hardware errors and other such.

And the most recent database hang I chased? That turned out to be a layer 2 switch port that had lost its mind.

Stephen Hoffman
HoffmanLabs LLC

ps: thanks for the complement. Both of you.

[Riverhawk]

Hoff I apologize. I let frustration with systems I don't truly understand get the best of me.

[Riverhawk]

The webserver is running on a Windows NT 4.0 box with Cold Fusion 4.0.

The VMS database is running Oracle RDB Version 7.0-3.

[Hoff]

Ok; so you're probably running an equivalent and old version of the Oracle Rdb ODBC server; see if there are any logs. Most of the Rdb server pieces are decent at generating logs when they go weird, though I've hit cases back in that range that did occasionally lock up.

One of the sneakiest cases I dealt with was due to too-low process quotas; they were marginal enough to mostly work, but not sufficient to actually work.

And there's nothing here that rules out bad or buggy hardware, too. Old disks can and variously do go walkabout, and with minimal or no hardware errors or warnings. Termination can be an issue, too. A SCSI reset often clears those cases. But these can and do tend to re-occur until remediated.

Oracle is almost certainly going to recommend an ECO kit or three for this version of Rdb, and there are many ECOs for problems in OpenVMS Alpha V7.2-2. This isn't as good as upgrading, but it might well be a "mostly free" and minimally invasive option that's available for you.

Frustrating? Sure. Been there. You would not have wanted to hear the comments when I first landed on [expurgated] platform. Parentage was questioned. Again, get yourself some help or some training or an escalation path; these little text boxes and the back-and-forth that's inevitable in these cases just don't work well.

[Brad McCusker]

As others have said - this could be anything.

And don't hold your breath waiting on DECevent - that will only help if it's a limited set of problems.

How long had this been working before you had a problem? Any changes on the web server side (or anything external)? Not trying to lay blame - just trying to get some ideas.

As Hoff and Hein said - VMS and Rdb versions are old. Lots of work in recent years with Rdb's ODBC driver - that would be one thing I'd look at updating, if it's at all possible on your version.

I also like Hoff's idea of quotas raising their head and biting you...

Brad McCusker
www.sciinc.com