topic Re: LVM 'device busy' error all the time in Operating System - HP-UX

LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 04:48:31 GMT

Hi Guys / Gals ,
Firstly , Im fairly new to HP/UX from a build perspective.
Im hoping you can help me. I have v strange issue. Was setting up a new RP4410 with HP/UX 11i. All was fine until I patched it for Oracle Installs. Now I cant build Volume Groups or anything in LVM. It just keeps saying stuff like 'device busy' and hangs for ages. I've even wiped the system at this stage to start from scratch and still cannot get this to work. Any ideas on such an issue. Its holding me up on a big project , Im really stuck and would appreciate any ideas or suggesstions. I reckon I may have 'overwritten' something when I was patching , could that be possible ?

Re: LVM 'device busy' error all the time

Sivakumar MJ._1 — Fri, 24 Jul 2009 05:13:12 GMT

Hi..

Pls. go through our forum posting related to your issue...

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1152812

Re: LVM 'device busy' error all the time

Duncan Edmonstone — Fri, 24 Jul 2009 06:34:31 GMT

Ok, so what are the disks you are trying to run this command on? Local or SAN? ANd what type? HP EVA, EMC DMX? Something else?

Can you actually show the set of commands you are running and the output...

HTH

Duncan

Re: LVM 'device busy' error all the time

Roopesh Francis_1 — Fri, 24 Jul 2009 07:40:51 GMT

Davey,

can you please post the output of the command which you excuted.
Thanks

Re: LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 12:26:09 GMT

Hi Everyone , thanks for replying to me already on this. Well basically I've tried using the sam menu / creating volume groups and thats where its stuck , see below disks on the system. If I try create a volume group on either of the 'LOGICAL' external disks below it says device busy on both , I've tried everything , vgscans , etc , even thought it might be a hardware problem and tried diff disks etc and no joy. Im mainly an Oracle DBA , if there is anything else I could try to check that may help you help me , let me know what to run and I paste output etc

Group Mbytes Description â
â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â â
â â 0/0/3/0.0.0.0 1 Unused -- 0 DVD-ROM Drive ( ^ â
â â 0/1/1/0.0.0 1 LVM -- 0 HP 73.4GST37345 â
â â 0/1/1/0.1.0 1 LVM vg00 70007 HP 73.4GST37345 â
â â 0/6/1/0/4/0.0.0 1 Unused -- 138920 HP LOGICAL â
â â 0/6/1/0/4/0.0.1 1 Unused -- 138920 HP LOGICAL â

Re: LVM 'device busy' error all the time

Duncan Edmonstone — Fri, 24 Jul 2009 12:29:37 GMT

OK a usefull starting point would be to post the output of the command:

ioscan -kfn

HTH

Duncan

Re: LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 13:07:53 GMT

Hiya doin Duncan , ok here we go

dubhptst: # ioscan -kfn
Class I H/W Path Driver S/W State H/W Type Description
=============================================================================
root 0 root CLAIMED BUS_NEXUS
ioa 0 0 sba CLAIMED BUS_NEXUS System Bus Adap
ter (880)
ba 0 0/0 lba CLAIMED BUS_NEXUS Local PCI-X Bus
Adapter (783)
tty 0 0/0/1/0 asio0 CLAIMED INTERFACE PCI SimpleComm
(103c1290)
/dev/diag/mux0 /dev/mux0 /dev/tty0p0
tty 1 0/0/1/1 asio0 CLAIMED INTERFACE PCI Serial (103
c1048)
/dev/GSPdiag1 /dev/mux1 /dev/tty1p2
/dev/diag/mux1 /dev/tty1p0 /dev/tty1p4
usb 0 0/0/2/0 hcd CLAIMED INTERFACE NEC USB Interfa
ce
usbhub 0 0/0/2/0.1 hub CLAIMED DEVICE USB Root Hub
usb 1 0/0/2/1 hcd CLAIMED INTERFACE NEC USB Interfa
ce
usbhub 1 0/0/2/1.1 hub CLAIMED DEVICE USB Root Hub
/dev/hub
usb 2 0/0/2/2 ehci CLAIMED INTERFACE NEC USB Interfa
ce
sideba 0 0/0/3/0 side_ba CLAIMED INTERFACE CMD IDE control
ler
ext_bus 0 0/0/3/0.0 side CLAIMED INTERFACE IDE Primary Cha
nnel
target 0 0/0/3/0.0.0 tgt CLAIMED DEVICE
disk 0 0/0/3/0.0.0.0 sdisk CLAIMED DEVICE TEAC DV-28E-
N
/dev/dsk/c0t0d0 /dev/rdsk/c0t0d0
target 1 0/0/3/0.0.7 tgt CLAIMED DEVICE
ctl 0 0/0/3/0.0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c0t7d0
ext_bus 1 0/0/3/0.1 side CLAIMED INTERFACE IDE Secondary C
hannel
target 2 0/0/3/0.1.7 tgt CLAIMED DEVICE
ctl 1 0/0/3/0.1.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c1t7d0
ba 1 0/1 lba CLAIMED BUS_NEXUS Local PCI-X Bus
Adapter (783)
ext_bus 2 0/1/1/0 mpt CLAIMED INTERFACE SCSI Ultra320 A
6961-60011
/dev/mpt2
target 3 0/1/1/0.0 tgt CLAIMED DEVICE
disk 1 0/1/1/0.0.0 sdisk CLAIMED DEVICE HP 73.4GST37345
4LC
/dev/dsk/c2t0d0 /dev/rdsk/c2t0d0
target 4 0/1/1/0.1 tgt CLAIMED DEVICE
disk 2 0/1/1/0.1.0 sdisk CLAIMED DEVICE HP 73.4GST37345
4LC
/dev/dsk/c2t1d0 /dev/rdsk/c2t1d0
target 5 0/1/1/0.7 tgt CLAIMED DEVICE
ctl 2 0/1/1/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c2t7d0
ext_bus 3 0/1/1/1 mpt CLAIMED INTERFACE SCSI Ultra320 A
6961-60011
/dev/mpt3
target 6 0/1/1/1.7 tgt CLAIMED DEVICE
ctl 3 0/1/1/1.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c3t7d0
lan 0 0/1/2/0 iether CLAIMED INTERFACE HP AB352-60003
PCI/PCI-X 1000Base-T Dual-port Core
lan 1 0/1/2/1 iether CLAIMED INTERFACE HP AB352-60003
PCI/PCI-X 1000Base-T Dual-port Core
ba 2 0/2 lba CLAIMED BUS_NEXUS Local PCI-X Bus
Adapter (783)
ba 3 0/4 lba CLAIMED BUS_NEXUS Local PCI-X Bus
Adapter (783)
%U username given at login time
%M maximum allowed number of users in this class
%N current number of users in this class

The message will only be displayed once to avoid annoying the
user. Remember that when messages are triggered by an anonymous
ba 4 0/5 lba CLAIMED BUS_NEXUS Local PCI-X Bus
Adapter (783)
ba 5 0/6 lba CLAIMED BUS_NEXUS Local PCI-X Bus
Adapter (783)
ba 6 0/6/1/0 PCItoPCI CLAIMED BUS_NEXUS PCItoPCI Bridge
ext_bus 4 0/6/1/0/4/0 ciss CLAIMED INTERFACE PCI-X SmartArra
y 6402 RAID Controller
/dev/ciss4
target 7 0/6/1/0/4/0.0 tgt CLAIMED DEVICE
disk 3 0/6/1/0/4/0.0.0 sdisk CLAIMED DEVICE HP LOGIC
AL VOLUME
/dev/dsk/c4t0d0 /dev/rdsk/c4t0d0
disk 4 0/6/1/0/4/0.0.1 sdisk CLAIMED DEVICE HP LOGIC
AL VOLUME
/dev/dsk/c4t0d1 /dev/rdsk/c4t0d1
target 8 0/6/1/0/4/0.7 tgt CLAIMED DEVICE
ctl 4 0/6/1/0/4/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c4t7d0
memory 0 8 memory CLAIMED MEMORY Memory
ipmi 0 16 ipmi CLAIMED INTERFACE IPMI Controller
/dev/ipmi
processor 0 128 processor CLAIMED PROCESSOR Processor
processor 1 129 processor CLAIMED PROCESSOR Processor

Re: LVM 'device busy' error all the time

Mel Burslan — Fri, 24 Jul 2009 13:11:01 GMT

Other useful commands to see output from, are:

strings /etc/lvmtab
vgdisplay
swlist -l fileset -a state

they have rather long outputs, so if you can capture your screen to log file and attach that, it would be the easiest.

Re: LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 13:19:08 GMT

Okay , thats no problem , Wow yes theres a good bit of info. Find attached doc with output

Re: LVM 'device busy' error all the time

Duncan Edmonstone — Fri, 24 Jul 2009 13:26:03 GMT

So the disk devices you are trying to use seem to be RAID volumes configured through a SmartARRAY 6402 card. To see the current status of those 2 RAID volumes, what output do you get from:

saconfig /dev/ciss4

Let's also see if the disks are responding at all...

diskinfo /dev/rdsk/c4t0d0
diskinfo /dev/rdsk/c4t0d1

and check there are no LVM headers already on these disks:

pvdisplay /dev/dsk/c4t0d0
pvdisplay /dev/dsk/c4t0d1

HTH

Duncan

Re: LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 13:50:50 GMT

Hi again Duncan , some interesting output from your last requests there . Some errors for sure. See attached

Re: LVM 'device busy' error all the time

Duncan Edmonstone — Fri, 24 Jul 2009 14:02:58 GMT

well something is wrong with those 2 drives:

Let's see the output of:

sautil /dev/ciss4

HTH

Duncan

Re: LVM 'device busy' error all the time

TTr — Fri, 24 Jul 2009 14:11:41 GMT

It appears that the special file /dev/rdsk/c4t0d0 may have been overwritten with a regular file. Post back the output of
ll /dev/rdsk/c4t0d0
ll /dev/dsk/c4t0d0

Re: LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 14:12:21 GMT

Hi Again Duncan , ok ,here we go , see attached output as per requested command. Its all very strange , As I say , it was all working fine until I patched for Oracle , afterwhich when it didnt work after 3 or 4 days of struggling I basically wiped everything bar the O/S , but it looks like something is hanging onto those disks somewhere along the line

Re: LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 14:15:29 GMT

dubhptst: #
dubhptst: # ll /dev/rdsk/c4t0d0
crw-r----- 1 bin sys 188 0x040000 Jul 21 16:40 /dev/rdsk/c4t0d0
dubhptst: # ll /dev/dsk/c4t0d0
brw-r----- 1 bin sys 31 0x040000 Jan 12 2009 /dev/dsk/c4t0d0
dubhptst: #

Re: LVM 'device busy' error all the time

TTr — Fri, 24 Jul 2009 14:33:43 GMT

The disk files are fine. Lookin at all your attachements they all have extraneous text in them which is probably caused by an incorrect terminal setting. Nothing to do with your disk problem.

Have you installed any patches on this server?

The logical drives are marked failed when all 3 physical drives are OK. The only reason I can think of is if you pulled out the disks (and they are marked as having been pulled) and while you were doing that the controller was rebuilding the disks. You probably have to recreate them. Maybe Duncan will see something else. (Duncan, sorry for jumping in)

Re: LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 14:37:12 GMT

HI TTR , Thanks for that indeed , when you say rebuild them , what do you mean exactly ? , (btw , yes it is a possibility that I pulled them during a rebuild all the various hair pulling exercises trying to get this solved myself)

Re: LVM 'device busy' error all the time

Duncan Edmonstone — Fri, 24 Jul 2009 15:15:08 GMT

I'm no expert on these RAID cards, but what you said in your last post concerns me - are you saying you might have pulled out these disks from their enclosure? And possibly not put them back in the same location??

Anyway at this stage, I think you might as well try resetting the RAID card and rescanning, to see if that rests the logical volumes...

sautil /dev/ciss4 reset_ctlr
sautil /dev/ciss4 scan
saconfig /dev/ciss4

HTH

Duncan

Re: LVM 'device busy' error all the time

The Gunners — Fri, 24 Jul 2009 16:25:49 GMT

Hi Duncan , yes I will try that , just to confirm that the issue was already there before I started pulling disks out etc , it was kinda last resort type stuff when I thought that mabey the issue had been a hardware related one.

Re: LVM 'device busy' error all the time

TTr — Fri, 24 Jul 2009 17:37:15 GMT

Perhaps "rebuilding" is not the right word to use when talking about disk arrays. I was refering to having to delete them and recreate them. If you have data on them it will be lost so you may want to play around a little bit more before resorting to deleting the logical disks. I don't know what status your firmware is but since this is an old server you might want to check if any firmware upgrades are needed on the server and the raid controller.