Not a valid block device

[Tonatiuh]

Red Hat Enterprise Linux AS 3U6

I have re-installed S.O. but it cannot recognize the existent partitions in a SAN (and other servers that have not been reinstaled still recognize the same partitions.

I receive the following message:

/dev/sdX is not a valid block device.

It also happend with an ext3, local partition in a second internal hard disk when I re-create the partitions at the installation time.

[Steven E. Protter]

Shalom,

Perhaps this is corruption that comes from the problem that caused you to need to reinstall the OS.

Can you tell us why you needed to re-install the OS. Was it due to a problem?

SEP

[Tonatiuh]

A more complete history.

3-Node Oracle Cluster. HP Itanium servers with Red Hat Enterprise Linux AS 3U6. HP SAN with Multipath (2 FC switches).

Everythig was working well and the local (internal) hard disk of node1 failed. The disk was replaced but the S.O. (when it is installed) does not recognize the SAN´s partitions. It see the LUNS (Example: /dev/sdc) but does not see the particions (example: /dev/sdc1).

Eventhough, the other two servers in the cluster (node2 and node3) still see the SAN's particions well. And are still working fine.

Another data: The server re-installed (node1) had two local hard disks (A and B) and only failed one (A). When I re-installed the S.O. (in HD A) I only partitioned the HD A, and once installed I only tryed to mount the HD B (that never failed), but the S.O. showed exactly the same message about the local HD B (Is not a valid block device). I re-partitioned the local HD B and it is working well.

But I do not want to re-partion the SAN too. All my data is there and I cannot be sure that re-partitioning the SAN everythig was going to work well.

[Steven E. Protter]

Shalom again,

I see no reason to touch the SAN, clearly it is not the problem.

If you lost one of the two disks on node1, I see no reason to reinstall the OS if you had raid 1 mirroring set up on the hardware.

The OS re-install should not have effected the ability of the node to talk to the SAN volumes because this is done by World Wide Name.

I speculate that the driver for the HBA card is not correctly installed in the system.

So tell us what kind of card it is and perhaps we can pull a configuration file from the working node. /etc/modprobe.conf comes to mind.

SEP

[Tonatiuh]

Shalom Steven,

I think I have confirmed that the problem is the SAN.

I'll give more info that I have omitted and new info about tests I did after the last message:

1) It was necessary to re-install the S.O. because the local internal HDs of the node1 was not in RAID (no mirror).

2) The SAN had a Hard Disk alarmed since several days.

3) I created a new LUN (with its partition) and the new just installed node1, was able to see and work fine with the new partition.

4) I restarted the node3 (who was seeing well the "old" partitions that node1 was not seeing) and after restart...

a) node3 is not seeing the "old" partitions,

b) but it can see the new partition. This is, exactly the same behavior than node1.

5) I think the SAN's partition table is loaded in memory bay the node, so, if the node2 and node3 had it in memory at the moment it sufered a damage, node2 and node3 does not know about the damage, because they had it in memory well. But the node1 who needed to look for this partition table, was the first that discovered the damage.

6) I will create new LUNs and using the only one node that is still seeing the "old" partition (node2), I will move data to new partitions and use them, while I can replace de disk and a HP technician comes to make a complete diagnostic about the problem.

[Steven E. Protter]

Well, good luck.

I'll look at the data again, but I can't see at this time how you determined the SAN is the problem.

You seem to be operating with more information than I am. I'd see what you can do with the SAN to make the LUNS visible again. If a reset is possible, that might help.

SEP

[Tonatiuh]

A reset is possible because the system is almost offline know (33% of the performance si almost nothing).

But I am afraid that if I reset the SAN, partitions could be completely lost. I need to create the new partitions and copy the data to be sure, before to reset the SAN.

[Steven E. Protter]

Shalom again,

I agree, see about backing up the data before proceeding. Either through the working node or SAN utils.

Consult support for the SAN maker before proceeding. The risk of data loss is high.

SEP

[Tonatiuh]

Right. The risk of data loss is extremly high.

The SAN is an HP VA7410. I am using multipath with qla2300 FC adapters and two Dell FC Switches.