This problem was seen after a concurrent recource group was defined with a volume group. When HACMP was started, the RG failed to start on the secondary node, but the primary node was ok. Stopping and restarting the RG, after varying off and back on the VG on the failed node brings the RG online.
This is the solution from an IBM PMR raised.
HACMP is not able to release the reserve, because it lacks the EMC utility to do this. There should never be a reserve set on concurrent VGs, this might be left over from a previous time when the LUN was used for something else or when the VG was varied on non-concurrently.
You need to configure the OEM disk method for powerpath, this will be vital for your non-concurrent VGs in the event of one node failing and the other having to take over its disks.
This is the information I have for powerpath V3. I see you have version 4. I am 90% certain it is still the same, but please check with EMC support:
The emcpowerreset binary takes as options two parameters, adapter name and device name. These two parameters are automatically passed to the binary whenever it is invoked within the HACMP script logic.
This binary can be obtained from the EMC ftp server location
ftp://ftp.emc.com/pub/elab/powerpath/aix/emcpowerreset.tar.
The fix is also available for download from Powerlink using the url
http://powerlink.emc.com/support/index.jhtml
and searching for emcpowerreset. There is 1 binary in the tar file called emcpowerreset.
Installation:
a) Create the directory below to contain the binary obtained from EMC.
Note: Symmetrix customers, running the EMC ODM Kit, should already have this directory. The same directory will be used for both Symmetrix and/or CLARION arrays.
mkdir -p /usr/lpp/Symmetrix/bin
b) Copy the emcpowerreset binary into the /usr/lpp/Symmetrix/bin subdirectory.
c) Make sure the “emcpowerreset” binary is root executable.
d) The steps used to configure the new “emcpowerreset” utility into HACMP will vary depending on the version of HACMP used. Follow the steps in the applicable section below to complete the installation process.
Also, apart from that you need to configure Custom disk method in hacmp to that emcpowerreset gets called instead of cl_fscsilunreset.
For HACMP all included versions (Command Line)
1. Enter the following at the AIX command prompt on each node in the configuration;
/usr/sbin/cluster/utilities/clcustomdisk -c -tdisk/pseudo/power
-Ndisk/pseudo/power -gSCSI3 -hSCSI_TUR
-b/usr/lpp/Symmetrix/bin/emcpowerreset -pfalse -mMKDEV
2. Synchronize the cluster resources. The cluster verification process ensures that the method that you configured exists and is executable on all nodes. The synchronization process ensures that the ODM entries are the same on all nodes, but will not synchronize the methods named in the ODM entries.
The HACMPdisktype ODM object class will get updated with the following attributes:
HACMPdisktype:
PdDvLn = “disk/pseudo/power”
ghostdisks = “SCSI3”
checkres = “SCSI_TUR”
breakres = “/usr/lpp/Symmetrix/bin/emcpowerreset”
parallel = “false”
makedev = “MKDEV”
reserved1 = “”
reserved2 = “”
reserved3 = “”
Please install this on both nodes with cluster services stopped, then restart cluster services.
