Saturday, January 31, 2015

CA IDM - Jboss Clustering issue

I am getting the below error in Jboss while starting the second node, any suggestions ?

Environment:
Jboss 5.1.0 GA
RedHat Linux 6.4
CA IdentityMinder r12.6 SP2


13:48:21,626 ERROR [ScopedProfileServiceController] Error installing to Create: name=ProfileKey@43886a34[domain=default, server=default, name=farm] state=Configured mode=On Demand requiredState=Installed
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.jboss.system.server.profileservice.repository.AbstractProfileLifeCycleAction.invoke(AbstractProfileLifeCycleAction.java:97)
        at org.jboss.system.server.profileservice.repository.AbstractProfileLifeCycleAction.invoke(AbstractProfileLifeCycleAction.java:77)
        at org.jboss.system.server.profileservice.repository.AbstractProfileLifeCycleAction.install(AbstractProfileLifeCycleAction.java:49)
        at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
        at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
        at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
        at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
        at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
        at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
        at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
        at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
        at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
        at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.java:306)
        at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
        at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
        at org.jboss.Main.boot(Main.java:221)
        at org.jboss.Main$1.run(Main.java:556)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalArgumentException: Null toCopy
        at org.jboss.system.server.profileservice.repository.clustered.metadata.RepositoryContentMetadata.(RepositoryContentMetadata.java:97)
        at org.jboss.profileservice.cluster.repository.DefaultRepositoryClusteringHandler$RpcTarget.initiateSynchronization(DefaultRepositoryClusteringHandler.java:997)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.jgroups.blocks.MethodCall.invoke(MethodCall.java:331)
        at org.jboss.ha.framework.server.ClusterPartition$RpcHandler.handle(ClusterPartition.java:2266)
        at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
        at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
        at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
        at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
        at org.jgroups.JChannel.up(JChannel.java:1339)
        at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:462)
        at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:501)
        at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:144)
        at org.jgroups.protocols.FRAG2.unfragment(FRAG2.java:294)
        at org.jgroups.protocols.FRAG2.up(FRAG2.java:169)
        at org.jgroups.protocols.FC.up(FC.java:473)
        at org.jgroups.protocols.pbcast.GMS.up(GMS.java:824)
        at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
        at org.jgroups.protocols.UNICAST.up(UNICAST.java:299)
        at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:884)
        at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:716)
        at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
        at org.jgroups.protocols.FD.up(FD.java:284)
        at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:309)
        at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
        at org.jgroups.protocols.Discovery.up(Discovery.java:264)
        at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2327)
        at org.jgroups.protocols.TP.passMessageUp(TP.java:1261)
        at org.jgroups.protocols.TP.access$100(TP.java:49)
        at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1838)
        at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1812)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
        ... 1 more
13:48:21,644 ERROR [STDERR] Failed to boot JBoss:
13:48:21,644 ERROR [STDERR] java.lang.IllegalStateException: Incompletely deployed:

*** PROFILES IN ERROR: Name -> Error
Profile: ProfileKey@43886a34[domain=default, server=default, name=farm] in error due to java.lang.reflect.InvocationTargetException

*** PROFILES MISSING DEPENDENCIES: Name -> Dependency{Required State:Actual State}
ProfileKey@466c137c[domain=default, server=default, name=all] is missing following dependencies: ProfileKey@43886a34[domain=default, server=default, name=farm]{Installed:**ERROR**}
13:48:21,644 ERROR [STDERR]     at org.jboss.system.server.profileservice.repository.AbstractProfileService.logErrors(AbstractProfileService.java:567)
13:48:21,644 ERROR [STDERR]     at org.jboss.system.server.profileservice.repository.AbstractProfileService.validate(AbstractProfileService.java:348)
13:48:21,644 ERROR [STDERR]     at org.jboss.system.server.profileservice.repository.AbstractProfileService.validateProfile(AbstractProfileService.java:331)
13:48:21,644 ERROR [STDERR]     at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:272)
13:48:21,644 ERROR [STDERR]     at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
13:48:21,644 ERROR [STDERR]     at org.jboss.Main.boot(Main.java:221)
13:48:21,645 ERROR [STDERR]     at org.jboss.Main$1.run(Main.java:556)
13:48:21,645 ERROR [STDERR]     at java.lang.Thread.run(Thread.java:662)



Tuesday, January 27, 2015

CA IdentityMinder JBoss Clustering Question


IDM Version - CA IdentityMinder12.6 Sp2
App Server - Jboss 5.1 GA
OS - ReHat Linux 6.4

I had 2 IDM's(1 & 2) running in a Cluster mode without any issue. As part of the new application on-boarding, I had to add 2 more IDM's (3 & 4)to the existing cluster. I installed it and configured it succesfully. I am able to start 3 and 4 in standalone mode without any issue.

When i start all 4 of them in cluster, 2 of them are getting started and 2 of them are failing with the deployment error. I tried the below sequences, either way i am able to start only one set of the servers,

- if i start, IDM-1 & 2 in clsuter and 3 & 4 fails with deployment error.
- if i start, IDM-3 & 4 in clsuter and 1 & 2 fails with deployment error.

My JBoss servers are TCP clustering enabled due to the environment limitations. Below is the command which i use it for starting JBoss servers.


/opt/jboss/bin/run.sh -c all -Djboss.default.jgroups.stack=tcp -Djgroups.tcpping.initial_hosts=sso-idm1[7600],sso-idm2[7600],sso-idm3[7600],sso-idm4[7600] &

Do i need to follow any specific sequence to start Jboss servers or is there any diff set of command do i need to follow ?

Any help is very much appreciated.