Unable to start messaging engine

[v***@cgi-ams.com]

This is in WAS 6.0.2.9

I go to Bus-->MyBus-->BusMembers and click the start button, but the messaging engine does not start and fails with the following error "The messaging engine AMS-57757Node01.server1-JMSBus cannot be started."
[6/28/06 14:44:25:103 EDT] 00000026 WebApp A SRVE0180I: [adminconsole] [/ibm/console] [Servlet.LOG]: action: SIBMessagingEngineCollectionAction: Cannot start Messaging Engine AMS-57757Node01.server1-JMSBus:MBeanServer cannot find MBean with ObjectName *:*

What am i missing here?

[v***@cgi-ams.com]

Here is a more exhaustive error messages from the logs --

[6/28/06 16:38:03:491 EDT] 00000019 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1538I: The messaging engine, ME_UUID=E52AB8AEE2F05510, INC_UUID=37dd77d71c5cddd4, is attempting to obtain an exclusive lock on the data store.
[6/28/06 16:38:03:663 EDT] 0000001a SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1545I: A single previous owner was found in the messaging engine's data store, ME_UUID=A15FEDAF0185937C, INC_UUID=232837b710a40edd
[6/28/06 16:38:03:772 EDT] 0000000a WebContainer A SRVE0161I: IBM WebSphere Application Server - Web Container. Copyright IBM Corp. 1998-2004
[6/28/06 16:38:03:772 EDT] 0000001a SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1535E: The messaging engine's unique id does not match that found in the data store. ME_UUID=E52AB8AEE2F05510, ME_UUID(DB)=A15FEDAF0185937C
[6/28/06 16:38:03:772 EDT] 0000000a WebContainer A SRVE0162I: Servlet Specification Level: 2.4
[6/28/06 16:38:03:772 EDT] 00000019 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1546I: The messaging engine, ME_UUID=E52AB8AEE2F05510, INC_UUID=37dd77d71c5cddd4, has lost an existing lock or failed to gain an initial lock on the data store.
[6/28/06 16:38:03:804 EDT] 0000000a WebContainer A SRVE0163I: Supported JSP Specification Level: 2.0
[6/28/06 16:38:03:913 EDT] 00000015 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data source has produced an unexpected exception: com.ibm.ws.sib.msgstore.persistence.DatasourceWrapperStateException: New connections cannot be provided because the persistence layer has been stopped
[6/28/06 16:38:03:929 EDT] 00000015 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0035E: Messaging engine AMS-57757Node01.server1-JMSBus cannot be started; detected error reported during com.ibm.ws.sib.msgstore.impl.MessageStoreImpl start()
[6/28/06 16:38:03:929 EDT] 00000019 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1519E: Messaging engine AMS-57757Node01.server1-JMSBus cannot obtain the lock on its data store, which ensures it has exclusive access to the data.
[6/28/06 16:38:03:976 EDT] 0000000a WebContainer A SRVE0239I: Extension Factory [class com.ibm.ws.webcontainer.extension.ExtHandshakeVHostExtensionFactory] was registered successfully.
[6/28/06 16:38:03:976 EDT] 00000015 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0027I: Messaging engine AMS-57757Node01.server1-JMSBus cannot be restarted because a serious error has been reported.
[6/28/06 16:38:03:976 EDT] 0000000a WebContainer A SRVE0240I: Extension Factory [class com.ibm.ws.webcontainer.extension.ExtHandshakeVHostExtensionFactory] has been associated with patterns [VH:_WS_EH* ].
[6/28/06 16:38:03:991 EDT] 00000015 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0016I: Messaging engine AMS-57757Node01.server1-JMSBus is in state Stopped.
[6/28/06 16:38:04:054 EDT] 0000000a WebGroup A SRVE0169I: Loading Web Module: Default Web Application.
[6/28/06 16:38:04:085 EDT] 00000015 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0016I: Messaging engine AMS-57757Node01.server1-JMSBus is in state Joined.
[6/28/06 16:38:04:085 EDT] 00000015 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0039E: HAManager-initiated activation has failed, messaging engine AMS-57757Node01.server1-JMSBus will be disabled.
[6/28/06 16:38:04:147 EDT] 00000015 HAGroupImpl I HMGR0124I: An activate or deactive request for the local member of group WSAF_SIB_BUS=JMSBus,WSAF_SIB_MESSAGING_ENGINE=AMS-57757Node01.server1-JMSBus,type=WSAF_SIB failed. The reason is < Messaging Engine AMS-57757Node01.server1-JMSBus could not be activated: Refer to earlier error messages > and the data is <null>
[6/28/06 16:38:04:257 EDT] 00000015 HAGroupImpl I HMGR0129I: The local member of group WSAF_SIB_BUS=JMSBus,WSAF_SIB_MESSAGING_ENGINE=AMS-57757Node01.server1-JMSBus,type=WSAF_SIB has been disabled. The reason is disable called internally, the reason is < Messaging Engine AMS-57757Node01.server1-JMSBus could not be activated: Refer to earlier error messages >.

[Paul Ilechko]

***@cgi-ams.com wrote:

This looks interesting:

[6/28/06 16:38:03:913 EDT] 00000015 SibMessage E
[JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS0002E: The messaging engine
encountered an exception while starting. Exception:
com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data
source has produced an unexpected exception:
com.ibm.ws.sib.msgstore.persistence.DatasourceWrapperStateException: New
connections cannot be provided because the persistence layer has been
stopped

- is the database unavailable for some reason ?

[v***@cgi-ams.com]

I use the IBM's internal datastore (cloudscape) for JMS. Should i do anything manually to get cloudscape up and running. I thought it will get started when i start the application server. I still the exception below. Any thoughts are appreciated.




7/5/06 10:53:44:082 EDT] 00000014 InternalGener I DSRA8203I: Database product name : DBMS:db2j
[7/5/06 10:53:44:098 EDT] 00000014 InternalGener I DSRA8204I: Database product version : 5.1.60.38
[7/5/06 10:53:44:098 EDT] 0000000a WebContainer A SRVE0240I: Extension Factory [class com.ibm.ws.jsp.webcontainerext.JSPExtensionFactory] has been associated with patterns [*.jsp *.jspx *.jsw *.jsv ].
[7/5/06 10:53:44:098 EDT] 00000014 InternalGener I DSRA8205I: JDBC driver name : Cloudscape Embedded JDBC Driver
[7/5/06 10:53:44:145 EDT] 0000000a WebContainer A SRVE0239I: Extension Factory [class com.ibm.ws.jsf.extprocessor.JSFExtensionFactory] was registered successfully.
[7/5/06 10:53:44:145 EDT] 0000000a WebContainer A SRVE0240I: Extension Factory [class com.ibm.ws.jsf.extprocessor.JSFExtensionFactory] has been associated with patterns [""].
[7/5/06 10:53:44:161 EDT] 00000014 InternalGener I DSRA8206I: JDBC driver version : 5.1.60.38
[7/5/06 10:53:46:238 EDT] 00000015 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1538I: The messaging engine, ME_UUID=E52AB8AEE2F05510, INC_UUID=25746cbf3f2e2d4f, is attempting to obtain an exclusive lock on the data store.
[7/5/06 10:53:46:348 EDT] 00000016 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1545I: A single previous owner was found in the messaging engine's data store, ME_UUID=A15FEDAF0185937C, INC_UUID=232837b710a40edd
[7/5/06 10:53:46:380 EDT] 0000000a ApplicationMg A WSVR0200I: Starting application: DefaultApplication
[7/5/06 10:53:46:395 EDT] 00000016 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1535E: The messaging engine's unique id does not match that found in the data store. ME_UUID=E52AB8AEE2F05510, ME_UUID(DB)=A15FEDAF0185937C
[7/5/06 10:53:46:395 EDT] 00000015 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1546I: The messaging engine, ME_UUID=E52AB8AEE2F05510, INC_UUID=25746cbf3f2e2d4f, has lost an existing lock or failed to gain an initial lock on the data store.
[7/5/06 10:53:46:694 EDT] 0000000a EJBContainerI I WSVR0207I: Preparing to start EJB jar: Increment.jar
[7/5/06 10:53:46:726 EDT] 0000000a EJBContainerI I WSVR0037I: Starting EJB jar: Increment.jar
[7/5/06 10:53:46:773 EDT] 00000014 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data source has produced an unexpected exception: com.ibm.ws.sib.msgstore.persistence.DatasourceWrapperStateException: New connections cannot be provided because the persistence layer has been stopped
[7/5/06 10:53:46:789 EDT] 00000015 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1519E: Messaging engine AMS-57757Node01.server1-JMSBus cannot obtain the lock on its data store, which ensures it has exclusive access to the data.
[7/5/06 10:53:46:930 EDT] 00000014 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0035E: Messaging engine AMS-57757Node01.server1-JMSBus cannot be started; detected error reported during com.ibm.ws.sib.msgstore.impl.MessageStoreImpl start()
[7/5/06 10:53:46:946 EDT] 00000014 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0027I: Messaging engine AMS-57757Node01.server1-JMSBus cannot be restarted because a serious error has been reported.
[7/5/06 10:53:47:009 EDT] 00000014 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0016I: Messaging engine AMS-57757Node01.server1-JMSBus is in state Stopped.

[Paul Ilechko]

Post by v***@cgi-ams.com
The messaging engine's unique id does not match that found in the

data store.

This looks like you've something odd - I would blow away the bus and
start again.

[v***@cgi-ams.com]

I am using IBM's cloudscape as the peristant store for the JMS messages . I thought that the database instance starts up automatically when i start the application server. Is there anything that i have to do externally(manuall) to satrt the server. Here is the exception that i get.



[7/5/06 9:21:26:178 EDT] 0000000a SchedulerServ I SCHD0001I: The Scheduler Service has started.
[7/5/06 9:21:26:350 EDT] 00000015 InternalGener I DSRA8203I: Database product name : DBMS:db2j
[7/5/06 9:21:26:459 EDT] 00000015 InternalGener I DSRA8204I: Database product version : 5.1.60.38
[7/5/06 9:21:26:459 EDT] 00000015 InternalGener I DSRA8205I: JDBC driver name : Cloudscape Embedded JDBC Driver
[7/5/06 9:21:26:490 EDT] 00000015 InternalGener I DSRA8206I: JDBC driver version : 5.1.60.38
[7/5/06 9:21:27:410 EDT] 0000000a WebContainer A SRVE0239I: Extension Factory [class com.ibm.ws.jsp.webcontainerext.JSPExtensionFactory] was registered successfully.
[7/5/06 9:21:27:410 EDT] 0000000a WebContainer A SRVE0240I: Extension Factory [class com.ibm.ws.jsp.webcontainerext.JSPExtensionFactory] has been associated with patterns [*.jsp *.jspx *.jsw *.jsv ].
[7/5/06 9:21:27:565 EDT] 0000000a WebContainer A SRVE0239I: Extension Factory [class com.ibm.ws.jsf.extprocessor.JSFExtensionFactory] was registered successfully.
[7/5/06 9:21:27:565 EDT] 0000000a WebContainer A SRVE0240I: Extension Factory [class com.ibm.ws.jsf.extprocessor.JSFExtensionFactory] has been associated with patterns [""].
[7/5/06 9:21:28:968 EDT] 00000017 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1538I: The messaging engine, ME_UUID=E52AB8AEE2F05510, INC_UUID=244519e53ed9af79, is attempting to obtain an exclusive lock on the data store.
[7/5/06 9:21:29:093 EDT] 00000018 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1545I: A single previous owner was found in the messaging engine's data store, ME_UUID=A15FEDAF0185937C, INC_UUID=232837b710a40edd
[7/5/06 9:21:29:108 EDT] 00000018 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1535E: The messaging engine's unique id does not match that found in the data store. ME_UUID=E52AB8AEE2F05510, ME_UUID(DB)=A15FEDAF0185937C
[7/5/06 9:21:29:108 EDT] 00000017 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1546I: The messaging engine, ME_UUID=E52AB8AEE2F05510, INC_UUID=244519e53ed9af79, has lost an existing lock or failed to gain an initial lock on the data store.
[7/5/06 9:21:29:358 EDT] 00000015 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data source has produced an unexpected exception: com.ibm.ws.sib.msgstore.persistence.DatasourceWrapperStateException: New connections cannot be provided because the persistence layer has been stopped
[7/5/06 9:21:29:358 EDT] 00000017 SibMessage I [JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1519E: Messaging engine AMS-57757Node01.server1-JMSBus cannot obtain the lock on its data store, which ensures it has exclusive access to the data.
[7/5/06 9:21:29:389 EDT] 00000015 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0035E: Messaging engine AMS-57757Node01.server1-JMSBus cannot be started; detected error reported during com.ibm.ws.sib.msgstore.impl.MessageStoreImpl start()
[7/5/06 9:21:29:389 EDT] 00000015 SibMessage E [JMSBus:AMS-57757Node01.server1-JMSBus] CWSID0027I: Messaging engine AMS-57757Node01.server1-JMSBus cannot be restarted because a serious error has been reported.
[7/5/06 9:21:29:389 EDT] 00000015 SibMessage I [JM

[v***@cgi-ams.com]

I blew away the BUS and re created it and when i tried to start the messaging engine, i get the same exception as before. But what intrested me was -

[7/6/06 8:41:28:192 EDT] 00000021 WebApp A SRVE0180I: [adminconsole] [/ibm/console] [Servlet.LOG]: action: SIBMessagingEngineCollectionAction: Cannot start Messaging Engine AMS-57757Node01.server1-JMSBUS:MBeanServer cannot find MBean with ObjectName *:*
[7/6/06 8:41:28:630 EDT] 0000001f ServletWrappe A SRVE0242I: [adminconsole] [/ibm/console] [/secure/javascriptToSession.jsp]: Initialization successful.

Am i missing a configuration step here while configuring the BUS?

[MartinP]

[6/28/06 16:38:03:772 EDT] 0000001a SibMessage I
[JMSBus:AMS-57757Node01.server1-JMSBus] CWSIS1535E: The messaging engine's
unique id does not match that found in the data store.
ME_UUID=E52AB8AEE2F05510, ME_UUID(DB)=A15FEDAF0185937C

This bit is your problem. What you have almost certainly done is to create
a Bus and messaging engine, use it, and then delete it and recreate the bus
and messaging engine. Each time you create a messaging engine it has a
unique id, and that id is stored in the datastore. On the recreate, even
though the messaging engine name is the same (<node>-<server>.<bus>), the id
is different. The ME checks that the datastore it is accessing contains the
same id to ensure that you do not accidentally configure 2 messaging engines
to point to the same datastore (e.g. tables in a database) and will not
start if the id doesn't match it's own to prevent anything getting screwed
up.

Somel solutions for you:

1: Stop the WAS server, drop the database/tables in the database that the
datastore is in, restart the server.
If you are using the default datastore of Cloudscape/Derby you can do this
by deleting the folder <was_profile_root>\databases\com.ibm.ws.sib\<MEName>
Be aware that you will loose any messages currently stored in the datastore.


2: Change the schema name being used in your datastore (this may not work
for all databases). Find your messaging engine in the Admin pannels, select
it's datastore and edit the schema name. save, synchronize and restart the
sever.

3: Delete your existing bus and create a bus of a different name.


For the default Cloudscape/Derby database this usability problem has been
fixed in later releases (definately 6.0.2+).

MartinP.

[v***@cgi-ams.com]

Ok..i figured out what the problem was! When u create the persistant store for the messages and use the default schema that IBM provides, then u get an exception that an messaging engine already exists in the data store. All i had to do was to change the schema name to something else.

[mwalters]

Post by v***@cgi-ams.com
Ok..i figured out what the problem was! When u create the persistant store for the messages and use the default schema that IBM provides, then u get an exception that an messaging engine already exists in the data store. All i had to do was to change the schema name to something else.

I was curious how you deleted the db and changed the schema. I'm
assuming you're using the cloudscape db? How did you get the tables
recreated?

Thanks,
Michael

[MartinP]

With the default Cloudscape database and most of the other supported
databases, the Messaging Engine creates the tables itself when it need them,
assuming the user id given for accessing the database has sufficient
authorities. The only database that the Messaging Engine cannot ever do
this with is zOS DB2.

With default Cloudscape, the Messaging Engine can also create the database
through the datasource. When specifying a cloudscape datasource you have to
specify a database location (e.g. the place in the filesystem where it will
appear - c:\was\databases\mydatabase) If you add ";create=true" to the end
of this database location then the database will be created if it does not
currently exist (c:\was\databases\mydatabase;create=true).

Changing the Schema name can be done in the definition of the Messagingn
Engine's DataStore - there is a field there for schema name.

Martin.

Post by mwalters

Post by v***@cgi-ams.com
Ok..i figured out what the problem was! When u create the persistant
store for the messages and use the default schema that IBM provides, then
u get an exception that an messaging engine already exists in the data
store. All i had to do was to change the schema name to something else.

I was curious how you deleted the db and changed the schema. I'm
assuming you're using the cloudscape db? How did you get the tables
recreated?
Thanks,
Michael