Tuesday, September 20, 2011

Redelivery of faulty JMS messages can bring a server to its knees

This morning CPU 200% and 100MB logs a minute.

A quick examination shows this:

WliSbTransports BEA-381502

Exception in JmsInboundMDB.onMessage: com
.bea.wli.sb.transports.TransportException: Unexpected type of message received: weblogic.jms.common.ObjectMessageImpl

No message redelivery limit was set, so the faulty messages were simply being redeliveder immediately several times a second.

Impossible to delete messages, because they were constantly in "receive" or "receive transaction" state.

Only way out: go to JMSServer, Monitoring, pause Consumption on all queues,
go to the queues, Monitoring, select messages and delete all.

And, of course, set a redelivery limit on the queues.

No comments: