Changing the verbose level in JRockit

http://edocs.bea.com/jrockit/geninfo/diagnos/verbose.html

put this in your ctrlhandler.act:

verbosity set=memory,memdbg,referents filename=/tmp/memory.log
set_filename filename=/tmp/output.txt
print_object_summary cutoff=0 cutoffpointsto=0
print_class_summary
heap_diagnostics
print_threads

timestamp
jrarecording nativesamples=false latency filename=bcappli1 time=180
stop


or simply do:

jrcmd 23681 verbosity set=memory,memdbg,referents filename=/users/kim/memory.log

you will get this:


23681:
Current logstatus:
jrockit : level=WARN, decorations=103, sanity=NONE
memory (gc) : level=INFO, decorations=103, sanity=NONE
nursery (yc) : level=WARN, decorations=103, sanity=NONE
model : level=WARN, decorations=103, sanity=NONE
devirtual : level=WARN, decorations=103, sanity=NONE
codegen (code) : level=WARN, decorations=103, sanity=NONE
native (jni) : level=WARN, decorations=103, sanity=NONE
thread : level=WARN, decorations=103, sanity=NONE
opt : level=WARN, decorations=103, sanity=NONE
optq (optqueue) : level=WARN, decorations=103, sanity=NONE
callprof : level=WARN, decorations=103, sanity=NONE
cprfall : level=WARN, decorations=103, sanity=NONE
load (lib) : level=WARN, decorations=103, sanity=NONE
ssa : level=WARN, decorations=103, sanity=NONE
constprop : level=WARN, decorations=103, sanity=NONE
inline : level=WARN, decorations=103, sanity=NONE
preprocess : level=WARN, decorations=103, sanity=NONE
jvmpi : level=WARN, decorations=103, sanity=NONE
memdbg : level=INFO, decorations=103, sanity=NONE
jvmtidbg : level=WARN, decorations=103, sanity=NONE
sampling (hotspot) : level=WARN, decorations=103, sanity=NONE
cpuinfo : level=WARN, decorations=103, sanity=NONE
cpuid : level=WARN, decorations=103, sanity=NONE
sym : level=WARN, decorations=103, sanity=NONE
codegc : level=WARN, decorations=103, sanity=NONE
classgc : level=WARN, decorations=103, sanity=NONE
errorsim (esim) : level=WARN, decorations=103, sanity=NONE
memprof (mp) : level=WARN, decorations=103, sanity=NONE
jvmti : level=WARN, decorations=103, sanity=NONE
gcpause : level=WARN, decorations=103, sanity=NONE
gcreport : level=WARN, decorations=103, sanity=NONE
finalizer : level=WARN, decorations=103, sanity=NONE
memleak : level=WARN, decorations=103, sanity=NONE
compaction : level=WARN, decorations=103, sanity=NONE
dpgo : level=WARN, decorations=103, sanity=NONE
license : level=WARN, decorations=103, sanity=NONE
tasks : level=WARN, decorations=103, sanity=NONE
exceptions : level=WARN, decorations=143, sanity=NONE
stackoverflow : level=WARN, decorations=103, sanity=NONE
sysgc (systemgc) : level=WARN, decorations=103, sanity=NONE
locks : level=WARN, decorations=103, sanity=NONE
strcmp : level=WARN, decorations=103, sanity=NONE
bca : level=WARN, decorations=103, sanity=NONE
ir : level=WARN, decorations=103, sanity=NONE
starttime : level=WARN, decorations=103, sanity=NONE
largepages : level=WARN, decorations=103, sanity=NONE
mgmnt (management) : level=WARN, decorations=103, sanity=NONE
jmm : level=WARN, decorations=103, sanity=NONE
perf : level=WARN, decorations=103, sanity=NONE
attach : level=WARN, decorations=103, sanity=NONE
referents (verboserefs) : level=INFO, decorations=103, sanity=NONE
refobj : level=WARN, decorations=103, sanity=NONE
class : level=WARN, decorations=103, sanity=NONE
intrinsics (intrinsic) : level=WARN, decorations=103, sanity=NONE
verify (verifier) : level=WARN, decorations=103, sanity=NONE
jra : level=WARN, decorations=103, sanity=NONE
latency : level=WARN, decorations=103, sanity=NONE
timing : level=WARN, decorations=103, sanity=NONE
instrumentation : level=WARN, decorations=103, sanity=NONE



the content of memory.log will be:

[memdbg ][Mon Aug 31 18:22:06 2009][23681] GC reason: Large object allocation failed (1143032 bytes), cause: Alloc Queue
[memdbg ][Mon Aug 31 18:22:06 2009][23681] Stopping of javathreads took 0.801 ms
[memdbg ][Mon Aug 31 18:22:06 2009][23681] old collection 1852 started
[memdbg ][Mon Aug 31 18:22:06 2009][23681] Alloc Queue size before GC: 1143032, tlas: 0, oldest: 0
[memdbg ][Mon Aug 31 18:22:06 2009][23681] Compacting 8 heap parts at index 8 (type internal) (exceptional false)
[memdbg ][Mon Aug 31 18:22:06 2009][23681] Starting parallel marking phase.
[memdbg ][Mon Aug 31 18:22:08 2009][23681] Pointerset limit hit, compaction aborted.
[memdbg ][Mon Aug 31 18:22:08 2009][23681] Removing 20 permanent work packets from pool, now 9813 packets
[memdbg ][Mon Aug 31 18:22:08 2009][23681] total mark time: 1699.875 ms
[memdbg ][Mon Aug 31 18:22:08 2009][23681] ending marking phase
[refobj ][Mon Aug 31 18:22:08 2009][23681] SoftRef: Reach: 2666 Act: 5365 PrevAct: 1 Null: 59248
[refobj ][Mon Aug 31 18:22:08 2009][23681] WeakRef: Reach: 99251 Act: 4857 PrevAct: 2 Null: 403944
[refobj ][Mon Aug 31 18:22:08 2009][23681] Phantom: Reach: 519 Act: 0 PrevAct: 20166 Null: 0
[refobj ][Mon Aug 31 18:22:08 2009][23681] ObjMoni: Reach: 92 Act: 355 PrevAct: 0 Null: 0
[refobj ][Mon Aug 31 18:22:08 2009][23681] Finaliz: Reach: 104833 Act: 10078 PrevAct: 0 Null: 0
[refobj ][Mon Aug 31 18:22:08 2009][23681] WeakHnd: Reach: 91824 Act: 4284 PrevAct: 0 Null: 0
[refobj ][Mon Aug 31 18:22:08 2009][23681] SoftRef: @Mark: 60839 @Preclean: 0 @FinalMark: 6441
[refobj ][Mon Aug 31 18:22:08 2009][23681] WeakRef: @Mark: 503085 @Preclean: 0 @FinalMark: 4969
[refobj ][Mon Aug 31 18:22:08 2009][23681] Phantom: @Mark: 20177 @Preclean: 0 @FinalMark: 508
[refobj ][Mon Aug 31 18:22:08 2009][23681] ObjMoni: @Mark: 91 @Preclean: 0 @FinalMark: 356
[refobj ][Mon Aug 31 18:22:08 2009][23681] Finaliz: @Mark: 0 @Preclean: 0 @FinalMark: 114911
[refobj ][Mon Aug 31 18:22:08 2009][23681] WeakHnd: @Mark: 0 @Preclean: 0 @FinalMark: 96108
[refobj ][Mon Aug 31 18:22:08 2009][23681] SoftRef: SoftAliveOnly: 0 SoftAliveAndReach: 0
[refobj ][Mon Aug 31 18:22:08 2009][23681] Printing Reference Objects statistics per class:
----- Reference Objects statistics separated per class -----
Total Reach Act PrevAct Null
----- ----- --- ------- ----
Soft References:
67280 2666 5365 1 59248 Total for all Soft References

java/lang/ref/SoftReference =>
66850 2288 5314 0 59248 Total
59248 0 0 0 59248 => null
2177 1886 291 0 0 => java/lang/reflect/Method
1116 0 1116 0 0 => com/acme/info/advertisement/commons/AdLanguageInfos
905 5 900 0 0 => weblogic/utils/io/Chunk
716 270 446 0 0 => java/io/ObjectStreamClass
437 0 437 0 0 => com/acme/info/advertisement/commons/AdMktClassifCodeId
309 0 309 0 0 => [Ljava/lang/reflect/Method;
200 0 200 0 0 => com/acme/info/advertisement/commons/AdvertisementExtr
179 0 179 0 0 => javelin/jsp/el/ELNode$Name
169 71 98 0 0 => java/io/ObjectStreamClass$FieldReflector
160 0 160 0 0 => [Ljava/lang/reflect/Field;
157 0 157 0 0 => javelin/jsp/el/ELNode$RefExpression
109 0 109 0 0 => javelin/jsp/el/ELNode$RelationalExpression
98 0 98 0 0 => javelin/jsp/el/ELNode$LogicalExpression
90 0 90 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberFunctionality
88 0 88 0 0 => com/acme/info/advertisement/commons/AdContent
72 0 72 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberProduct
63 0 63 0 0 => [Ljava/lang/reflect/Constructor;
39 2 37 0 0 => com/acme/corp/subscribingMgt/commons/Subscriber
39 2 37 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberCreator
32 0 32 0 0 => com/acme/info/marketingInformation/commons/plc/EstabMktInfo
30 0 30 0 0 => com/acme/info/marketingInformation/commons/plc/ActivityTextPLC
28 0 28 0 0 => com/acme/corp/subscribingMgt/commons/SubCtrtAdm
28 0 28 0 0 => com/acme/corp/subscribingMgt/commons/SubCtrtProfile
28 0 28 0 0 => com/acme/corp/subscribingMgt/commons/SubCtrtSub
26 0 26 0 0 => java/lang/StringCoding$CharsetSD
21 0 21 0 0 => com/acme/corp/contract/commons/ElementaryProduct
21 0 21 0 0 => java/lang/StringCoding$CharsetSE
20 0 20 0 0 => com/acme/info/marketingInformation/commons/plc/ExportCountryPLC
19 0 19 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberContract
14 3 11 0 0 => com/acme/info/tender/commons/plc/SupplierPLC
14 3 11 0 0 => com/acme/info/tender/commons/plc/SubscriberPLC
13 0 13 0 0 => com/acme/info/advertisement/commons/AdvertisementDetailLocalService
11 2 9 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberContact
10 2 8 0 0 => com/acme/corp/userParameters/commons/ScopedValue
10 0 10 0 0 => com/acme/corp/contract/commons/context/ApplicationContext
9 0 9 0 0 => javelin/jsp/el/ELNode$BinaryExpression
9 0 9 0 0 => com/acme/info/tender/commons/plc/BidPLC
9 0 9 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberContractExtract
8 0 8 0 0 => com/acme/corp/contract/commons/condition/invoice/ConditionInvoiceAmountScaleDetail
8 8 0 0 0 => com/acme/corp/broadcast/commons/Tag
6 0 6 0 0 => com/acme/corp/thirdParty/commons/ThirdPartyContact
6 0 6 0 0 => javelin/jsp/el/ELNode$UnaryExpression
6 1 5 0 0 => com/acme/info/tender/commons/plc/KompassProductLanguagePLC
4 4 0 0 0 => com/acme/corp/request/commons/Status
4 4 0 0 0 => [[Ljava/lang/String;
4 0 4 0 0 => com/acme/info/tender/commons/plc/AuctionPLC
3 3 0 0 0 => org/apache/xmlbeans/impl/schema/SchemaTypeLoaderImpl
3 3 0 0 0 => com/bea/xml/QNameCache
3 0 3 0 0 => javelin/jsp/el/ELNode$Function
3 0 3 0 0 => com/acme/info/tender/commons/plc/AuctionProductLanguagePLC
3 1 2 0 0 => com/acme/info/tender/commons/plc/EnquiryProductLanguagePLC
3 1 2 0 0 => com/acme/info/tender/commons/plc/AttachedFilePLC
3 1 2 0 0 => com/acme/info/tender/commons/plc/EnquiryPLC
3 0 3 0 0 => com/acme/corp/address/commons/AddressLang
3 0 3 0 0 => com/acme/corp/address/commons/Address
2 2 0 0 0 => com/acme/corp/request/commons/RequestSerialized
2 0 2 0 0 => weblogic/cache/CacheValue
2 0 2 0 0 => com/acme/info/marketingInformation/commons/plc/ExportAreaPLC
2 0 2 0 0 => com/acme/info/marketingInformation/commons/plc/PhoneCommunicationPLC
2 0 2 0 0 => com/acme/corp/contract/commons/ReferenceDocument
2 0 2 0 0 => com/acme/corp/contract/commons/condition/invoice/ConditionInvoiceUnitCountScaleDetail
2 0 2 0 0 => com/acme/corp/contract/commons/ContractMainData
2 0 2 0 0 => com/acme/corp/contract/commons/Contract
2 0 2 0 0 => com/acme/info/tender/commons/plc/ProposalPLC
2 0 2 0 0 => com/acme/corp/address/commons/Contact
2 0 2 0 0 => com/acme/info/marketingInformation/commons/plc/TradenamePLC
2 2 0 0 0 => com/acme/corp/broadcast/commons/UserTarget
2 2 0 0 0 => com/acme/corp/broadcast/commons/TemplateNotificationRecord
2 2 0 0 0 => com/acme/corp/broadcast/commons/MediaFile
2 2 0 0 0 => com/acme/corp/broadcast/commons/MediaType
1 1 0 0 0 => java/lang/Class
1 1 0 0 0 => weblogic/utils/classloaders/ChangeAwareClassLoader
1 1 0 0 0 => org/apache/xmlbeans/QNameCache
1 1 0 0 0 => com/bea/xbean/schema/SchemaTypeLoaderImpl
1 0 1 0 0 => com/acme/corp/thirdParty/commons/ThirdPartyFull
1 0 1 0 0 => com/acme/corp/thirdParty/commons/ThirdPartyAddress
1 0 1 0 0 => com/sun/jndi/ldap/SimpleClientId
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/WebSitePLC
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/MailingPLC
1 0 1 0 0 => com/acme/corp/contract/commons/role/RoleContractual
1 0 1 0 0 => weblogic/management/configuration/EmbeddedLDAPMBeanImpl$Helper
1 0 1 0 0 => sun/text/resources/DateFormatZoneData_en_GB
1 0 1 0 0 => sun/text/resources/LocaleElements_en_GB
1 0 1 0 0 => sun/text/resources/DateFormatZoneData_en
1 0 1 0 0 => sun/text/resources/LocaleElements_en
1 0 1 0 0 => com/acme/corp/framework/security/providers/authentification/OIDAuthenticatorMBeanImpl$He lper
1 0 1 0 0 => [Ljava/lang/String;
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/EstabMktProfilePLC
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/EmailPLC
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/ProfileTagsPLC
1 1 0 0 0 => com/acme/corp/broadcast/commons/Component
1 1 0 0 0 => com/acme/corp/broadcast/commons/Notification
1 0 1 0 0 => weblogic/j2ee/descriptor/wl/WeblogicWebAppBeanImpl$Helper
1 0 1 0 0 => com/acme/corp/thirdParty/commons/RoleReference

sun/misc/SoftCache$ValueCell =>
426 375 51 0 0 Total
282 252 30 0 0 => java/util/PropertyResourceBundle
108 108 0 0 0 => java/lang/Object
6 0 6 0 0 => com/acme/info/marketingView/common/translation/Messages_en
3 3 0 0 0 => weblogic/deployment/descriptors/DDValidationBundle
3 0 3 0 0 => com/acme/info/marketingView/common/translation/Messages_fr
3 0 3 0 0 => oracle/toplink/internal/localization/i18n/TraceLocalizationResource
3 3 0 0 0 => sun/awt/resources/awt
2 2 0 0 0 => sun/security/util/AuthResources_fr
2 2 0 0 0 => sun/security/util/Resources_fr
2 2 0 0 0 => com/sun/org/apache/xalan/internal/xsltc/compiler/util/ErrorMessages_fr
2 0 2 0 0 => java/text/RuleBasedCollator
1 1 0 0 0 => com/sun/org/apache/xalan/internal/xsltc/compiler/util/ErrorMessages
1 1 0 0 0 => sun/security/util/Resources
1 1 0 0 0 => sun/security/util/AuthResources
1 0 1 0 0 => sun/text/resources/DateFormatZoneData_en_GB
1 0 1 0 0 => sun/text/resources/DateFormatZoneData_en
1 0 1 0 0 => sun/text/resources/LocaleElements_en_GB
1 0 1 0 0 => sun/text/resources/LocaleElements_en
1 0 1 0 0 => sun/text/resources/LocaleElements
1 0 1 0 0 => com/acme/info/marketingView/common/translation/Messages
1 0 1 0 0 => sun/text/resources/DateFormatZoneData

sun/security/util/MemoryCache$SoftCacheEntry =>
4 3 0 1 0 Total
3 3 0 0 0 => sun/security/x509/X509CertImpl
1 0 0 1 0 => null

Weak References:
508054 99251 4857 2 403944 Total for all Weak References

java/lang/ref/WeakReference =>
493838 87169 3367 0 403302 Total
403302 0 0 0 403302 => null
40244 40244 0 0 0 => weblogic/servlet/internal/MuxableSocketHTTP
20122 20122 0 0 0 => weblogic/servlet/internal/ServletResponseImpl
18787 18787 0 0 0 => java/lang/Class
3707 3707 0 0 0 => java/lang/reflect/Method
3546 3546 0 0 0 => com/bea/netuix/nf/factories/MetaUIControl
2037 0 2037 0 0 => com/acme/info/advertisement/commons/AdLanguageInfos
437 0 437 0 0 => com/acme/info/advertisement/commons/AdMktClassifCodeId
355 355 0 0 0 => weblogic/rmi/internal/ClientRuntimeDescriptor
200 0 200 0 0 => com/acme/info/advertisement/commons/AdvertisementExtr
181 181 0 0 0 => weblogic/utils/classloaders/ChangeAwareClassLoader
99 99 0 0 0 => weblogic/utils/classloaders/GenericClassLoader
90 0 90 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberFunctionality
88 0 88 0 0 => com/acme/info/advertisement/commons/AdContent
72 0 72 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberProduct
39 2 37 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberCreator
39 2 37 0 0 => com/acme/corp/subscribingMgt/commons/Subscriber
33 33 0 0 0 => sun/misc/Launcher$AppClassLoader
32 0 32 0 0 => com/acme/info/marketingInformation/commons/plc/EstabMktInfo
30 0 30 0 0 => com/acme/info/marketingInformation/commons/plc/ActivityTextPLC
28 0 28 0 0 => com/acme/corp/subscribingMgt/commons/SubCtrtAdm
28 0 28 0 0 => com/acme/corp/subscribingMgt/commons/SubCtrtSub
28 0 28 0 0 => com/acme/corp/subscribingMgt/commons/SubCtrtProfile
23 23 0 0 0 => java/util/logging/Logger
21 0 21 0 0 => com/acme/corp/contract/commons/ElementaryProduct
20 0 20 0 0 => com/acme/info/marketingInformation/commons/plc/ExportCountryPLC
19 0 19 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberContract
14 0 14 0 0 => weblogic/jndi/factories/java/javaURLContextFactory
14 3 11 0 0 => com/acme/info/tender/commons/plc/SupplierPLC
14 3 11 0 0 => com/acme/info/tender/commons/plc/SubscriberPLC
13 0 13 0 0 => com/acme/info/advertisement/commons/AdvertisementDetailLocalService
11 2 9 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberContact
10 2 8 0 0 => com/acme/corp/userParameters/commons/ScopedValue
10 0 10 0 0 => com/acme/corp/contract/commons/context/ApplicationContext
9 0 9 0 0 => com/acme/info/tender/commons/plc/BidPLC
9 0 9 0 0 => com/acme/corp/subscribingMgt/commons/SubscriberContractExtract
8 0 8 0 0 => com/acme/corp/contract/commons/condition/invoice/ConditionInvoiceAmountScaleDetail
8 8 0 0 0 => com/acme/corp/broadcast/commons/Tag
7 7 0 0 0 => java/net/URLClassLoader
6 0 6 0 0 => com/acme/corp/thirdParty/commons/ThirdPartyContact
6 1 5 0 0 => com/acme/info/tender/commons/plc/KompassProductLanguagePLC
5 5 0 0 0 => weblogic/diagnostics/archive/filestore/JMSLogFileDataArchive
4 4 0 0 0 => com/acme/corp/request/commons/Status
4 0 4 0 0 => java/security/ProtectionDomain
4 0 4 0 0 => com/acme/info/tender/commons/plc/AuctionPLC
3 1 2 0 0 => com/acme/info/tender/commons/plc/AttachedFilePLC
3 1 2 0 0 => com/acme/info/tender/commons/plc/EnquiryProductLanguagePLC
3 0 3 0 0 => com/acme/info/tender/commons/plc/AuctionProductLanguagePLC
3 1 2 0 0 => com/acme/info/tender/commons/plc/EnquiryPLC
3 0 3 0 0 => com/acme/corp/address/commons/AddressLang
3 0 3 0 0 => com/acme/corp/address/commons/Address
2 2 0 0 0 => weblogic/rjvm/t3/MuxableSocketT3
2 2 0 0 0 => com/acme/corp/request/commons/RequestSerialized
2 0 2 0 0 => com/acme/info/marketingInformation/commons/plc/ExportAreaPLC
2 0 2 0 0 => com/acme/corp/contract/commons/ReferenceDocument
2 0 2 0 0 => com/acme/corp/contract/commons/condition/invoice/ConditionInvoiceUnitCountScaleDetail
2 0 2 0 0 => com/acme/info/tender/commons/plc/ProposalPLC
2 0 2 0 0 => com/acme/corp/contract/commons/ContractMainData
2 0 2 0 0 => com/acme/corp/contract/commons/Contract
2 2 0 0 0 => weblogic/ldap/MuxableSocketLDAP
2 2 0 0 0 => weblogic/iiop/MuxableSocketIIOP
2 2 0 0 0 => javax/management/remote/rmi/RMIConnectionImpl
2 0 2 0 0 => com/acme/corp/address/commons/Contact
2 0 2 0 0 => com/acme/info/marketingInformation/commons/plc/TradenamePLC
2 0 2 0 0 => com/acme/info/marketingInformation/commons/plc/PhoneCommunicationPLC
2 2 0 0 0 => com/acme/corp/broadcast/commons/MediaType
2 2 0 0 0 => com/acme/corp/broadcast/commons/MediaFile
2 2 0 0 0 => com/acme/corp/broadcast/commons/TemplateNotificationRecord
2 2 0 0 0 => com/acme/corp/broadcast/commons/UserTarget
1 1 0 0 0 => weblogic/rjvm/t3/MuxableSocketT3$T3MsgAbbrevJVMConnection
1 1 0 0 0 => weblogic/diagnostics/archive/filestore/AccessLogFileDataArchive
1 1 0 0 0 => weblogic/diagnostics/archive/filestore/ServerLogFileDataArchive
1 1 0 0 0 => com/bea/xbean/schema/SchemaTypeLoaderImpl
1 1 0 0 0 => com/bea/wlw/netui/pageflow/internal/WebLogicURLTemplateFactory
1 1 0 0 0 => com/bea/wlw/netui/pageflow/internal/WebLogicURLTemplateFactory
1 1 0 0 0 => com/bea/wlw/netui/pageflow/internal/WebLogicURLTemplateFactory
1 1 0 0 0 => com/bea/wlw/netui/pageflow/internal/WebLogicURLTemplateFactory
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/ProfileTagsPLC
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/EstabMktProfilePLC
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/EmailPLC
1 0 1 0 0 => com/acme/corp/thirdParty/commons/RoleReference
1 0 1 0 0 => com/acme/corp/thirdParty/commons/ThirdPartyFull
1 0 1 0 0 => com/acme/corp/thirdParty/commons/ThirdPartyAddress
1 1 0 0 0 => javelin/Javelin
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/MailingPLC
1 0 1 0 0 => com/acme/info/marketingInformation/commons/plc/WebSitePLC
1 0 1 0 0 => com/acme/corp/contract/commons/role/RoleContractual
1 1 0 0 0 => weblogic/protocol/AsyncMessageSenderImpl
1 1 0 0 0 => com/acme/corp/broadcast/commons/Component
1 1 0 0 0 => com/acme/corp/broadcast/commons/Notification

java/util/WeakHashMap$Entry =>
11105 9636 1467 2 0 Total
3707 3707 0 0 0 => java/lang/reflect/Method
3546 3546 0 0 0 => com/bea/netuix/nf/factories/MetaUIControl
1457 0 1457 0 0 => java/util/logging/LogRecord
1217 1217 0 0 0 => java/lang/Class
547 547 0 0 0 => weblogic/utils/classloaders/GenericClassLoader
355 355 0 0 0 => weblogic/rmi/internal/ClientRuntimeDescriptor
165 165 0 0 0 => weblogic/management/jmx/modelmbean/ModelMBeanInfoWrapper
83 83 0 0 0 => weblogic/utils/classloaders/ChangeAwareClassLoader
6 0 6 0 0 => org/apache/commons/beanutils/MethodUtils$MethodDescriptor
4 4 0 0 0 => java/security/ProtectionDomain
4 4 0 0 0 => java/net/URLClassLoader
3 0 3 0 0 => weblogic/i18ntools/L10nLookup$LocalizerKey
2 0 0 2 0 => null
2 2 0 0 0 => sun/misc/Launcher$AppClassLoader
2 2 0 0 0 => java/lang/String
1 1 0 0 0 => weblogic/rjvm/ConnectionManagerServer
1 1 0 0 0 => java/lang/Object
1 1 0 0 0 => weblogic/diagnostics/debug/ServerDebugProvider
1 1 0 0 0 => weblogic/servlet/jsp/TagFileClassLoader
1 0 1 0 0 => com/sun/jndi/ldap/SimpleClientId

java/lang/ThreadLocal$ThreadLocalMap$Entry =>
798 796 0 0 2 Total
277 277 0 0 0 => java/lang/ThreadLocal
104 104 0 0 0 => java/lang/InheritableThreadLocal
103 103 0 0 0 => weblogic/kernel/ResettableThreadLocal$1
97 97 0 0 0 => weblogic/kernel/FinalThreadLocal$1
50 50 0 0 0 => sun/misc/FloatingDecimal$1
50 50 0 0 0 => org/apache/beehive/netui/pageflow/internal/DeferredSessionStorageHandler$1
49 49 0 0 0 => jrockit/vm/CharBufferThreadLocal
49 49 0 0 0 => weblogic/servlet/internal/ResponseHeaders$DateHeaderFormat$1
4 4 0 0 0 => com/bea/xml/XmlBeans$1
4 4 0 0 0 => com/bea/xbean/store/CharUtil$1
3 3 0 0 0 => weblogic/descriptor/internal/DescriptorImpl$1
2 0 0 0 2 => null
2 2 0 0 0 => com/bea/xbean/store/Locale$1
1 1 0 0 0 => org/apache/xmlbeans/impl/store/Locale$1
1 1 0 0 0 => org/apache/xmlbeans/impl/schema/SchemaTypeLoaderImpl$SchemaTypeLoaderCache$1
1 1 0 0 0 => org/apache/xmlbeans/XmlBeans$1
1 1 0 0 0 => org/apache/xmlbeans/impl/store/CharUtil$1

java/io/ObjectStreamClass$WeakClassKey =>
732 732 0 0 0 Total
732 732 0 0 0 => java/lang/Class

oracle/xml/util/QxNameHash$WeakValue =>
687 57 0 0 630 Total
630 0 0 0 630 => null
57 57 0 0 0 => oracle/xml/util/QxName

java/util/ResourceBundle$LoaderReference =>
424 424 0 0 0 Total
162 162 0 0 0 => sun/misc/Launcher$AppClassLoader
145 145 0 0 0 => weblogic/utils/classloaders/ChangeAwareClassLoader
114 114 0 0 0 => weblogic/utils/classloaders/GenericClassLoader
3 3 0 0 0 => java/net/URLClassLoader

java/io/ObjectStreamClass$FieldReflectorKey =>
327 326 0 0 1 Total
326 326 0 0 0 => java/lang/Class
1 0 0 0 1 => null

weblogic/utils/collections/WeakConcurrentHashMap$Entry =>
108 108 0 0 0 Total
69 69 0 0 0 => weblogic/timers/internal/TimerImpl
36 36 0 0 0 => java/lang/Class
1 1 0 0 0 => weblogic/servlet/internal/MuxableSocketHTTP
1 1 0 0 0 => weblogic/ldap/MuxableSocketLDAP
1 1 0 0 0 => weblogic/iiop/MuxableSocketIIOP

weblogic/corba/utils/ClassInfo$ClassInfoReference =>
31 0 23 0 8 Total
23 0 23 0 0 => weblogic/corba/utils/ClassInfo
8 0 0 0 8 => null

weblogic/corba/utils/RemoteInfo$RemoteInfoReference =>
2 2 0 0 0 Total
2 2 0 0 0 => weblogic/corba/utils/RemoteInfo

workshop/util/WeakHashListSet$Entry =>
1 0 0 0 1 Total
1 0 0 0 1 => null

com/sun/jndi/ldap/pool/ConnectionsWeakRef =>
1 1 0 0 0 Total
1 1 0 0 0 => com/sun/jndi/ldap/pool/ConnectionsRef

Phantom References:
20685 519 0 20166 0 Total for all Phantom References

com/bea/xbean/store/Locale$Ref =>
20166 0 0 20166 0 Total
20166 0 0 20166 0 => com/bea/xbean/store/Cursor

sun/misc/Cleaner =>
508 508 0 0 0 Total
508 508 0 0 0 => java/nio/DirectByteBuffer

weblogic/rmi/internal/PhantomRef =>
11 11 0 0 0 Total
11 11 0 0 0 => weblogic/rmi/internal/BasicServerRef

ObjectMonitors:
447 92 355 0 0 Total for all ObjectMonitors
165 15 150 0 0 => netscape/ldap/LDAPSearchListener
102 2 100 0 0 => netscape/ldap/LDAPResponseListener
63 0 63 0 0 => com/sun/jndi/ldap/LdapRequest
52 52 0 0 0 => weblogic/work/ExecuteThread
22 0 22 0 0 => weblogic/common/CompletionRequest
11 1 10 0 0 => weblogic/transaction/internal/ResourceCheckpoint
11 1 10 0 0 => weblogic/transaction/internal/ServerCheckpoint
5 5 0 0 0 => weblogic/kernel/ServerExecuteThread
5 5 0 0 0 => java/util/TaskQueue
2 2 0 0 0 => com/bea/netuix/application/manager/util/CacheKey
1 1 0 0 0 => java/lang/ref/ReferenceQueue$Lock
1 1 0 0 0 => weblogic/timers/internal/TimerThread
1 1 0 0 0 => weblogic/t3/srvr/T3Srvr
1 1 0 0 0 => weblogic/server/channels/DynamicListenThreadManager
1 1 0 0 0 => netscape/ldap/TTLTimer
1 1 0 0 0 => java/lang/String
1 1 0 0 0 => com/octetstring/vde/backend/standard/TransactionProcessor
1 1 0 0 0 => weblogic/time/common/internal/TimeTable
1 1 0 0 0 => weblogic/t3/srvr/SubsystemRequest

Finalizers:
114911 104833 10078 0 0 Total for all Finalizers
43245 40299 2946 0 0 => java/net/SocksSocketImpl
21945 20171 1774 0 0 => java/net/SocketOutputStream
21719 20170 1549 0 0 => java/net/SocketInputStream
21521 20124 1397 0 0 => weblogic/socket/WeblogicSocket
1844 1844 0 0 0 => weblogic/utils/classloaders/ClasspathClassFinder$ZipFileWrapper
1615 235 1380 0 0 => java/io/FileInputStream
744 739 5 0 0 => java/util/zip/ZipFile
583 577 6 0 0 => java/util/zip/Inflater
321 320 1 0 0 => java/util/jar/JarFile
232 0 232 0 0 => com/acme/corp/framework/remote/RmiServiceLocator
197 36 161 0 0 => oracle/toplink/internal/sessions/FinalizedClientSession
162 12 150 0 0 => com/rsa/jsafe/JA_DES
162 12 150 0 0 => com/rsa/jsafe/JA_DESKey
145 69 76 0 0 => oracle/toplink/internal/sessions/IsolatedClientSession
63 0 63 0 0 => com/sun/jndi/ldap/LdapCtx
54 4 50 0 0 => com/rsa/jsafe/JG_BlockCipher
54 4 50 0 0 => com/rsa/jsafe/JA_3DES_EDE
54 4 50 0 0 => com/rsa/jsafe/JA_CBC
36 36 0 0 0 => sun/net/www/protocol/jar/URLJarFile
36 0 36 0 0 => com/sun/jndi/ldap/LdapSearchEnumeration
32 32 0 0 0 => java/io/FileOutputStream
23 23 0 0 0 => weblogic/rmi/internal/LeasedRemoteRef
21 21 0 0 0 => oracle/toplink/threetier/ServerSession
19 19 0 0 0 => org/apache/log4j/RollingFileAppender
17 17 0 0 0 => netscape/ldap/LDAPConnection
11 11 0 0 0 => weblogic/ldap/EmbeddedLDAPConnection
10 10 0 0 0 => weblogic/jms/frontend/FEConnection
9 9 0 0 0 => java/lang/ClassLoader$NativeLibrary
5 5 0 0 0 => weblogic/jms/backend/BackEnd
5 5 0 0 0 => jrockit/ext/epoll/EPollEventBuffer
4 4 0 0 0 => java/util/Timer$1
4 2 2 0 0 => com/sun/jndi/ldap/LdapClient
4 4 0 0 0 => com/rsa/jsafe/JA_PKCS5Padding
3 3 0 0 0 => weblogic/security/utils/Pool
3 3 0 0 0 => com/octetstring/vde/backend/standard/RAFPool
2 2 0 0 0 => java/net/PlainDatagramSocketImpl
1 1 0 0 0 => weblogic/management/mbeans/custom/Domain
1 1 0 0 0 => bea/jmapi/JVMImpl
1 1 0 0 0 => org/apache/log4j/ConsoleAppender
1 1 0 0 0 => com/octetstring/vde/backend/standard/TransactionProcessor
1 1 0 0 0 => com/octetstring/vde/util/Logger
1 1 0 0 0 => com/octetstring/vde/backend/standard/TransactionLog
1 1 0 0 0 => jrockit/ext/epoll/EPoll

Weak Handles:
96108 91824 4284 0 0 Total for all Weak Handles
37650 35007 2643 0 0 => java/lang/String
18124 18124 0 0 0 => sun/misc/Launcher$AppClassLoader
12186 12186 0 0 0 => weblogic/utils/classloaders/GenericClassLoader
9925 8286 1639 0 0 => sun/reflect/DelegatingClassLoader
8142 8142 0 0 0 => weblogic/utils/classloaders/ChangeAwareClassLoader
7508 7508 0 0 0 => weblogic/servlet/jsp/JspClassLoader
2537 2537 0 0 0 => java/net/URLClassLoader
15 15 0 0 0 => sun/misc/Launcher$ExtClassLoader
12 12 0 0 0 => javax/management/remote/rmi/NoCallStackClassLoader
4 4 0 0 0 => java/lang/Class
3 1 2 0 0 => java/lang/Thread
1 1 0 0 0 => bea/jmapi/JVMImpl
1 1 0 0 0 => com/octetstring/vde/LDAPServer
Soft reachable referents not used for at least 1184.744 s cleared.
0 SoftReferences was soft alive but not reachable (when found by the GC),
0 was both soft alive and reachable, and 67280 was not soft alive.
----- End of Reference Objects statistics -----
[memdbg ][Mon Aug 31 18:22:08 2009][23681] Enough TLA allocoation since last OC. Turning on nursery.
[memory ][Mon Aug 31 18:22:08 2009][23681] Changing GC strategy to Generational Parallel Mark & Sweep
[memdbg ][Mon Aug 31 18:22:08 2009][23681] starting parallel sweeping phase
[memdbg ][Mon Aug 31 18:22:08 2009][23681] total sweep time: 13.993 ms
[memdbg ][Mon Aug 31 18:22:08 2009][23681] ending sweeping phase
[memdbg ][Mon Aug 31 18:22:08 2009][23681] Alloc Queue size after GC: 0, tlas: 0, oldest: 0
[memdbg ][Mon Aug 31 18:22:08 2009][23681] Nursery size after OC: 1241723384
[memory ][Mon Aug 31 18:22:08 2009][23681] 257477.036-257478.762: GC 4193701K->1759451K (4194304K), 1726.335 ms
[memdbg ][Mon Aug 31 18:22:08 2009][23681] Page faults before GC: 11, page faults after GC: 11, pages in heap: 1048576
[memdbg ][Mon Aug 31 18:22:08 2009][23681] Restarting of javathreads took 0.123 ms

Panic Action, Overload, OutOfMemory in Weblogic Server

http://j-francois.blogspot.com/2009/03/workmanager.html

in the server configuration, the Overload tab helps you define
policies to handle OutOfMemory and Stuck Threads.


I am trying to figure out a way to dump as much info as possible about
the Heap in case of OutOfMemory.... not many options available in
WL9.2....

you can start your JVM with

http://blog.xebia.fr/2008/11/27/analyse-memoire-dune-jvm/


or with JRockit -Djrockit.oomdiagnostics http://forums.oracle.com/forums/thread.jspa?threadID=697856&tstart=180

Scheduling heap analysis

write a shell script dumpHeap.sh with this content:

export mypid=$(ps -efa|grep Dweblogic.Name=myservername |grep -v grep| awk -F " " '{print $2}')
echo PID=$mypid
jrcmd $mypid print_object_summary cutoff=0 > heapcontent$1.log


then schedule it this way:

echo "dumpHeap.sh 1" | at 20:00
echo "dumpHeap.sh 2" | at 21:00
echo "dumpHeap.sh 3" | at 22:00
echo "dumpHeap.sh 4" | at 23:00
echo "dumpHeap.sh 5" | at 00:00 tomorrow
echo "dumpHeap.sh 6" | at 01:00 tomorrow
echo "dumpHeap.sh 7" | at 02:00 tomorrow
echo "dumpHeap.sh 8" | at 03:00 tomorrow

md5sum

a great way to discover if 2 nodes running WL applications have the same libraries, is to run "md5sum *" on the directory containing the applications and libraries.

Admin server crashing when you delete the stage directory and start all the MS at the same time

<12 août 2009 11 h 41 CEST> <Error> <Deployer> <BEA-149205> <Failed to initialize the application 'beehive-controls-1.0 [LibSpecVersion=1.0,LibImplVersion=1.]' due to error weblogic.management.DeploymentException: Exception occured while downloading files.

weblogic.management.DeploymentException: Exception occured while downloading files

at weblogic.deploy.internal.targetserver.datamanagement.AppDataUpdate.doDownload(AppDataUpdate.java:43)

at weblogic.deploy.internal.targetserver.datamanagement.DataUpdate.download(DataUpdate.java:56)

at weblogic.deploy.internal.targetserver.datamanagement.Data.prepareDataUpdate(Data.java:98)

at weblogic.deploy.internal.targetserver.BasicDeployment.prepareDataUpdate(BasicDeployment.java:669)

at weblogic.deploy.internal.targetserver.BasicDeployment.stageFilesForStatic(BasicDeployment.java:712)

Truncated. see log file for complete stacktrace

java.net.ConnectException: Tried all: '1' addresses, but could not connect over HTTP to server: 'myhost', port: '18501'

at weblogic.net.http.HttpClient.openServer(HttpClient.java:361)

at weblogic.net.http.HttpClient.openServer(HttpClient.java:430)

at weblogic.net.http.HttpClient.<init>(HttpClient.java:159)

at weblogic.net.http.HttpClient.<init>(HttpClient.java:149)

at weblogic.net.http.HttpClient.New(HttpClient.java:265)

Truncated. see log file for complete stacktrace

Dumping the content of a Diagnostic File Store

if in WLST you try:

openfilestore(store, dir='.', create='false')

and you get this:

Traceback (innermost last):
File "", line 1, in ?
File "", line 66, in openfilestore
TypeError: getParam(): 2nd arg can't be coerced to String


don't cry (or, rather, cry but don't despair).
You can still use

java weblogic.store.Admin

openfile -store WLS_DIAGNOSTICS -dir /path/to/data/store/diagnostics

dump -store WLS_DIAGNOSTICS -out /tmp/diags.log

this should give you statistics about the content of the file (events)

JRockit options

http://edocs.bea.com/jrockit/jrdocs/refman/optionXX.html the XX options


http://edocs.bea.com/wlrt/docs11/intro_wlrt/tuning.html tuning


GC logging options
-XgcPrio:throughput -Xverbose:gc,gcreport -XverboseTimeStamp -XverboseLog:logs/gc.log

Weblogic Guardian doesn't start

I have tried running BEA Oracle Weblogic Guardian, it doesn't want to start. Open guardian.ini and set to 512 both the min and max heap space, it should fix the problem.