This write up is the first part of the series “understanding
and analyzing javacore”. Here in this section I will provide you a brief
idea about the javacore (thread dump) file and its contents. Do not confuse javacore (thread dump) with operating
system core dump. A system core contains the dump of entire address space while
javacore (thread dump) is a snapshot of Java Virtual Machine.
Now let us try to understand
the javacore (thread dump) file. A typical IBM thread dump file looks like this
“javacore.20150617.204531.5220.0002.txt” .Below is the details of naming
convention of a typical IBM javacore file.
javacore. 20150617. 204531. 5220. 0002. txt
Prefix yyyymmdd hhmmss pid
sequence number File
extension
Now let us try to analyze the content of a javacore (thread
dump) file manually. It has many sub-components.
Note:- Below dump file is a manually generated example thread
dump of an IBM JVM in a windows 64 bit
architecture.
Each section in below example is an excerpt taken from
original dump for the demonstration purpose. The original dump contains more
details than displayed below. Also note that the section numbering is only for
demonstration purpose and the original dump does not have it.
1. 0SECTION
TITLE subcomponent dump routine
This will give you the cause of dump generation, Path of the dump file
name etc.
NULL
===============================
1TICHARSET 1252
1TISIGINFO Dump Requested By User (00100000) Through
com.ibm.jvm.Dump.JavaDump
1TIDATETIME Date: 2015/06/17 at 20:45:31
1TIFILENAME Javacore filename:
D:\IBM\WebSphere\AppServer\profiles\AppSrv01\javacore.20150617.204531.5220.0002.txt
1TIREQFLAGS Request Flags: 0x81
(exclusive+preempt)
1TIPREPSTATE Prep State: 0x6
(vm_access+exclusive_vm_access)
NULL
------------------------------------------------------------------------
2. 0SECTION GPINFO subcomponent dump routine
This provides the details of OS where JVM is running.
NULL
================================
2XHOSLEVEL OS Level : Windows 7 6.1 build 7601 Service
Pack 1
2XHCPUS Processors -
3XHCPUARCH Architecture : amd64
3XHNUMCPUS How Many : 4
3XHNUMASUP NUMA is either not
supported or has been disabled by user
NULL
1XHERROR2 Register dump
section only produced for SIGSEGV, SIGILL or SIGFPE.
NULL
NULL
------------------------------------------------------------------------
3. 0SECTION ENVINFO subcomponent dump routine
Here you will get complete information of the JVM which caused the dump
generation
NULL
=================================
1CIJAVAVERSION JRE 1.6.0 Windows 7 amd64-64 build
20110418_80450 (pwa6460_26fp1-20110419_01)
1CIVMVERSION
VM build R26_Java626_GA_FP1_20110418_1915_B80450
1CIJITVERSION
JIT enabled, AOT enabled - r11_20110215_18645ifx8
1CIGCVERSION
GC - R26_Java626_GA_FP1_20110418_1915_B80450_CMPRSS
1CIRUNNINGAS
Running as a standalone JVM
1CICMDLINE
D:\IBM\WebSphere\AppServer/java/bin/java -Declipse.security
-Dwas.status.socket=52403 -Dosgi.install.area=D:\IBM\WebSphere\AppServer
-Dosgi.configuration.area=D:\IBM\WebSphere\AppServer\profiles\AppSrv01/servers/server1/configuration
-Dosgi.framework.extensions=com.ibm.cds,com.ibm.ws.eclipse.adaptors
-Xshareclasses:name=webspherev80,nonFatal -Dsun.reflect.inflationThreshold=250
-Xbootclasspath/p:D:\IBM\WebSphere\AppServer/java/jre/lib/ext/ibmorb.jar;D:\IBM\WebSphere\AppServer/java/jre/lib/ext/ibmext.jar
-classpath
D:\IBM\WebSphere\AppServer\profiles\AppSrv01/properties;D:\IBM\WebSphere\AppServer/properties;D:\IBM\WebSphere\AppServer/lib/startup.jar;D:\IBM\WebSphere\AppServer/lib/bootstrap.jar;D:\IBM\WebSphere\AppServer/lib/jsf-nls.jar;D:\IBM\WebSphere\AppServer/lib/lmproxy.jar;D:\IBM\WebSphere\AppServer/lib/urlprotocols.jar;D:\IBM\WebSphere\AppServer/deploytool/itp/batchboot.jar;D:\IBM\WebSphere\AppServer/deploytool/itp/batch2.jar;D:\IBM\WebSphere\AppServer/java/lib/tools.jar
-Dibm.websphere.internalClassAccessMode=allow -Xms50m -Xmx256m -Xcompressedrefs
-Xscmaxaot4M -Xscmx90M
-Dws.ext.dirs=D:\IBM\WebSphere\AppServer/java/lib;D:\IBM\WebSphere\AppServer\profiles\AppSrv01/classes;D:\IBM\WebSphere\AppServer/classes;D:\IBM\WebSphere\AppServer/lib;D:\IBM\WebSphere\AppServer/installedChannels;D:\IBM\WebSphere\AppServer/lib/ext;D:\IBM\WebSphere\AppServer/web/help;D:\IBM\WebSphere\AppServer/deploytool/itp/plugins/com.ibm.etools.ejbdeploy/runtime
-Dderby.system.home=D:\IBM\WebSphere\AppServer/derby
-Dcom.ibm.itp.location=D:\IBM\WebSphere\AppServer/bin
-Djava.util.logging.configureByServer=true
-Duser.install.root=D:\IBM\WebSphere\AppServer\profiles\AppSrv01
-Djava.ext.dirs=D:\IBM\WebSphere\AppServer/tivoli/tam;D:\IBM\WebSphere\AppServer/java/jre/lib/ext
-Djavax.management.builder.initial=com.ibm.ws.management.PlatformMBeanServerBuilder
-Dpython.cachedir=D:\IBM\WebSphere\AppServer\profiles\AppSrv01/temp/cachedir
-Dwas.install.root=D:\IBM\WebSphere\AppServer
-Djava.util.logging.manager=com.ibm.ws.bootstrap.WsLogManager
-Dserver.root=D:\IBM\WebSphere\AppServer\profiles\AppSrv01
-Dcom.ibm.security.jgss.debug=off -Dcom.ibm.security.krb5.Krb5Debug=off
-Djava.library.path=D:\IBM\WebSphere\AppServer/lib/native/win/x86_64/;D:\IBM\WebSphere\AppServer\java\jre\bin\default;D:\IBM\WebSphere\AppServer\java\jre\bin;.;D:\IBM\WebSphere\AppServer\lib\native\win\x86_64;D:\IBM\WebSphere\AppServer\bin;D:\IBM\WebSphere\AppServer\java\bin;D:\IBM\WebSphere\AppServer\java\jre\bin;C:\app\Nithya\product\11.2.0\dbhome_2\bin;C:\app\Nithya\product\11.2.0\dbhome_1\bin;C:\Program
Files\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Common
Files\Microsoft Shared\Windows Live;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program
Files\Lenovo\Bluetooth Software\;C:\Program Files\Lenovo\Bluetooth
Software\syswow64;C:\Program Files (x86)\Windows
Live\Shared;D:\IBM\WebSphere\AppServer\java\jre\bin; -Djava.endorsed.dirs=D:\IBM\WebSphere\AppServer/endorsed_apis;D:\IBM\WebSphere\AppServer/java/jre/lib/endorsed;D:\IBM\WebSphere\AppServer\endorsed_apis;D:\IBM\WebSphere\AppServer\java\jre\lib\endorsed
-Djava.security.auth.login.config=D:\IBM\WebSphere\AppServer\profiles\AppSrv01/properties/wsjaas.conf
-Djava.security.policy=D:\IBM\WebSphere\AppServer\profiles\AppSrv01/properties/server.policy
com.ibm.wsspi.bootstrap.WSPreLauncher -nosplash -application
com.ibm.ws.bootstrap.WSLauncher com.ibm.ws.runtime.WsServer
D:\IBM\WebSphere\AppServer\profiles\AppSrv01\config Prasanth-PCCell01
Prasanth-PCNode01 server1
1CIJAVAHOMEDIR Java Home Dir: D:\IBM\WebSphere\AppServer\java\jre
1CIJAVADLLDIR Java
DLL Dir:
D:\IBM\WebSphere\AppServer\java\jre\bin
-----------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------
1.
0SECTION NATIVEMEMINFO subcomponent dump routine
This section contains the information about of native memory
NULL =================================
0MEMUSER
1MEMUSER JRE: 506,878,360 bytes / 7159 allocations
1MEMUSER |
2MEMUSER +--VM: 479,558,608 bytes / 6241
allocations
2MEMUSER |
|
3MEMUSER |
+--Classes: 99,900,376 bytes / 3183 allocations
3MEMUSER | | |
4MEMUSER | |
+--Shared Class Cache: 52,428,800 bytes / 1 allocation
3MEMUSER | | |
4MEMUSER | |
+--Other: 47,471,576 bytes / 3182 allocations
2MEMUSER | |
3MEMUSER | +--Memory Manager (GC): 277,002,624 bytes /
816 allocations
------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------
2. 0SECTION
MEMINFO subcomponent dump routine
Here you will get the Memory information
NULL
=================================
NULL
1STHEAPTYPE Object Memory
NULL region start end size name
1STHEAP 0x00000000001B3290 0x0000000001070000
0x00000000035F0000 0x0000000002580000 Tenured Region
1STHEAP 0x00000000001B30B0
0x0000000010110000 0x0000000010CC0000 0x0000000000BB0000 Nursery Region
1STHEAP 0x00000000001B31A0
0x0000000010CC0000 0x0000000011070000 0x00000000003B0000 Nursery Region
NULL
1STHEAPTOTAL Total memory: 55443456 (0x00000000034E0000)
1STHEAPINUSE Total memory in
use: 33983224 (0x0000000002068AF8)
1STHEAPFREE Total memory
free: 21460232 (0x0000000001477508)
NULL
1STSEGTYPE Internal Memory
NULL segment start alloc end type size
1STSEGMENT 0x000000001E33C118
0x00000000239FC490 0x00000000239FC490 0x0000000023A0C490 0x01000040 0x0000000000010000
1STSEGMENT 0x000000001E026C08
0x0000000023CAEAD0 0x0000000023CAEAD0 0x0000000023CBEAD0 0x01000040
0x0000000000010000
1STSEGMENT 0x000000001E026CC0 0x0000000024037FD0
0x0000000024037FD0 0x0000000024047FD0 0x01000040 0x0000000000010000
1STSEGMENT 0x000000001E6D2E78 0x0000000023CCEB70
0x0000000023CCEB70 0x0000000023CDEB70 0x01000040 0x0000000000010000
1STSEGMENT 0x000000001E58B7B0 0x000000002380C0E0
0x000000002380C0E0 0x000000002381C0E0 0x00800040 0x0000000000010000
1STSEGMENT 0x0000000023F24860
0x00000000244B9FF0 0x00000000244B9FF0 0x00000000244C9FF0 0x01000040
0x0000000000010000
1STSEGMENT 0x000000001E33BEF0
0x000000002459B5A0 0x000000002459B5A0 0x00000000245AB5A0 0x01000040
0x0000000000010000
1STSEGMENT 0x000000001E33ABB8 0x00000000240F7D40
0x00000000240F7D40 0x0000000024107D40 0x01000040 0x0000000000010000
1STSEGMENT 0x0000000023F246F0 0x00000000240E7CF0
0x00000000240E7CF0 0x00000000240F7CF0 0x01000040 0x0000000000010000
------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------
3. 0SECTION LOCKS subcomponent dump routine
This section provides the information about the monitor lock on threads
NULL
===============================
NULL
1LKPOOLINFO Monitor pool info:
2LKPOOLTOTAL Current total number of monitors: 125
NULL
1LKMONPOOLDUMP Monitor Pool Dump (flat & inflated object-monitors):
2LKMONINUSE sys_mon_t:0x0000000000D72FB0 infl_mon_t:
0x0000000000D73028:
3LKMONOBJECT
org/eclipse/core/internal/jobs/WorkerPool@0x0000000001975570/0x0000000001975574:
<unowned>
3LKNOTIFYQ Waiting to be notified:
3LKWAITNOTIFY "Worker-0" (0x0000000011483400)
2LKMONINUSE sys_mon_t:0x0000000000DAEDF8 infl_mon_t:
0x0000000000DAEE70:
3LKMONOBJECT
java/util/LinkedList@0x00000000024B3300/0x00000000024B3304:
<unowned>
3LKNOTIFYQ Waiting to be notified:
3LKWAITNOTIFY
"Thread-24" (0x000000001203D900)
2LKMONINUSE
sys_mon_t:0x0000000000DAF140 infl_mon_t: 0x0000000000DAF1B8:
3LKMONOBJECT
com/ibm/ejs/j2c/work/WorkScheduler@0x00000000028E8B78/0x00000000028E8B7C:
<unowned>
3LKNOTIFYQ Waiting to be notified:
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------
1. 0SECTION
THREADS subcomponent dump routine
Here you will get to know about the threads and its status, Priority etc
NULL
=================================
NULL
1XMTHDINFO Thread Details
NULL ------------------
NULL
3XMTHREADINFO
"IProfiler" J9VMThread:0x0000000011084100,
j9thread_t:0x0000000000D70530, java/lang/Thread:0x00000000010CAA18, state:CW,
prio=5
3XMTHREADINFO1 (native
thread ID:0x13C8, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO3 No Java
callstack associated with this thread
3XMTHREADINFO3 No native
callstack available on this platform
NULL
NULL
3XMTHREADINFO
"ThreadManager.JobsProcessorThread.DefaultCoreGroup.ApplicationLayerThread"
J9VMThread:0x000000001216F400, j9thread_t:0x0000000000D9F250,
java/lang/Thread:0x00000000024D6498, state:CW, prio=5
3XMTHREADINFO1 (native
thread ID:0x13A4, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO3 Java
callstack:
4XESTACKTRACE at
java/lang/Object.wait(Native Method)
4XESTACKTRACE at
java/lang/Object.wait(Object.java:167(Compiled Code))
4XESTACKTRACE at
com/ibm/ws/dcs/vri/common/JobsProcessorThread.run(JobsProcessorThread.java:249)
3XMTHREADINFO3 No native
callstack available on this platform
NULL
NULL
3XMTHREADINFO
"Thread-21" J9VMThread:0x0000000011B74300, j9thread_t:0x0000000000D9ABA0,
java/lang/Thread:0x00000000020C0210, state:CW, prio=5
3XMTHREADINFO1 (native
thread ID:0x12A0, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO3 Java
callstack:
4XESTACKTRACE at
java/lang/Object.wait(Native Method)
4XESTACKTRACE at
java/lang/Object.wait(Object.java:167)
4XESTACKTRACE at
java/util/Timer$TimerImpl.run(Timer.java:221)
3XMTHREADINFO3 No native
callstack available on this platform
NULL
NULL
-------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------
2. 0SECTION
SHARED CLASSES subcomponent dump routine
Here it provides the info about shared classes
NULL
========================================
NULL
1SCLTEXTCRTW Cache Created With
NULL ------------------
NULL
2SCLTEXTXNL
-Xnolinenumbers = false
NULL
1SCLTEXTCSUM Cache Summary
NULL ------------------
NULL
2SCLTEXTRCS ROMClass start
address =
0x0000000019A56000
2SCLTEXTRCE ROMClass end
address =
0x000000001BBD6C10
2SCLTEXTMSA Metadata start
address = 0x000000001BBD702C
2SCLTEXTCEA Cache end
address =
0x000000001CC00000
2SCLTEXTRTF Runtime
flags =
0x00000001ECA6A297
2SCLTEXTCGN Cache
generation = 11
NULL
2SCLTEXTCSZ Cache size = 52428192
2SCLTEXTFRB Free bytes = 1052
2SCLTEXTRCB ROMClass
bytes =
35130384
2SCLTEXTAOB AOT code
bytes = 7714160
2SCLTEXTADB AOT data
bytes = 136
2SCLTEXTAHB AOT class
hierarchy bytes = 13208
2SCLTEXTATB AOT thunk
bytes = 41940
2SCLTEXTJHB JIT hint bytes = 0
2SCLTEXTJPB JIT profile
bytes = 422132
2SCLTEXTNOB Java Object
bytes = 0
2SCLTEXTZCB Zip cache
bytes = 1126964
2SCLTEXTRWB ReadWrite
bytes = 351648
2SCLTEXTJCB JCL data
bytes = 0
2SCLTEXTBDA Byte data
bytes = 0
2SCLTEXTMDA Metadata
bytes = 859976
2SCLTEXTDAS Class debug area
size = 6766592
2SCLTEXTDAU Class debug area
% used = 100%
2SCLTEXTDAN Class
LineNumberTable bytes =
3150992
2SCLTEXTDAV Class LocalVariableTable
bytes = 3615600
NULL
2SCLTEXTNRC Number
ROMClasses =
12163
2SCLTEXTNAM Number AOT
Methods = 2899
2SCLTEXTNAD Number AOT Data
Entries = 1
2SCLTEXTNAH Number AOT Class
Hierarchy = 336
2SCLTEXTNAT Number AOT
Thunks = 854
2SCLTEXTNJH Number JIT
Hints = 0
2SCLTEXTNJP Number JIT
Profiles = 2309
2SCLTEXTNCP Number
Classpaths = 10
2SCLTEXTNUR Number URLs = 124
2SCLTEXTNTK Number
Tokens = 0
2SCLTEXTNOJ Number Java
Objects = 0
2SCLTEXTNZC Number Zip
Caches = 31
2SCLTEXTNJC Number JCL
Entries = 0
2SCLTEXTNST Number Stale
classes = 0
2SCLTEXTPST Percent Stale
classes = 0%
NULL
2SCLTEXTCPF Cache is 99%
full
--------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------
1. 0SECTION
CLASSES subcomponent dump routine
This section provides the classes and its related information
NULL
=================================
1CLTEXTCLLOS Classloader summaries
1CLTEXTCLLSS
12345678:
1=primordial,2=extension,3=shareable,4=middleware,5=system,6=trusted,7=application,8=delegating
2CLTEXTCLLOADER ----st--
Loader sun/reflect/DelegatingClassLoader(0x0000000010943E60), Parent
*none*(0x0000000000000000)
3CLNMBRLOADEDLIB Number
of loaded libraries 0
3CLNMBRLOADEDCL Number
of loaded classes 1
2CLTEXTCLLOADER ----st--
Loader sun/reflect/DelegatingClassLoader(0x0000000010944488), Parent
*none*(0x0000000000000000)
3CLNMBRLOADEDLIB Number
of loaded libraries 0
3CLNMBRLOADEDCL Number
of loaded classes 1
2CLTEXTCLLOADER -----t--
Loader sun/reflect/DelegatingClassLoader(0x0000000010944A38), Parent
org/eclipse/osgi/internal/baseadaptor/DefaultClassLoader(0x0000000001A553A0)
3CLNMBRLOADEDLIB Number
of loaded libraries 0
3CLNMBRLOADEDCL Number
of loaded classes 1
2CLTEXTCLLOADER ----st--
Loader sun/reflect/DelegatingClassLoader(0x00000000109453B0), Parent
*none*(0x0000000000000000)
3CLNMBRLOADEDLIB Number
of loaded libraries 0
3CLNMBRLOADEDCL Number
of loaded classes 1
--------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------
In the upcoming sections I will explain you about analyzing the thread
dump using popular IBM tool.
Keep watching this space for the same. Hope you enjoyed reading. Don’t forget
to post your comments about the write up.
