| 1 | <HTML>
|
|---|
| 2 |
|
|---|
| 3 |
|
|---|
| 4 | <HEAD>
|
|---|
| 5 | <TITLE>
|
|---|
| 6 | Using The HotSpot Serviceability Agent
|
|---|
| 7 | </TITLE>
|
|---|
| 8 | </HEAD>
|
|---|
| 9 |
|
|---|
| 10 | <BODY TEXT="#000000" BGCOLOR="#FFFFFF">
|
|---|
| 11 | <H1>
|
|---|
| 12 | Using The HotSpot Serviceability Agent
|
|---|
| 13 | </H1>
|
|---|
| 14 |
|
|---|
| 15 | <P>
|
|---|
| 16 | <H2>
|
|---|
| 17 | Contents
|
|---|
| 18 | </H2>
|
|---|
| 19 | </P>
|
|---|
| 20 |
|
|---|
| 21 | <UL>
|
|---|
| 22 | <LI> <A HREF = "#INTRODUCTION">Introduction</A>
|
|---|
| 23 | <LI> <A HREF = "#SOURCES">Organization of the sources</A>
|
|---|
| 24 | <LI> <A HREF = "#RUNNING">Running HSDB</A>
|
|---|
| 25 | <LI> <A HREF = "#NOTES">Notes</A>
|
|---|
| 26 | </UL>
|
|---|
| 27 |
|
|---|
| 28 | <H2>
|
|---|
| 29 | <A NAME="INTRODUCTION">
|
|---|
| 30 | Introduction
|
|---|
| 31 | </A>
|
|---|
| 32 | </H2>
|
|---|
| 33 |
|
|---|
| 34 | <P>
|
|---|
| 35 | The HotSpot Serviceability Agent (SA) is a set of Java APIs which
|
|---|
| 36 | mirror the internal APIs of the HotSpot VM and which can be used to
|
|---|
| 37 | examine the state of a HotSpot VM.
|
|---|
| 38 | </P>
|
|---|
| 39 |
|
|---|
| 40 | <P>
|
|---|
| 41 | The system understands the layout of certain VM data structures and is
|
|---|
| 42 | able to traverse these structures in an examination-only fashion; that
|
|---|
| 43 | is, it does not rely on being able to run code in the target VM. For
|
|---|
| 44 | this reason it transparently works with either a running VM or a core
|
|---|
| 45 | file.
|
|---|
| 46 | </P>
|
|---|
| 47 |
|
|---|
| 48 | <P>
|
|---|
| 49 | The system can reconstruct information about Java frames on the stack
|
|---|
| 50 | and objects in the heap. Many of the important data structures in the
|
|---|
| 51 | VM like the CodeCache, Universe, StubQueue, Frames, and VFrames have
|
|---|
| 52 | been exposed and have relatively complete (or at least useful)
|
|---|
| 53 | implementations.
|
|---|
| 54 | </P>
|
|---|
| 55 |
|
|---|
| 56 | <P>
|
|---|
| 57 | A small graphical debugger called HSDB (the "HotSpot Debugger") has
|
|---|
| 58 | been written using these APIs. It provides stack memory dumps
|
|---|
| 59 | annotated with method invocations, compiled-code inlining (if
|
|---|
| 60 | present), interpreted vs. compiled code, interpreter codelets (if
|
|---|
| 61 | interpreted), and live oops from oop-map information. It also provides
|
|---|
| 62 | a tree-based oop inspector. More information will be added as
|
|---|
| 63 | necessary; please <A HREF = "mailto:kenneth.russell@eng">send
|
|---|
| 64 | email</A> with suggestions on what would be useful.
|
|---|
| 65 | </P>
|
|---|
| 66 |
|
|---|
| 67 | <P>
|
|---|
| 68 | The SA currently only works on Solaris. It uses dbx to connect to the
|
|---|
| 69 | remote process or core file and communicates with a small piece of
|
|---|
| 70 | code (an "import module") loaded into the debugger.
|
|---|
| 71 | </P>
|
|---|
| 72 |
|
|---|
| 73 | <H2>
|
|---|
| 74 | <A NAME="SOURCES">
|
|---|
| 75 | Organization of the sources
|
|---|
| 76 | </A>
|
|---|
| 77 | </H2>
|
|---|
| 78 |
|
|---|
| 79 | <P>
|
|---|
| 80 | The Java-side source code, which is the bulk of the SA, is in
|
|---|
| 81 | src/share/vm/agent. The organization of the sun.jvm.hotspot package
|
|---|
| 82 | hierarchy mirrors the organization in the VM. This should allow
|
|---|
| 83 | engineers familiar with the HotSpot sources to quickly understand how
|
|---|
| 84 | the SA works and to make modifications if necessary. To build these
|
|---|
| 85 | sources, cd to src/share/vm/agent and type "make".
|
|---|
| 86 | </P>
|
|---|
| 87 |
|
|---|
| 88 | <P>
|
|---|
| 89 |
|
|---|
| 90 | The SA on Solaris works by communicating with a small piece of code
|
|---|
| 91 | (an "import module") loaded into dbx. The source code for this import
|
|---|
| 92 | module is in src/os/solaris/agent. To build this library, cd to
|
|---|
| 93 | src/os/solaris/agent and type "make 32bit" or "make 64bit". The
|
|---|
| 94 | distinction is necessary because the SPARC version of dbx ships with
|
|---|
| 95 | two versions of its executable, and depending on which architecture
|
|---|
| 96 | (v8 or v9) the debugger is running on selects the appropriate
|
|---|
| 97 | executable. The SA tries the v8 version first, but if you are running
|
|---|
| 98 | on a v9 machine you must provide both versions to the SA.
|
|---|
| 99 | </P>
|
|---|
| 100 |
|
|---|
| 101 | <P>
|
|---|
| 102 |
|
|---|
| 103 | The system is currently hardwired to look on jano for its dbx
|
|---|
| 104 | executable and import module. The relevant directory structure looks
|
|---|
| 105 | like this:
|
|---|
| 106 |
|
|---|
| 107 | <UL>
|
|---|
| 108 | <LI> .../hotspot/sa/
|
|---|
| 109 | <UL>
|
|---|
| 110 | <LI> solaris/
|
|---|
| 111 | <UL>
|
|---|
| 112 | <LI> sparc/
|
|---|
| 113 | <UL>
|
|---|
| 114 | <LI> bin/
|
|---|
| 115 | <UL>
|
|---|
| 116 | <LI> dbx: symlink to (v8) dbx 7.0 executable
|
|---|
| 117 | </UL>
|
|---|
| 118 | </UL>
|
|---|
| 119 | <UL>
|
|---|
| 120 | <LI> lib/
|
|---|
| 121 | <UL>
|
|---|
| 122 | <LI> libsvc_agent_dbx.so: 32-bit version of import module
|
|---|
| 123 | </UL>
|
|---|
| 124 | </UL>
|
|---|
| 125 | <LI> sparcv9/
|
|---|
| 126 | <UL>
|
|---|
| 127 | <LI> lib/
|
|---|
| 128 | <UL>
|
|---|
| 129 | <LI> libsvc_agent_dbx.so: 32-bit version of import module
|
|---|
| 130 | </UL>
|
|---|
| 131 | </UL>
|
|---|
| 132 | </UL>
|
|---|
| 133 | </UL>
|
|---|
| 134 | </UL>
|
|---|
| 135 | </P>
|
|---|
| 136 |
|
|---|
| 137 | <P>
|
|---|
| 138 | The code which builds up path names to these executables is contained
|
|---|
| 139 | in sun.jvm.hotspot.HotSpotAgent.java. There are hardcoded paths in
|
|---|
| 140 | this file to jano, but the rest of the system is isolated from this.
|
|---|
| 141 | </P>
|
|---|
| 142 |
|
|---|
| 143 | <P>
|
|---|
| 144 | (It would be nice to have a panel in the debugger allowing
|
|---|
| 145 | configuration of all of the known operating systems' options; right
|
|---|
| 146 | now Solaris is the only supported OS, making that easier.)
|
|---|
| 147 | </P>
|
|---|
| 148 |
|
|---|
| 149 | <H2>
|
|---|
| 150 | <A NAME="RUNNING">
|
|---|
| 151 | Running HSDB
|
|---|
| 152 | </A>
|
|---|
| 153 | </H2>
|
|---|
| 154 |
|
|---|
| 155 | <P>
|
|---|
| 156 | An installation of HSDB has been placed on jano. To access it, add the
|
|---|
| 157 | following directory to your PATH:
|
|---|
| 158 | </P>
|
|---|
| 159 |
|
|---|
| 160 | <P>
|
|---|
| 161 | <PRE>
|
|---|
| 162 | /net/jano/export/disk05/hotspot/sa/bin/common
|
|---|
| 163 | </PRE>
|
|---|
| 164 | </P>
|
|---|
| 165 |
|
|---|
| 166 | <P>
|
|---|
| 167 | To start the debugger, type "hsdb".
|
|---|
| 168 | </P>
|
|---|
| 169 |
|
|---|
| 170 | <P>
|
|---|
| 171 | Alternatively, you can start a local build of the debugger by building
|
|---|
| 172 | the sources in src/share/vm/agent, cd'ing to that directory, and
|
|---|
| 173 | typing "java sun.jvm.hotspot.HSDB".
|
|---|
| 174 | </P>
|
|---|
| 175 |
|
|---|
| 176 | <P>
|
|---|
| 177 | There are three modes for the debugger: attaching to a local process,
|
|---|
| 178 | opening a core file, and attaching to a remote "debug server". The
|
|---|
| 179 | remote case requires two programs to be running on the remote machine:
|
|---|
| 180 | the rmiregistry (see the script "start-rmiregistry" in this directory;
|
|---|
| 181 | run this in the background) and the debug server (see the script
|
|---|
| 182 | "start-debug-server"), in that order. start-rmiregistry takes no
|
|---|
| 183 | arguments; start-debug-server takes as argument the process ID or the
|
|---|
| 184 | executable and core file names to allow remote debugging of. Make sure
|
|---|
| 185 | you do NOT have a CLASSPATH environment variable set when you run
|
|---|
| 186 | these scripts. (The classes put into the rmiregistry are in sun.*, and
|
|---|
| 187 | there are permissions problems if they aren't placed on the boot
|
|---|
| 188 | classpath.)
|
|---|
| 189 | </P>
|
|---|
| 190 |
|
|---|
| 191 | <P>
|
|---|
| 192 | NOTE that the SA currently only works against VMs on Solaris/SPARC.
|
|---|
| 193 | Remote debugging of Solaris/SPARC VMs on arbitrary platforms is
|
|---|
| 194 | possible using the debug server; select "Connect to debug server..."
|
|---|
| 195 | in HSDB.
|
|---|
| 196 | </P>
|
|---|
| 197 |
|
|---|
| 198 | <P>
|
|---|
| 199 | Once the debugger has been launched, the threads list is displayed.
|
|---|
| 200 | The current set of functionality allows:
|
|---|
| 201 | </P>
|
|---|
| 202 |
|
|---|
| 203 | <UL>
|
|---|
| 204 | <LI> Browsing of the annotated stack memory ("Stack Memory" button). It
|
|---|
| 205 | is currently annotated with the following information:
|
|---|
| 206 | <UL>
|
|---|
| 207 | <LI> Method names of the Java frames and their extents (supporting
|
|---|
| 208 | inlined compiled methods)
|
|---|
| 209 | <LI> Locations and types of oops, found using the oop map information
|
|---|
| 210 | from compiled methods (interpreter oop maps coming soon)
|
|---|
| 211 | <LI> If a Java frame was interrupted by a signal (e.g., because of a
|
|---|
| 212 | crash), annotates the frame with the signal name and number
|
|---|
| 213 | <LI> Interpreter codelet descriptions for interpreted frames
|
|---|
| 214 | </UL>
|
|---|
| 215 | <LI> Finding which thread or threads caused a crash (currently
|
|---|
| 216 | identified by the presence of a signal handler frame)
|
|---|
| 217 | <LI> Browsing of oops using the Oop Inspector.
|
|---|
| 218 | <LI> Browsing of the java.lang.Thread object's oop.
|
|---|
| 219 | <LI> Object histogram and inspection of objects therein.
|
|---|
| 220 | </UL>
|
|---|
| 221 | </P>
|
|---|
| 222 |
|
|---|
| 223 | <P>
|
|---|
| 224 | More functionality is planned. Please <A HREF =
|
|---|
| 225 | "mailto:kenneth.russell@eng">send email</A> with suggestions on what
|
|---|
| 226 | would be useful, with any questions or comments, or if the debugger
|
|---|
| 227 | crashes.
|
|---|
| 228 | </P>
|
|---|
| 229 |
|
|---|
| 230 | <H2>
|
|---|
| 231 | <A NAME="NOTES">
|
|---|
| 232 | Notes
|
|---|
| 233 | </A>
|
|---|
| 234 | </H2>
|
|---|
| 235 |
|
|---|
| 236 | <P>
|
|---|
| 237 | HSDB does not suspend the system at a safepoint, but at an arbitrary
|
|---|
| 238 | point. This means that many of the invariants in the VM's code are not
|
|---|
| 239 | followed.
|
|---|
| 240 | </P>
|
|---|
| 241 |
|
|---|
| 242 | <P>
|
|---|
| 243 | As it happens, most of the places where the code ported over from the
|
|---|
| 244 | VM has failed involve the topmost frame on the stack. Some
|
|---|
| 245 | modifications have been made to allow the system to recognize
|
|---|
| 246 | problematic situations.
|
|---|
| 247 | </P>
|
|---|
| 248 |
|
|---|
| 249 | <P>
|
|---|
| 250 | Certainly, not all of the failure modes of the debugger have been
|
|---|
| 251 | found. Please <A HREF = "mailto:kenneth.russell@eng">send email</A> if
|
|---|
| 252 | HSDB throws an exception. The best debugging aid in these situations
|
|---|
| 253 | is a core file since it is a static view of the VM to which we can
|
|---|
| 254 | then adapt the debugger code, as opposed to having to try to suspend
|
|---|
| 255 | the VM over and over to reproduce the failure. gcore (1) is a useful
|
|---|
| 256 | tool. (NOTE: do not try gcore with any application using the DGA X
|
|---|
| 257 | server extension (example: Java2Demo); the kernel will panic. See bug
|
|---|
| 258 | 4343237.)
|
|---|
| 259 | </P>
|
|---|
| 260 |
|
|---|
| 261 | </BODY>
|
|---|
| 262 | </HTML>
|
|---|
