| 1 | <html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>ctdb-statistics</title><meta name="generator" content="DocBook XSL Stylesheets V1.78.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="refentry"><a name="ctdb-statistics.7"></a><div class="titlepage"></div><div class="refnamediv"><h2>Name</h2><p>ctdb-statistics — CTDB statistics output</p></div><div class="refsect1"><a name="idp52032112"></a><h2>OVERALL STATISTICS</h2><p>
|
|---|
| 2 | CTDB maintains information about various messages communicated
|
|---|
| 3 | and some of the important operations per node. See the
|
|---|
| 4 | <span class="citerefentry"><span class="refentrytitle">ctdb</span>(1)</span> commands
|
|---|
| 5 | <span class="command"><strong>statistics</strong></span> and <span class="command"><strong>statisticsreset</strong></span>
|
|---|
| 6 | for displaying statistics.
|
|---|
| 7 | </p><div class="refsect2"><a name="idp52333936"></a><h3>Example: ctdb statistics</h3><pre class="screen">
|
|---|
| 8 | CTDB version 1
|
|---|
| 9 | Current time of statistics : Fri Sep 12 13:32:32 2014
|
|---|
| 10 | Statistics collected since : (000 01:49:20) Fri Sep 12 11:43:12 2014
|
|---|
| 11 | num_clients 6
|
|---|
| 12 | frozen 0
|
|---|
| 13 | recovering 0
|
|---|
| 14 | num_recoveries 2
|
|---|
| 15 | client_packets_sent 281293
|
|---|
| 16 | client_packets_recv 296317
|
|---|
| 17 | node_packets_sent 452387
|
|---|
| 18 | node_packets_recv 182394
|
|---|
| 19 | keepalive_packets_sent 3927
|
|---|
| 20 | keepalive_packets_recv 3928
|
|---|
| 21 | node
|
|---|
| 22 | req_call 48605
|
|---|
| 23 | reply_call 1
|
|---|
| 24 | req_dmaster 23404
|
|---|
| 25 | reply_dmaster 24917
|
|---|
| 26 | reply_error 0
|
|---|
| 27 | req_message 958
|
|---|
| 28 | req_control 197513
|
|---|
| 29 | reply_control 153705
|
|---|
| 30 | client
|
|---|
| 31 | req_call 130866
|
|---|
| 32 | req_message 770
|
|---|
| 33 | req_control 168921
|
|---|
| 34 | timeouts
|
|---|
| 35 | call 0
|
|---|
| 36 | control 0
|
|---|
| 37 | traverse 0
|
|---|
| 38 | locks
|
|---|
| 39 | num_calls 220
|
|---|
| 40 | num_current 0
|
|---|
| 41 | num_pending 0
|
|---|
| 42 | num_failed 0
|
|---|
| 43 | total_calls 130866
|
|---|
| 44 | pending_calls 0
|
|---|
| 45 | childwrite_calls 1
|
|---|
| 46 | pending_childwrite_calls 0
|
|---|
| 47 | memory_used 334490
|
|---|
| 48 | max_hop_count 18
|
|---|
| 49 | total_ro_delegations 2
|
|---|
| 50 | total_ro_revokes 2
|
|---|
| 51 | hop_count_buckets: 42816 5464 26 1 0 0 0 0 0 0 0 0 0 0 0 0
|
|---|
| 52 | lock_buckets: 9 165 14 15 7 2 2 0 0 0 0 0 0 0 0 0
|
|---|
| 53 | locks_latency MIN/AVG/MAX 0.000685/0.160302/6.369342 sec out of 214
|
|---|
| 54 | reclock_ctdbd MIN/AVG/MAX 0.004940/0.004969/0.004998 sec out of 2
|
|---|
| 55 | reclock_recd MIN/AVG/MAX 0.000000/0.000000/0.000000 sec out of 0
|
|---|
| 56 | call_latency MIN/AVG/MAX 0.000006/0.000719/4.562991 sec out of 126626
|
|---|
| 57 | childwrite_latency MIN/AVG/MAX 0.014527/0.014527/0.014527 sec out of 1
|
|---|
| 58 | </pre></div><div class="refsect2"><a name="idp52992976"></a><h3>CTDB version</h3><p>
|
|---|
| 59 | Version of the ctdb protocol used by the node.
|
|---|
| 60 | </p></div><div class="refsect2"><a name="idp52639696"></a><h3>Current time of statistics</h3><p>
|
|---|
| 61 | Time when the statistics are generated.
|
|---|
| 62 | </p><p>
|
|---|
| 63 | This is useful when collecting statistics output periodically
|
|---|
| 64 | for post-processing.
|
|---|
| 65 | </p></div><div class="refsect2"><a name="idp52827040"></a><h3>Statistics collected since</h3><p>
|
|---|
| 66 | Time when ctdb was started or the last time statistics was reset.
|
|---|
| 67 | The output shows the duration and the timestamp.
|
|---|
| 68 | </p></div><div class="refsect2"><a name="idp52657472"></a><h3>num_clients</h3><p>
|
|---|
| 69 | Number of processes currently connected to CTDB's unix socket.
|
|---|
| 70 | This includes recovery daemon, ctdb tool and samba processes
|
|---|
| 71 | (smbd, winbindd).
|
|---|
| 72 | </p></div><div class="refsect2"><a name="idp50156720"></a><h3>frozen</h3><p>
|
|---|
| 73 | 1 if the the databases are currently frozen, 0 otherwise.
|
|---|
| 74 | </p></div><div class="refsect2"><a name="idp49233296"></a><h3>recovering</h3><p>
|
|---|
| 75 | 1 if recovery is active, 0 otherwise.
|
|---|
| 76 | </p></div><div class="refsect2"><a name="idp49234480"></a><h3>num_recoveries</h3><p>
|
|---|
| 77 | Number of recoveries since the start of ctdb or since the last
|
|---|
| 78 | statistics reset.
|
|---|
| 79 | </p></div><div class="refsect2"><a name="idp49235664"></a><h3>client_packets_sent</h3><p>
|
|---|
| 80 | Number of packets sent to client processes via unix domain socket.
|
|---|
| 81 | </p></div><div class="refsect2"><a name="idp53887440"></a><h3>client_packets_recv</h3><p>
|
|---|
| 82 | Number of packets received from client processes via unix domain socket.
|
|---|
| 83 | </p></div><div class="refsect2"><a name="idp53888624"></a><h3>node_packets_sent</h3><p>
|
|---|
| 84 | Number of packets sent to the other nodes in the cluster via TCP.
|
|---|
| 85 | </p></div><div class="refsect2"><a name="idp53889808"></a><h3>node_packets_recv</h3><p>
|
|---|
| 86 | Number of packets received from the other nodes in the cluster via TCP.
|
|---|
| 87 | </p></div><div class="refsect2"><a name="idp53890992"></a><h3>keepalive_packets_sent</h3><p>
|
|---|
| 88 | Number of keepalive messages sent to other nodes.
|
|---|
| 89 | </p><p>
|
|---|
| 90 | CTDB periodically sends keepalive messages to other nodes.
|
|---|
| 91 | See <em class="citetitle">KeepaliveInterval</em> tunable in
|
|---|
| 92 | <span class="citerefentry"><span class="refentrytitle">ctdb-tunables</span>(7)</span> for more details.
|
|---|
| 93 | </p></div><div class="refsect2"><a name="idp53893984"></a><h3>keepalive_packets_recv</h3><p>
|
|---|
| 94 | Number of keepalive messages received from other nodes.
|
|---|
| 95 | </p></div><div class="refsect2"><a name="idp53895168"></a><h3>node</h3><p>
|
|---|
| 96 | This section lists various types of messages processed which
|
|---|
| 97 | originated from other nodes via TCP.
|
|---|
| 98 | </p><div class="refsect3"><a name="idp53896352"></a><h4>req_call</h4><p>
|
|---|
| 99 | Number of REQ_CALL messages from the other nodes.
|
|---|
| 100 | </p></div><div class="refsect3"><a name="idp53897536"></a><h4>reply_call</h4><p>
|
|---|
| 101 | Number of REPLY_CALL messages from the other nodes.
|
|---|
| 102 | </p></div><div class="refsect3"><a name="idp53898720"></a><h4>req_dmaster</h4><p>
|
|---|
| 103 | Number of REQ_DMASTER messages from the other nodes.
|
|---|
| 104 | </p></div><div class="refsect3"><a name="idp53899904"></a><h4>reply_dmaster</h4><p>
|
|---|
| 105 | Number of REPLY_DMASTER messages from the other nodes.
|
|---|
| 106 | </p></div><div class="refsect3"><a name="idp53901088"></a><h4>reply_error</h4><p>
|
|---|
| 107 | Number of REPLY_ERROR messages from the other nodes.
|
|---|
| 108 | </p></div><div class="refsect3"><a name="idp53902352"></a><h4>req_message</h4><p>
|
|---|
| 109 | Number of REQ_MESSAGE messages from the other nodes.
|
|---|
| 110 | </p></div><div class="refsect3"><a name="idp49113072"></a><h4>req_control</h4><p>
|
|---|
| 111 | Number of REQ_CONTROL messages from the other nodes.
|
|---|
| 112 | </p></div><div class="refsect3"><a name="idp49114336"></a><h4>reply_control</h4><p>
|
|---|
| 113 | Number of REPLY_CONTROL messages from the other nodes.
|
|---|
| 114 | </p></div></div><div class="refsect2"><a name="idp49115728"></a><h3>client</h3><p>
|
|---|
| 115 | This section lists various types of messages processed which
|
|---|
| 116 | originated from clients via unix domain socket.
|
|---|
| 117 | </p><div class="refsect3"><a name="idp49116912"></a><h4>req_call</h4><p>
|
|---|
| 118 | Number of REQ_CALL messages from the clients.
|
|---|
| 119 | </p></div><div class="refsect3"><a name="idp49118176"></a><h4>req_message</h4><p>
|
|---|
| 120 | Number of REQ_MESSAGE messages from the clients.
|
|---|
| 121 | </p></div><div class="refsect3"><a name="idp49119440"></a><h4>req_control</h4><p>
|
|---|
| 122 | Number of REQ_CONTROL messages from the clients.
|
|---|
| 123 | </p></div></div><div class="refsect2"><a name="idp49120832"></a><h3>timeouts</h3><p>
|
|---|
| 124 | This section lists timeouts occurred when sending various messages.
|
|---|
| 125 | </p><div class="refsect3"><a name="idp49121888"></a><h4>call</h4><p>
|
|---|
| 126 | Number of timeouts for REQ_CALL messages.
|
|---|
| 127 | </p></div><div class="refsect3"><a name="idp49123152"></a><h4>control</h4><p>
|
|---|
| 128 | Number of timeouts for REQ_CONTROL messages.
|
|---|
| 129 | </p></div><div class="refsect3"><a name="idp49124416"></a><h4>traverse</h4><p>
|
|---|
| 130 | Number of timeouts for database traverse operations.
|
|---|
| 131 | </p></div></div><div class="refsect2"><a name="idp49125808"></a><h3>locks</h3><p>
|
|---|
| 132 | This section lists locking statistics.
|
|---|
| 133 | </p><div class="refsect3"><a name="idp49126864"></a><h4>num_calls</h4><p>
|
|---|
| 134 | Number of completed lock calls. This includes database locks
|
|---|
| 135 | and record locks.
|
|---|
| 136 | </p></div><div class="refsect3"><a name="idp49128160"></a><h4>num_current</h4><p>
|
|---|
| 137 | Number of scheduled lock calls. This includes database locks
|
|---|
| 138 | and record locks.
|
|---|
| 139 | </p></div><div class="refsect3"><a name="idp49129456"></a><h4>num_pending</h4><p>
|
|---|
| 140 | Number of queued lock calls. This includes database locks and
|
|---|
| 141 | record locks.
|
|---|
| 142 | </p></div><div class="refsect3"><a name="idp49135552"></a><h4>num_failed</h4><p>
|
|---|
| 143 | Number of failed lock calls. This includes database locks and
|
|---|
| 144 | record locks.
|
|---|
| 145 | </p></div></div><div class="refsect2"><a name="idp49136976"></a><h3>total_calls</h3><p>
|
|---|
| 146 | Number of req_call messages processed from clients. This number
|
|---|
| 147 | should be same as client --> req_call.
|
|---|
| 148 | </p></div><div class="refsect2"><a name="idp49138288"></a><h3>pending_calls</h3><p>
|
|---|
| 149 | Number of req_call messages which are currenly being processed.
|
|---|
| 150 | This number indicates the number of record migrations in flight.
|
|---|
| 151 | </p></div><div class="refsect2"><a name="idp49139632"></a><h3>childwrite_calls</h3><p>
|
|---|
| 152 | Number of record update calls. Record update calls are used to
|
|---|
| 153 | update a record under a transaction.
|
|---|
| 154 | </p></div><div class="refsect2"><a name="idp49140944"></a><h3>pending_childwrite_calls</h3><p>
|
|---|
| 155 | Number of record update calls currently active.
|
|---|
| 156 | </p></div><div class="refsect2"><a name="idp49142224"></a><h3>memory_used</h3><p>
|
|---|
| 157 | The amount of memory in bytes currently used by CTDB using
|
|---|
| 158 | talloc. This includes all the memory used for CTDB's internal
|
|---|
| 159 | data structures. This does not include the memory mapped TDB
|
|---|
| 160 | databases.
|
|---|
| 161 | </p></div><div class="refsect2"><a name="idp49143632"></a><h3>max_hop_count</h3><p>
|
|---|
| 162 | The maximum number of hops required for a record migration request
|
|---|
| 163 | to obtain the record. High numbers indicate record contention.
|
|---|
| 164 | </p></div><div class="refsect2"><a name="idp49144976"></a><h3>total_ro_delegations</h3><p>
|
|---|
| 165 | Number of readonly delegations created.
|
|---|
| 166 | </p></div><div class="refsect2"><a name="idp49146160"></a><h3>total_ro_revokes</h3><p>
|
|---|
| 167 | Number of readonly delegations that were revoked. The difference
|
|---|
| 168 | between total_ro_revokes and total_ro_delegations gives the
|
|---|
| 169 | number of currently active readonly delegations.
|
|---|
| 170 | </p></div><div class="refsect2"><a name="idp49147552"></a><h3>hop_count_buckets</h3><p>
|
|---|
| 171 | Distribution of migration requests based on hop counts values.
|
|---|
| 172 | Buckets are 1, < 4, < 8, < 16, < 32, < 64, <
|
|---|
| 173 | 128, < 256, < 512, ≥ 512.
|
|---|
| 174 | </p></div><div class="refsect2"><a name="idp49154896"></a><h3>lock_buckets</h3><p>
|
|---|
| 175 | Distribution of record lock requests based on time required to
|
|---|
| 176 | obtain locks. Buckets are < 1ms, < 10ms, < 100ms,
|
|---|
| 177 | < 1s, < 2s, < 4s, < 8s, < 16s, < 32s, <
|
|---|
| 178 | 64s, ≥ 64s.
|
|---|
| 179 | </p></div><div class="refsect2"><a name="idp49156320"></a><h3>locks_latency</h3><p>
|
|---|
| 180 | The minimum, the average and the maximum time (in seconds)
|
|---|
| 181 | required to obtain record locks.
|
|---|
| 182 | </p></div><div class="refsect2"><a name="idp49157472"></a><h3>reclock_ctdbd</h3><p>
|
|---|
| 183 | The minimum, the average and the maximum time (in seconds)
|
|---|
| 184 | required to check if recovery lock is still held by recovery
|
|---|
| 185 | daemon when recovery mode is changed. This check is done in ctdb daemon.
|
|---|
| 186 | </p></div><div class="refsect2"><a name="idp49149392"></a><h3>reclock_recd</h3><p>
|
|---|
| 187 | The minimum, the average and the maximum time (in seconds)
|
|---|
| 188 | required to check if recovery lock is still held by recovery
|
|---|
| 189 | daemon during recovery. This check is done in recovery daemon.
|
|---|
| 190 | </p></div><div class="refsect2"><a name="idp49150768"></a><h3>call_latency</h3><p>
|
|---|
| 191 | The minimum, the average and the maximum time (in seconds) required
|
|---|
| 192 | to process a REQ_CALL message from client. This includes the time
|
|---|
| 193 | required to migrate a record from remote node, if the record is
|
|---|
| 194 | not available on the local node.
|
|---|
| 195 | </p></div><div class="refsect2"><a name="idp49152208"></a><h3>childwrite_latency</h3><p>Default: 0</p><p>
|
|---|
| 196 | The minimum, the average and the maximum time (in seconds)
|
|---|
| 197 | required to update records under a transaction.
|
|---|
| 198 | </p></div></div><div class="refsect1"><a name="idp55020752"></a><h2>DATABASE STATISTICS</h2><p>
|
|---|
| 199 | CTDB maintains per database statistics about important operations.
|
|---|
| 200 | See the <span class="citerefentry"><span class="refentrytitle">ctdb</span>(1)</span> command
|
|---|
| 201 | <span class="command"><strong>dbstatistics</strong></span> for displaying database statistics.
|
|---|
| 202 | </p><div class="refsect2"><a name="idp55023408"></a><h3>Example: ctdb dbstatistics notify_index.tdb</h3><pre class="screen">
|
|---|
| 203 | DB Statistics: notify_index.tdb
|
|---|
| 204 | ro_delegations 0
|
|---|
| 205 | ro_revokes 0
|
|---|
| 206 | locks
|
|---|
| 207 | total 131
|
|---|
| 208 | failed 0
|
|---|
| 209 | current 0
|
|---|
| 210 | pending 0
|
|---|
| 211 | hop_count_buckets: 9890 5454 26 1 0 0 0 0 0 0 0 0 0 0 0 0
|
|---|
| 212 | lock_buckets: 4 117 10 0 0 0 0 0 0 0 0 0 0 0 0 0
|
|---|
| 213 | locks_latency MIN/AVG/MAX 0.000683/0.004198/0.014730 sec out of 131
|
|---|
| 214 | Num Hot Keys: 3
|
|---|
| 215 | Count:7 Key:2f636c75737465726673
|
|---|
| 216 | Count:18 Key:2f636c757374657266732f64617461
|
|---|
| 217 | Count:7 Key:2f636c757374657266732f646174612f636c69656e7473
|
|---|
| 218 | </pre></div><div class="refsect2"><a name="idp55025440"></a><h3>DB Statistics</h3><p>
|
|---|
| 219 | Name of the database.
|
|---|
| 220 | </p></div><div class="refsect2"><a name="idp55026592"></a><h3>ro_delegations</h3><p>
|
|---|
| 221 | Number of readonly delegations created in the database.
|
|---|
| 222 | </p></div><div class="refsect2"><a name="idp55027744"></a><h3>ro_revokes</h3><p>
|
|---|
| 223 | Number of readonly delegations revoked. The difference in
|
|---|
| 224 | ro_delegations and ro_revokes indicates the currently active
|
|---|
| 225 | readonly delegations.
|
|---|
| 226 | </p></div><div class="refsect2"><a name="idp55029072"></a><h3>locks</h3><p>
|
|---|
| 227 | This section lists locking statistics.
|
|---|
| 228 | </p><div class="refsect3"><a name="idp55030096"></a><h4>total</h4><p>
|
|---|
| 229 | Number of completed lock calls. This includes database locks
|
|---|
| 230 | and record locks.
|
|---|
| 231 | </p></div><div class="refsect3"><a name="idp55031360"></a><h4>failed</h4><p>
|
|---|
| 232 | Number of failed lock calls. This includes database locks and
|
|---|
| 233 | record locks.
|
|---|
| 234 | </p></div><div class="refsect3"><a name="idp55032624"></a><h4>current</h4><p>
|
|---|
| 235 | Number of scheduled lock calls. This includes database locks
|
|---|
| 236 | and record locks.
|
|---|
| 237 | </p></div><div class="refsect3"><a name="idp55033888"></a><h4>pending</h4><p>
|
|---|
| 238 | Number of queued lock calls. This includes database locks and
|
|---|
| 239 | record locks.
|
|---|
| 240 | </p></div></div><div class="refsect2"><a name="idp55035280"></a><h3>hop_count_buckets</h3><p>
|
|---|
| 241 | Distribution of migration requests based on hop counts values.
|
|---|
| 242 | Buckets are 1, < 4, < 8, < 16, < 32, < 64, <
|
|---|
| 243 | 128, < 256, < 512, ≥ 512.
|
|---|
| 244 | </p></div><div class="refsect2"><a name="idp55036640"></a><h3>lock_buckets</h3><p>
|
|---|
| 245 | Distribution of record lock requests based on time required to
|
|---|
| 246 | obtain locks. Buckets are < 1ms, < 10ms, < 100ms,
|
|---|
| 247 | < 1s, < 2s, < 4s, < 8s, < 16s, < 32s, <
|
|---|
| 248 | 64s, ≥ 64s.
|
|---|
| 249 | </p></div><div class="refsect2"><a name="idp55038064"></a><h3>locks_latency</h3><p>
|
|---|
| 250 | The minimum, the average and the maximum time (in seconds)
|
|---|
| 251 | required to obtain record locks.
|
|---|
| 252 | </p></div><div class="refsect2"><a name="idp55039328"></a><h3>Num Hot Keys</h3><p>
|
|---|
| 253 | Number of contended records determined by hop count. CTDB keeps
|
|---|
| 254 | track of top 10 hot records and the output shows hex encoded
|
|---|
| 255 | keys for the hot records.
|
|---|
| 256 | </p></div></div><div class="refsect1"><a name="idp55040800"></a><h2>SEE ALSO</h2><p>
|
|---|
| 257 | <span class="citerefentry"><span class="refentrytitle">ctdb</span>(1)</span>,
|
|---|
| 258 |
|
|---|
| 259 | <span class="citerefentry"><span class="refentrytitle">ctdbd</span>(1)</span>,
|
|---|
| 260 |
|
|---|
| 261 | <span class="citerefentry"><span class="refentrytitle">ctdb-tunables</span>(7)</span>,
|
|---|
| 262 |
|
|---|
| 263 | <a class="ulink" href="http://ctdb.samba.org/" target="_top">http://ctdb.samba.org/</a>
|
|---|
| 264 | </p></div></div></body></html>
|
|---|