kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jun...@apache.org
Subject svn commit: r1657920 - /kafka/site/082/ops.html
Date Fri, 06 Feb 2015 18:50:41 GMT
Author: junrao
Date: Fri Feb  6 18:50:41 2015
New Revision: 1657920

URL: http://svn.apache.org/r1657920
Log:
add new producer metrics to 0.8.2 doc

Modified:
    kafka/site/082/ops.html

Modified: kafka/site/082/ops.html
URL: http://svn.apache.org/viewvc/kafka/site/082/ops.html?rev=1657920&r1=1657919&r2=1657920&view=diff
==============================================================================
--- kafka/site/082/ops.html (original)
+++ kafka/site/082/ops.html Fri Feb  6 18:50:41 2015
@@ -586,6 +586,253 @@ We pay particular we do graphing and ale
     </tr>
 </tbody></table>
 
+<h4><a id="new_producer_monitoring">New producer monitoring</a></h4>
+
+The following metrics are available on new producer instances.
+
+<table class="data-table">
+<tbody><tr>
+      <th>Metric/Attribute name</th>
+      <th>Description</th>
+      <th>Mbean name</th>
+    </tr>
+      <tr>
+      <td>waiting-threads</td>
+      <td>The number of user threads blocked waiting for buffer memory to enqueue their
records</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>buffer-total-bytes</td>
+      <td>The maximum amount of buffer memory the client can use (whether or not it
is currently used).</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>buffer-available-bytes</td>
+      <td>The total amount of buffer memory that is not being used (either unallocated
or in the free list).</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>bufferpool-wait-time</td>
+      <td>The fraction of time an appender waits for space allocation.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>batch-size-avg</td>
+      <td>The average number of bytes sent per partition per-request.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>batch-size-max</td>
+      <td>The max number of bytes sent per partition per-request.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>compression-rate-avg</td>
+      <td>The average compression rate of record batches.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-queue-time-avg</td>
+      <td>The average time in ms record batches spent in the record accumulator.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-queue-time-max</td>
+      <td>The maximum time in ms record batches spent in the record accumulator</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>request-latency-avg</td>
+      <td>The average request latency in ms</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>request-latency-max</td>
+      <td>The maximum request latency in ms</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-send-rate</td>
+      <td>The average number of records sent per second.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>records-per-request-avg</td>
+      <td>The average number of records per request.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-retry-rate</td>
+      <td>The average per-second number of retried record sends</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-error-rate</td>
+      <td>The average per-second number of record sends that resulted in errors</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-size-max</td>
+      <td>The maximum record size</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-size-avg</td>
+      <td>The average record size</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>requests-in-flight</td>
+      <td>The current number of in-flight requests awaiting a response.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>metadata-age</td>
+      <td>The age in seconds of the current producer metadata being used.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>connection-close-rate</td>
+      <td>Connections closed per second in the window.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>connection-creation-rate</td>
+      <td>New connections established per second in the window.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>network-io-rate</td>
+      <td>The average number of network operations (reads or writes) on all connections
per second.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>outgoing-byte-rate</td>
+      <td>The average number of outgoing bytes sent per second to all servers.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>request-rate</td>
+      <td>The average number of requests sent per second.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>request-size-avg</td>
+      <td>The average size of all requests in the window.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>request-size-max</td>
+      <td>The maximum size of any request sent in the window.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>incoming-byte-rate</td>
+      <td>Bytes/second read off all sockets</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>response-rate</td>
+      <td>Responses received sent per second.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>select-rate</td>
+      <td>Number of times the I/O layer checked for new I/O to perform per second</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>io-wait-time-ns-avg</td>
+      <td>The average length of time the I/O thread spent waiting for a socket ready
for reads or writes in nanoseconds.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>io-wait-ratio</td>
+      <td>The fraction of time the I/O thread spent waiting.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>io-time-ns-avg</td>
+      <td>The average length of time for I/O per select call in nanoseconds.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>io-ratio</td>
+      <td>The fraction of time the I/O thread spent doing I/O</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>connection-count</td>
+      <td>The current number of active connections.</td>
+      <td>kafka.producer:type=producer-metrics,client-id=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>outgoing-byte-rate</td>
+      <td>The average number of outgoing bytes sent per second for a node.</td>
+      <td>kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)</td>
+    </tr>
+    <tr>
+      <td>request-rate</td>
+      <td>The average number of requests sent per second for a node.</td>
+      <td>kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)</td>
+    </tr>
+    <tr>
+      <td>request-size-avg</td>
+      <td>The average size of all requests in the window for a node.</td>
+      <td>kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)</td>
+    </tr>
+    <tr>
+      <td>request-size-max</td>
+      <td>The maximum size of any request sent in the window for a node.</td>
+      <td>kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)</td>
+    </tr>
+    <tr>
+      <td>incoming-byte-rate</td>
+      <td>The average number of responses received per second for a node.</td>
+      <td>kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)</td>
+    </tr>
+    <tr>
+      <td>request-latency-avg</td>
+      <td>The average request latency in ms for a node.</td>
+      <td>kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)</td>
+    </tr>
+    <tr>
+      <td>request-latency-max</td>
+      <td>The maximum request latency in ms for a node.</td>
+      <td>kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)</td>
+    </tr>
+    <tr>
+      <td>response-rate</td>
+      <td>Responses received sent per second for a node.</td>
+      <td>kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)</td>
+    </tr>
+    <tr>
+      <td>record-send-rate</td>
+      <td>The average number of records sent per second for a topic.</td>
+      <td>kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>byte-rate</td>
+      <td>The average number of bytes sent per second for a topic.</td>
+      <td>kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>compression-rate</td>
+      <td>The average compression rate of record batches for a topic.</td>
+      <td>kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-retry-rate</td>
+      <td>The average per-second number of retried record sends for a topic</td>
+      <td>kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)</td>
+    </tr>
+    <tr>
+      <td>record-error-rate</td>
+      <td>The average per-second number of record sends that resulted in errors for
a topic.</td>
+      <td>kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)</td>
+    </tr>
+</tbody></table>
+
 We recommend monitor GC time and other stats and various server stats such as CPU utilization,
I/O service time, etc.
 
 On the client side, we recommend monitor the message/byte rate (global and per topic), request
rate/size/time, and on the consumer side, max lag in messages among all partitions and min
fetch request rate. For a consumer to keep up, max lag needs to be less than a threshold and
min fetch rate needs to be larger than 0.



Mime
View raw message