kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guozh...@apache.org
Subject [kafka] branch trunk updated: KAFKA-7223: document suppression buffer metrics (#6024)
Date Thu, 13 Dec 2018 02:23:06 GMT
This is an automated email from the ASF dual-hosted git repository.

guozhang pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/trunk by this push:
     new fdd33bc  KAFKA-7223: document suppression buffer metrics (#6024)
fdd33bc is described below

commit fdd33bcab0b727d87ceb8f242de67d4452ead6e1
Author: John Roesler <vvcephei@users.noreply.github.com>
AuthorDate: Wed Dec 12 20:22:54 2018 -0600

    KAFKA-7223: document suppression buffer metrics (#6024)
    
    Document the new metrics added in #5795
    
    Reviewers: Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
---
 docs/ops.html | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 67 insertions(+), 7 deletions(-)

diff --git a/docs/ops.html b/docs/ops.html
index f487911..64f7159 100644
--- a/docs/ops.html
+++ b/docs/ops.html
@@ -1441,7 +1441,7 @@
 <pre>metrics.recording.level="info"</pre>
 
 <h5><a id="kafka_streams_thread_monitoring" href="#kafka_streams_thread_monitoring">Thread
Metrics</a></h5>
-All the following metrics have a recording level of ``info``:
+All the following metrics have a recording level of <code>info</code>:
 <table class="data-table">
     <tbody>
       <tr>
@@ -1563,7 +1563,7 @@ All the following metrics have a recording level of ``info``:
 </table>
 
 <h5><a id="kafka_streams_task_monitoring" href="#kafka_streams_task_monitoring">Task
Metrics</a></h5>
-All the following metrics have a recording level of ``debug``:
+All the following metrics have a recording level of <code>debug</code>:
  <table class="data-table">
       <tbody>
       <tr>
@@ -1605,7 +1605,7 @@ All the following metrics have a recording level of ``debug``:
 </table>
 
  <h5><a id="kafka_streams_node_monitoring" href="#kafka_streams_node_monitoring">Processor
Node Metrics</a></h5>
- All the following metrics have a recording level of ``debug``:
+ All the following metrics have a recording level of <code>debug</code>:
  <table class="data-table">
       <tbody>
       <tr>
@@ -1703,13 +1703,35 @@ All the following metrics have a recording level of ``debug``:
         <td>The total number of of records being forwarded downstream, from source
nodes only. </td>
         <td>kafka.streams:type=stream-processor-node-metrics,client-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)</td>
       </tr>
+      <tr>
+        <td>suppression-emit-rate</td>
+        <td>
+          The rate at which records that have been emitted downstream from suppression operation
nodes.
+          Compare with the <code>process-rate</code> metric to determine how
many updates are being suppressed.
+        </td>
+        <td>kafka.streams:type=stream-processor-node-metrics,client-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>suppression-emit-total</td>
+        <td>
+          The total number of records that have been emitted downstream from suppression
operation nodes.
+          Compare with the <code>process-total</code> metric to determine how
many updates are being suppressed.
+        </td>
+        <td>kafka.streams:type=stream-processor-node-metrics,client-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)</td>
+      </tr>
  </tbody>
  </table>
 
  <h5><a id="kafka_streams_store_monitoring" href="#kafka_streams_store_monitoring">State
Store Metrics</a></h5>
- All the following metrics have a recording level of ``debug``. Note that the ``store-scope``
value is specified in <code>StoreSupplier#metricsScope()</code> for user's customized
- state stores; for built-in state stores, currently we have <code>in-memory-state</code>,
<code>in-memory-lru-state</code>, <code>rocksdb-state</code> (for
RocksDB backed key-value store),
-  <code>rocksdb-window-state</code> (for RocksDB backed window store) and <code>rocksdb-session-state</code>
(for RocksDB backed session store).
+ All the following metrics have a recording level of <code>debug</code>. Note
that the <code>store-scope</code> value is specified in <code>StoreSupplier#metricsScope()</code>
for user's customized
+ state stores; for built-in state stores, currently we have:
+  <ul>
+    <li><code>in-memory-state</code></li>
+    <li><code>in-memory-lru-state</code></li>
+    <li><code>rocksdb-state</code> (for RocksDB backed key-value store)</li>
+    <li><code>rocksdb-window-state</code> (for RocksDB backed window store)</li>
+    <li><code>rocksdb-session-state</code> (for RocksDB backed session
store)</li>
+  </ul>
 
   <table class="data-table">
       <tbody>
@@ -1902,7 +1924,7 @@ All the following metrics have a recording level of ``debug``:
  </table>
 
   <h5><a id="kafka_streams_cache_monitoring" href="#kafka_streams_cache_monitoring">Record
Cache Metrics</a></h5>
-  All the following metrics have a recording level of ``debug``:
+  All the following metrics have a recording level of <code>debug</code>:
 
   <table class="data-table">
       <tbody>
@@ -1929,6 +1951,44 @@ All the following metrics have a recording level of ``debug``:
     </tbody>
  </table>
 
+  <h5><a id="kafka_streams_buffer_monitoring" href="#kafka_streams_buffer_monitoring">Suppression
Buffer Metrics</a></h5>
+  All the following metrics have a recording level of <code>debug</code>:
+
+  <table class="data-table">
+      <tbody>
+      <tr>
+        <td>suppression-buffer-size-current</td>
+        <td>The current total size, in bytes, of the buffered data.</td>
+        <td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>suppression-buffer-size-avg</td>
+        <td>The average total size, in bytes, of the buffered data over the sampling
window.</td>
+        <td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>suppression-buffer-size-max</td>
+        <td>The maximum total size, in bytes, of the buffered data over the sampling
window.</td>
+        <td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>suppression-buffer-count-current</td>
+        <td>The current number of records buffered.</td>
+        <td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>suppression-buffer-size-avg</td>
+        <td>The average number of records buffered over the sampling window.</td>
+        <td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>suppression-buffer-size-max</td>
+        <td>The maximum number of records buffered over the sampling window.</td>
+        <td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+      </tr>
+    </tbody>
+ </table>
+
   <h4><a id="others_monitoring" href="#others_monitoring">Others</a></h4>
 
   We recommend monitoring GC time and other stats and various server stats such as CPU utilization,
I/O service time, etc.


Mime
View raw message