kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guozh...@apache.org
Subject kafka git commit: MINOR: Stream metrics documentation
Date Thu, 16 Feb 2017 00:32:05 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk 2c91b324d -> 2e1b00f26


MINOR: Stream metrics documentation

Author: Eno Thereska <eno@confluent.io>

Reviewers: Matthias J. Sax, Guozhang Wang

Closes #2542 from enothereska/minor-streams-metrics


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/2e1b00f2
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/2e1b00f2
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/2e1b00f2

Branch: refs/heads/trunk
Commit: 2e1b00f262947c4c3211df205df259ef517763f4
Parents: 2c91b32
Author: Eno Thereska <eno@confluent.io>
Authored: Wed Feb 15 16:32:02 2017 -0800
Committer: Guozhang Wang <wangguoz@gmail.com>
Committed: Wed Feb 15 16:32:02 2017 -0800

----------------------------------------------------------------------
 docs/ops.html | 130 ++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 124 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/2e1b00f2/docs/ops.html
----------------------------------------------------------------------
diff --git a/docs/ops.html b/docs/ops.html
index a3423a7..9232f65 100644
--- a/docs/ops.html
+++ b/docs/ops.html
@@ -14,8 +14,8 @@
  See the License for the specific language governing permissions and
  limitations under the License.
 -->
-
 <script id="ops-template" type="text/x-handlebars-template">
+  
   Here is some information on actually running Kafka as a production system based on usage
and experience at LinkedIn. Please send us any additional tips you know of.
 
   <h3><a id="basic_ops" href="#basic_ops">6.1 Basic Kafka Operations</a></h3>
@@ -842,9 +842,9 @@
       </tr>
   </tbody></table>
 
-  <h4><a id="selector_monitoring" href="#selector_monitoring">Common monitoring
metrics for producer/consumer/connect</a></h4>
+  <h4><a id="selector_monitoring" href="#selector_monitoring">Common monitoring
metrics for producer/consumer/connect/streams</a></h4>
 
-  The following metrics are available on producer/consumer/connector instances.  For specific
metrics, please see following sections.
+  The following metrics are available on producer/consumer/connector/streams instances. 
For specific metrics, please see following sections.
 
   <table class="data-table">
     <tbody>
@@ -931,9 +931,9 @@
     </tbody>
   </table>
 
-  <h4><a id="common_node_monitoring" href="#common_node_monitoring">Common Per-broker
metrics for producer/consumer/connect</a></h4>
+  <h4><a id="common_node_monitoring" href="#common_node_monitoring">Common Per-broker
metrics for producer/consumer/connect/streams</a></h4>
 
-  The following metrics are available on producer/consumer/connector instances.  For specific
metrics, please see following sections.
+  The following metrics are available on producer/consumer/connector/streams instances. 
For specific metrics, please see following sections.
 
   <table class="data-table">
     <tbody>
@@ -1314,7 +1314,125 @@
     </tbody>
   </table>
 
-  <h5><a id="others_monitoring" href="#others_monitoring">Others</a></h5>
+
+
+  <h4><a id="kafka_streams_monitoring" href="#kafka_streams_monitoring">Streams
Monitoring</a></h4>
+
+  A Kafka Streams instance contains all the producer and consumer metrics as well as additional
metrics specific to streams. By default Kafka Streams has metrics with two recording levels:
debug and info. The debug level records all metrics, while the info level records only the
thread-level metrics.  Use the following configuration option to specify which metrics you
want collected:
+<pre>metrics.recording.level="info"</pre>
+
+<h5><a id="kafka_streams_thread_monitoring" href="#kafka_streams_thread_monitoring">Thread
Metrics</a></h5>
+All the following metrics have a recording level of ``info``:
+<table class="data-table">
+    <tbody>
+      <tr>
+        <th>Metric/Attribute name</th>
+        <th>Description</th>
+        <th>Mbean name</th>
+      </tr>
+      <tr>
+        <td>[commit | poll | process | punctuate]-latency-[avg | max]</td>
+        <td>The [average | maximum] execution time in ms, for the respective operation,
across all running tasks of this thread.</td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>[commit | poll | process | punctuate]-rate</td>
+        <td>The average number of respective operations per second across all tasks.</td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>task-created-rate</td>
+        <td>The average number of newly created tasks per second.</td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>task-closed-rate</td>
+        <td>The average number of tasks closed per second.</td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>skipped-records-rate</td>
+        <td>The average number of skipped records per second. </td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+ </tbody>
+</table>
+
+<h5><a id="kafka_streams_task_monitoring" href="#kafka_streams_task_monitoring">Task
Metrics</a></h5>
+All the following metrics have a recording level of ``debug``:
+ <table class="data-table">
+      <tbody>
+      <tr>
+        <th>Metric/Attribute name</th>
+        <th>Description</th>
+        <th>Mbean name</th>
+      </tr>
+      <tr>
+        <td>commit-latency-[avg | max]</td>
+        <td>The [average | maximum] commit time in ns for this task. </td>
+        <td>kafka.streams:type=stream-task-metrics,streams-task-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>commit-rate</td>
+        <td>The average number of commit calls per second. </td>
+        <td>kafka.streams:type=stream-task-metrics,streams-task-id=([-.\w]+)</td>
+      </tr>
+ </tbody>
+</table>
+
+ <h5><a id="kafka_streams_node_monitoring" href="#kafka_streams_node_monitoring">Processor
Node Metrics</a></h5>
+All the following metrics have a recording level of ``debug``:
+ <table class="data-table">
+      <tbody>
+      <tr>
+        <th>Metric/Attribute name</th>
+        <th>Description</th>
+        <th>Mbean name</th>
+      </tr>
+      <tr>
+        <td>[process | punctuate | create | destroy]-latency-[avg | max]</td>
+        <td>The [average | maximum] execution time in ns, for the respective operation.
</td>
+        <td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>[process | punctuate | create | destroy]-rate</td>
+        <td>The average number of respective operations per second. </td>
+        <td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>forward-rate</td>
+        <td>The average rate of records being forwarded downstream, from source nodes
only, per second. </td>
+        <td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
+      </tr>
+ </tbody>
+ </table>
+
+ <h5><a id="kafka_streams_store_monitoring" href="#kafka_streams_store_monitoring">State
Store Metrics</a></h5>
+All the following metrics have a recording level of ``debug``:
+
+ <table class="data-table">
+      <tbody>
+      <tr>
+        <th>Metric/Attribute name</th>
+        <th>Description</th>
+        <th>Mbean name</th>
+      </tr>
+      <tr>
+        <td>[put | put-if-absent | get | delete | put-all | all | range | flush | restore]-latency-[avg
| max]</td>
+        <td>The average execution time in ns, for the respective operation. </td>
+        <td>kafka.streams:type=stream-[store-type]-metrics</td>
+      </tr>
+        <tr>
+        <td>[put | put-if-absent | get | delete | put-all | all | range | flush | restore]-rate</td>
+        <td>The average rate of respective operations per second for this store.</td>
+        <td>kafka.streams:type=stream-[store-type]-metrics</td>
+      </tr>
+      
+    </tbody>
+</table>
+
+
+  <h4><a id="others_monitoring" href="#others_monitoring">Others</a></h4>
 
   We recommend monitoring GC time and other stats and various server stats such as CPU utilization,
I/O service time, etc.
 


Mime
View raw message