kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ij...@apache.org
Subject kafka git commit: MINOR: document increased network bandwidth of 0.10 under replication
Date Mon, 16 May 2016 12:40:21 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk 7ded19a29 -> 62985f313


MINOR: document increased network bandwidth of 0.10 under replication

If you're pushing close to the network capacity, 0.10's additional 8 bytes per message can
lead to overload of your network. We (Heroku Kafka) ran into this issue whilst benchmarking
0.10 RC and the ijuma suggested it belonged in the update note.

Comments/suggestions welcome.

Author: Tom Crayford <tcrayford@googlemail.com>

Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #1389 from tcrayford/upgrade_note_about_increased_network_bandwidth


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/62985f31
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/62985f31
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/62985f31

Branch: refs/heads/trunk
Commit: 62985f313f4b66d3810b05650d2a3f42b6cb054e
Parents: 7ded19a
Author: Tom Crayford <tcrayford@googlemail.com>
Authored: Mon May 16 13:39:33 2016 +0100
Committer: Ismael Juma <ismael@juma.me.uk>
Committed: Mon May 16 13:39:33 2016 +0100

----------------------------------------------------------------------
 docs/upgrade.html | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/62985f31/docs/upgrade.html
----------------------------------------------------------------------
diff --git a/docs/upgrade.html b/docs/upgrade.html
index c5431aa..2972f26 100644
--- a/docs/upgrade.html
+++ b/docs/upgrade.html
@@ -65,7 +65,10 @@ work with 0.10.0.x brokers. Therefore, 0.9.0.0 clients should be upgraded
to 0.9
 </p>
 <p>
     <b>Note:</b> Due to the additional timestamp introduced in each message,
producers sending small messages may see a
-    message throughput degradation because of the increased overhead. When receiving compressed
messages, 0.10.0
+    message throughput degradation because of the increased overhead.
+    Likewise, replication now transmits an additional 8 bytes per message.
+    If you're running close to the network capacity of your cluster, it's possible that you'll
overwhelm the network cards
+    and see failures and performance issues due to the overload. When receiving compressed
messages, 0.10.0
     brokers avoid recompressing the messages, which in general reduces the latency and improves
the throughput. In
     certain cases, this may reduce the batching size on the producer, which could lead to
worse throughput. If this
     happens, users can tune linger.ms and batch.size of the producer for better throughput.


Mime
View raw message