kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ij...@apache.org
Subject kafka git commit: MINOR: document increased network bandwidth of 0.10 under replication
Date Mon, 16 May 2016 12:40:32 GMT
Repository: kafka
Updated Branches:
  refs/heads/0.10.0 5834a6a56 -> 7bfeade97

MINOR: document increased network bandwidth of 0.10 under replication

If you're pushing close to the network capacity, 0.10's additional 8 bytes per message can
lead to overload of your network. We (Heroku Kafka) ran into this issue whilst benchmarking
0.10 RC and the ijuma suggested it belonged in the update note.

Comments/suggestions welcome.

Author: Tom Crayford <tcrayford@googlemail.com>

Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #1389 from tcrayford/upgrade_note_about_increased_network_bandwidth

(cherry picked from commit 62985f313f4b66d3810b05650d2a3f42b6cb054e)
Signed-off-by: Ismael Juma <ismael@juma.me.uk>

Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/7bfeade9
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/7bfeade9
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/7bfeade9

Branch: refs/heads/0.10.0
Commit: 7bfeade97a120690ca179c71ea715e21ea40f6ea
Parents: 5834a6a
Author: Tom Crayford <tcrayford@googlemail.com>
Authored: Mon May 16 13:39:33 2016 +0100
Committer: Ismael Juma <ismael@juma.me.uk>
Committed: Mon May 16 13:40:26 2016 +0100

 docs/upgrade.html | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/docs/upgrade.html b/docs/upgrade.html
index c5431aa..2972f26 100644
--- a/docs/upgrade.html
+++ b/docs/upgrade.html
@@ -65,7 +65,10 @@ work with 0.10.0.x brokers. Therefore, clients should be upgraded
to 0.9
     <b>Note:</b> Due to the additional timestamp introduced in each message,
producers sending small messages may see a
-    message throughput degradation because of the increased overhead. When receiving compressed
messages, 0.10.0
+    message throughput degradation because of the increased overhead.
+    Likewise, replication now transmits an additional 8 bytes per message.
+    If you're running close to the network capacity of your cluster, it's possible that you'll
overwhelm the network cards
+    and see failures and performance issues due to the overload. When receiving compressed
messages, 0.10.0
     brokers avoid recompressing the messages, which in general reduces the latency and improves
the throughput. In
     certain cases, this may reduce the batching size on the producer, which could lead to
worse throughput. If this
     happens, users can tune linger.ms and batch.size of the producer for better throughput.

View raw message