kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ij...@apache.org
Subject kafka git commit: KAFKA-3393; Updated the docs to reflect the deprecation of block.on.buffer.full and usage of max.block.ms
Date Mon, 23 May 2016 21:20:56 GMT
Repository: kafka
Updated Branches:
  refs/heads/0.10.0 a1838755a -> 23b70efa8


KAFKA-3393; Updated the docs to reflect the deprecation of block.on.buffer.full and usage
of max.block.ms

Author: MayureshGharat <gharatmayuresh15@gmail.com>

Reviewers: Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #1060 from MayureshGharat/KAFKA-3393


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/23b70efa
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/23b70efa
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/23b70efa

Branch: refs/heads/0.10.0
Commit: 23b70efa8d65ef56f7a31e22afcbe653dc736079
Parents: a183875
Author: MayureshGharat <gharatmayuresh15@gmail.com>
Authored: Wed May 18 01:49:54 2016 +0100
Committer: Ismael Juma <ismael@juma.me.uk>
Committed: Mon May 23 22:20:22 2016 +0100

----------------------------------------------------------------------
 .../org/apache/kafka/clients/producer/KafkaProducer.java |  6 +++---
 .../apache/kafka/clients/producer/ProducerConfig.java    | 11 +++++------
 docs/upgrade.html                                        |  1 +
 3 files changed, 9 insertions(+), 9 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/23b70efa/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java
----------------------------------------------------------------------
diff --git a/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java b/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java
index 5b05272..fd3eb09 100644
--- a/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java
+++ b/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java
@@ -119,8 +119,8 @@ import org.slf4j.LoggerFactory;
  * <p>
  * The <code>buffer.memory</code> controls the total amount of memory available
to the producer for buffering. If records
  * are sent faster than they can be transmitted to the server then this buffer space will
be exhausted. When the buffer space is
- * exhausted additional send calls will block. For uses where you want to avoid any blocking
you can set <code>block.on.buffer.full=false</code> which
- * will cause the send call to result in an exception.
+ * exhausted additional send calls will block. The threshold for time to block is determined
by <code>max.block.ms</code> after which it throws
+ * a TimeoutException.
  * <p>
  * The <code>key.serializer</code> and <code>value.serializer</code>
instruct how to turn the key and value objects the user provides with
  * their <code>ProducerRecord</code> into bytes. You can use the included {@link
org.apache.kafka.common.serialization.ByteArraySerializer} or
@@ -420,7 +420,7 @@ public class KafkaProducer<K, V> implements Producer<K, V>
{
      *
      * @throws InterruptException If the thread is interrupted while blocked
      * @throws SerializationException If the key or value are not valid objects given the
configured serializers
-     * @throws BufferExhaustedException If <code>block.on.buffer.full=false</code>
and the buffer is full.
+     * @throws TimeoutException if the time taken for fetching metadata or allocating memory
for the record has surpassed <code>max.block.ms</code>.
      *
      */
     @Override

http://git-wip-us.apache.org/repos/asf/kafka/blob/23b70efa/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java
----------------------------------------------------------------------
diff --git a/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java b/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java
index 5b7a296..4ed083b 100644
--- a/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java
+++ b/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java
@@ -139,14 +139,14 @@ public class ProducerConfig extends AbstractConfig {
 
     /** <code>block.on.buffer.full</code> */
     /**
-     * @deprecated This config will be removed in a future release. Also, the {@link #METADATA_FETCH_TIMEOUT_CONFIG}
is no longer honored when this property is set to true.
+     * @deprecated This config will be removed in a future release. Please use {@link #MAX_BLOCK_MS_CONFIG}.
      */
     @Deprecated
     public static final String BLOCK_ON_BUFFER_FULL_CONFIG = "block.on.buffer.full";
     private static final String BLOCK_ON_BUFFER_FULL_DOC = "When our memory buffer is exhausted
we must either stop accepting new records (block) or throw errors. "
-                                                           + "By default this setting is
false and the producer will throw a BufferExhaustedException if a record is sent and the buffer
space is full. "
-                                                           + "However in some scenarios getting
an error is not desirable and it is better to block. Setting this to <code>true</code>
will accomplish that."
-                                                           + "<em>If this property
is set to true, parameter <code>" + METADATA_FETCH_TIMEOUT_CONFIG + "</code> is
not longer honored.</em>"
+                                                           + "By default this setting is
false and the producer will no longer throw a BufferExhaustException but instead will use
the {@link #MAX_BLOCK_MS_CONFIG} "
+                                                           + "value to block, after which
it will throw a TimeoutException. Setting this property to true will set the <code>"
+ MAX_BLOCK_MS_CONFIG + "</code> to Long.MAX_VALUE."
+                                                           + "<em>Also if this property
is set to true, parameter <code>" + METADATA_FETCH_TIMEOUT_CONFIG + "</code> is
not longer honored.</em>"
                                                            + "<p>"
                                                            + "This parameter is deprecated
and will be removed in a future release. "
                                                            + "Parameter <code>" + MAX_BLOCK_MS_CONFIG
+ "</code> should be used instead.";
@@ -154,8 +154,7 @@ public class ProducerConfig extends AbstractConfig {
     /** <code>buffer.memory</code> */
     public static final String BUFFER_MEMORY_CONFIG = "buffer.memory";
     private static final String BUFFER_MEMORY_DOC = "The total bytes of memory the producer
can use to buffer records waiting to be sent to the server. If records are "
-                                                    + "sent faster than they can be delivered
to the server the producer will either block or throw an exception based "
-                                                    + "on the preference specified by <code>"
+ BLOCK_ON_BUFFER_FULL_CONFIG + "</code>. "
+                                                    + "sent faster than they can be delivered
to the server the producer will block for <code>" + MAX_BLOCK_MS_CONFIG + "</code>
after which it will throw an exception."
                                                     + "<p>"
                                                     + "This setting should correspond roughly
to the total memory the producer will use, but is not a hard bound since "
                                                     + "not all memory the producer uses is
used for buffering. Some additional memory will be used for compression (if "

http://git-wip-us.apache.org/repos/asf/kafka/blob/23b70efa/docs/upgrade.html
----------------------------------------------------------------------
diff --git a/docs/upgrade.html b/docs/upgrade.html
index d09b9d7..dec0808 100644
--- a/docs/upgrade.html
+++ b/docs/upgrade.html
@@ -164,6 +164,7 @@ work with 0.10.0.x brokers. Therefore, 0.9.0.0 clients should be upgraded
to 0.9
     <li> Altering topic configuration from the kafka-topics.sh script (kafka.admin.TopicCommand)
has been deprecated. Going forward, please use the kafka-configs.sh script (kafka.admin.ConfigCommand)
for this functionality. </li>
     <li> The kafka-consumer-offset-checker.sh (kafka.tools.ConsumerOffsetChecker) has
been deprecated. Going forward, please use kafka-consumer-groups.sh (kafka.admin.ConsumerGroupCommand)
for this functionality. </li>
     <li> The kafka.tools.ProducerPerformance class has been deprecated. Going forward,
please use org.apache.kafka.tools.ProducerPerformance for this functionality (kafka-producer-perf-test.sh
will also be changed to use the new class). </li>
+    <li> The producer config block.on.buffer.full has been deprecated and will be removed
in future release. Currently its default value has been changed to false. The KafkaProducer
will no longer throw BufferExhaustedException but instead will use max.block.ms value to block,
after which it will throw a TimeoutException. If block.on.buffer.full property is set to true
explicitly, it will set the max.block.ms to Long.MAX_VALUE and metadata.fetch.timeout.ms will
not be honoured</li>
 </ul>
 
 <h4><a id="upgrade_82" href="#upgrade_82">Upgrading from 0.8.1 to 0.8.2</a></h4>


Mime
View raw message