kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guozh...@apache.org
Subject kafka git commit: MINOR: Increase max.poll time for streams consumers
Date Thu, 30 Mar 2017 23:00:15 GMT
Repository: kafka
Updated Branches:
  refs/heads/0.10.2 e3c5f4122 -> 76f8028d9


MINOR: Increase max.poll time for streams consumers

Author: Eno Thereska <eno@confluent.io>

Reviewers: Damian Guy, Matthias J. Sax, Guozhang Wang

Closes #2770 from enothereska/minor-increase-max-poll

(cherry picked from commit 5f88cf79fb51996e77614aafdfbb15e4989e159b)
Signed-off-by: Guozhang Wang <wangguoz@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/76f8028d
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/76f8028d
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/76f8028d

Branch: refs/heads/0.10.2
Commit: 76f8028d9d6473e6ec084e4258bf0e2d84f8a428
Parents: e3c5f41
Author: Eno Thereska <eno@confluent.io>
Authored: Thu Mar 30 15:58:24 2017 -0700
Committer: Guozhang Wang <wangguoz@gmail.com>
Committed: Thu Mar 30 15:59:55 2017 -0700

----------------------------------------------------------------------
 .../src/main/java/org/apache/kafka/streams/StreamsConfig.java | 7 +++++++
 1 file changed, 7 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/76f8028d/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
----------------------------------------------------------------------
diff --git a/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java b/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
index 57db027..3d22742 100644
--- a/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
+++ b/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
@@ -404,6 +404,13 @@ public class StreamsConfig extends AbstractConfig {
         tempConsumerDefaultOverrides.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, "1000");
         tempConsumerDefaultOverrides.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
         tempConsumerDefaultOverrides.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false");
+        // MAX_POLL_INTERVAL_MS_CONFIG needs to be large for streams to handle cases when
+        // streams is recovering data from state stores. We may set it to Integer.MAX_VALUE
since
+        // the streams code itself catches most exceptions and acts accordingly without needing
+        // this timeout. Note however that deadlocks are not detected (by definition) so
we
+        // are losing the ability to detect them by setting this value to large. Hopefully
+        // deadlocks happen very rarely or never.
+        tempConsumerDefaultOverrides.put(ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG, Integer.toString(Integer.MAX_VALUE));
 
         CONSUMER_DEFAULT_OVERRIDES = Collections.unmodifiableMap(tempConsumerDefaultOverrides);
     }


Mime
View raw message