kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ewe...@apache.org
Subject kafka git commit: KAFKA-3243: Fix Kafka basic ops documentation for Mirror maker, blacklist is not supported for new consumers
Date Fri, 26 Feb 2016 18:38:01 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk d51dd5fc7 -> 4542b027c


KAFKA-3243: Fix Kafka basic ops documentation for Mirror maker, blacklist is not supported
for new consumers

…list is not supported for new consumers

Author: Ashish Singh <asingh@cloudera.com>

Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #923 from SinghAsDev/KAFKA-3243


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/4542b027
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/4542b027
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/4542b027

Branch: refs/heads/trunk
Commit: 4542b027c1ac361cc8770383b66abac2129a05e7
Parents: d51dd5f
Author: Ashish Singh <asingh@cloudera.com>
Authored: Fri Feb 26 10:37:41 2016 -0800
Committer: Ewen Cheslack-Postava <me@ewencp.org>
Committed: Fri Feb 26 10:37:41 2016 -0800

----------------------------------------------------------------------
 .../main/scala/kafka/tools/MirrorMaker.scala    | 21 +++++++++++++++-----
 docs/ops.html                                   |  2 +-
 2 files changed, 17 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/4542b027/core/src/main/scala/kafka/tools/MirrorMaker.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/kafka/tools/MirrorMaker.scala b/core/src/main/scala/kafka/tools/MirrorMaker.scala
index 95b0aad..4bc38d9 100755
--- a/core/src/main/scala/kafka/tools/MirrorMaker.scala
+++ b/core/src/main/scala/kafka/tools/MirrorMaker.scala
@@ -173,9 +173,22 @@ object MirrorMaker extends Logging with KafkaMetricsGroup {
       }
 
       CommandLineUtils.checkRequiredArgs(parser, options, consumerConfigOpt, producerConfigOpt)
-      if (List(whitelistOpt, blacklistOpt).count(options.has) != 1) {
-        println("Exactly one of whitelist or blacklist is required.")
-        System.exit(1)
+
+      val useNewConsumer = options.has(useNewConsumerOpt)
+      if (useNewConsumer) {
+        if (options.has(blacklistOpt)) {
+          error("blacklist can not be used when using new consumer in mirror maker. Use whitelist
instead.")
+          System.exit(1)
+        }
+        if (!options.has(whitelistOpt)) {
+          error("whitelist must be specified when using new consumer in mirror maker.")
+          System.exit(1)
+        }
+      } else {
+        if (List(whitelistOpt, blacklistOpt).count(options.has) != 1) {
+          error("Exactly one of whitelist or blacklist is required.")
+          System.exit(1)
+        }
       }
 
       abortOnSendFailure = options.valueOf(abortOnSendFailureOpt).toBoolean
@@ -200,8 +213,6 @@ object MirrorMaker extends Logging with KafkaMetricsGroup {
       producerProps.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.ByteArraySerializer")
       producer = new MirrorMakerProducer(producerProps)
 
-      val useNewConsumer = options.has(useNewConsumerOpt)
-
       // Create consumers
       val mirrorMakerConsumers = if (!useNewConsumer) {
         val customRebalanceListener = {

http://git-wip-us.apache.org/repos/asf/kafka/blob/4542b027/docs/ops.html
----------------------------------------------------------------------
diff --git a/docs/ops.html b/docs/ops.html
index 33080e2..3cec8d6 100644
--- a/docs/ops.html
+++ b/docs/ops.html
@@ -118,7 +118,7 @@ Here is an example showing how to mirror a single topic (named <i>my-topic</i>)
 </pre>
 Note that we specify the list of topics with the <code>--whitelist</code> option.
This option allows any regular expression using <a href="http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html">Java-style
regular expressions</a>. So you could mirror two topics named <i>A</i> and
<i>B</i> using <code>--whitelist 'A|B'</code>. Or you could mirror
<i>all</i> topics using <code>--whitelist '*'</code>. Make sure to
quote any regular expression to ensure the shell doesn't try to expand it as a file path.
For convenience we allow the use of ',' instead of '|' to specify a list of topics.
 <p>
-Sometime it is easier to say what it is that you <i>don't</i> want. Instead of
using <code>--whitelist</code> to say what you want to mirror you can use <code>--blacklist</code>
to say what to exclude. This also takes a regular expression argument.
+Sometimes it is easier to say what it is that you <i>don't</i> want. Instead
of using <code>--whitelist</code> to say what you want to mirror you can use <code>--blacklist</code>
to say what to exclude. This also takes a regular expression argument. However, <code>--blacklist</code>
is not supported when using <code>--new.consumer</code>.
 <p>
 Combining mirroring with the configuration <code>auto.create.topics.enable=true</code>
makes it possible to have a replica cluster that will automatically create and replicate all
data in a source cluster even as new topics are added.
 


Mime
View raw message