kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gwens...@apache.org
Subject kafka git commit: KAFKA-3219: Fix long topic name validation
Date Tue, 22 Mar 2016 20:10:23 GMT
Repository: kafka
Updated Branches:
  refs/heads/0.10.0 687d2494f -> 05984de69


KAFKA-3219: Fix long topic name validation

This fixes an issue with long topic names by considering, during topic
validation, the '-' and the partition id that is appended to the log
folder created for each topic partition.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Gwen Shapira, Grant Henke

Closes #898 from vahidhashemian/KAFKA-3219

(cherry picked from commit ad3dfc6ab25c3f80d2425e24e72ae732b850dc60)
Signed-off-by: Gwen Shapira <cshapi@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/05984de6
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/05984de6
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/05984de6

Branch: refs/heads/0.10.0
Commit: 05984de69ee8eb06f214a2e42abd789a4fbc8cef
Parents: 687d249
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Authored: Tue Mar 22 13:10:07 2016 -0700
Committer: Gwen Shapira <cshapi@gmail.com>
Committed: Tue Mar 22 13:10:19 2016 -0700

----------------------------------------------------------------------
 core/src/main/scala/kafka/common/Topic.scala          | 2 +-
 core/src/test/scala/unit/kafka/common/TopicTest.scala | 3 ++-
 docs/ops.html                                         | 2 ++
 3 files changed, 5 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/05984de6/core/src/main/scala/kafka/common/Topic.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/kafka/common/Topic.scala b/core/src/main/scala/kafka/common/Topic.scala
index 930d0e4..6067712 100644
--- a/core/src/main/scala/kafka/common/Topic.scala
+++ b/core/src/main/scala/kafka/common/Topic.scala
@@ -22,7 +22,7 @@ import kafka.coordinator.GroupCoordinator
 
 object Topic {
   val legalChars = "[a-zA-Z0-9\\._\\-]"
-  private val maxNameLength = 255
+  private val maxNameLength = 249
   private val rgx = new Regex(legalChars + "+")
 
   def validate(topic: String) {

http://git-wip-us.apache.org/repos/asf/kafka/blob/05984de6/core/src/test/scala/unit/kafka/common/TopicTest.scala
----------------------------------------------------------------------
diff --git a/core/src/test/scala/unit/kafka/common/TopicTest.scala b/core/src/test/scala/unit/kafka/common/TopicTest.scala
index da6083a..66549af 100644
--- a/core/src/test/scala/unit/kafka/common/TopicTest.scala
+++ b/core/src/test/scala/unit/kafka/common/TopicTest.scala
@@ -31,6 +31,7 @@ class TopicTest {
     for (i <- 1 to 6)
       longName += longName
     invalidTopicNames += longName
+    invalidTopicNames += longName.drop(6)
     val badChars = Array('/', '\\', ',', '\u0000', ':', "\"", '\'', ';', '*', '?', ' ', '\t',
'\r', '\n', '=')
     for (weirdChar <- badChars) {
       invalidTopicNames += "Is" + weirdChar + "illegal"
@@ -47,7 +48,7 @@ class TopicTest {
     }
 
     val validTopicNames = new ArrayBuffer[String]()
-    validTopicNames += ("valid", "TOPIC", "nAmEs", "ar6", "VaL1d", "_0-9_.")
+    validTopicNames += ("valid", "TOPIC", "nAmEs", "ar6", "VaL1d", "_0-9_.", longName.drop(7))
     for (i <- 0 until validTopicNames.size) {
       try {
         Topic.validate(validTopicNames(i))

http://git-wip-us.apache.org/repos/asf/kafka/blob/05984de6/docs/ops.html
----------------------------------------------------------------------
diff --git a/docs/ops.html b/docs/ops.html
index 4cfe17b..541a01d 100644
--- a/docs/ops.html
+++ b/docs/ops.html
@@ -34,6 +34,8 @@ The replication factor controls how many servers will replicate each message
tha
 <p>
 The partition count controls how many logs the topic will be sharded into. There are several
impacts of the partition count. First each partition must fit entirely on a single server.
So if you have 20 partitions the full data set (and read and write load) will be handled by
no more than 20 servers (no counting replicas). Finally the partition count impacts the maximum
parallelism of your consumers. This is discussed in greater detail in the <a href="#intro_consumers">concepts
section</a>.
 <p>
+Each sharded partition log is placed into its own folder under the Kafka log directory. The
name of such folders consists of the topic name, appended by a dash (-) and the partition
id. Since a typical folder name can not be over 255 characters long, there will be a limitation
on the length of topic names. We assume the number of partitions will not ever be above 100,000.
Therefore, topic names cannot be longer than 249 characters. This leaves just enough room
in the folder name for a dash and a potentially 5 digit long partition id.
+<p>
 The configurations added on the command line override the default settings the server has
for things like the length of time data should be retained. The complete set of per-topic
configurations is documented <a href="#topic-config">here</a>.
 
 <h4><a id="basic_ops_modify_topic" href="#basic_ops_modify_topic">Modifying topics</a></h4>


Mime
View raw message