kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gwens...@apache.org
Subject kafka git commit: KAFKA-2724: ZK Auth documentation.
Date Tue, 03 Nov 2015 17:42:37 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk 694e03c35 -> 840e95897


KAFKA-2724: ZK Auth documentation.

Author: Flavio Junqueira <fpj@apache.org>

Reviewers: Gwen Shapira

Closes #409 from fpj/KAFKA-2724


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/840e9589
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/840e9589
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/840e9589

Branch: refs/heads/trunk
Commit: 840e95897fdb5118aa1308ec1a3be27369b9d71e
Parents: 694e03c
Author: Flavio Junqueira <fpj@apache.org>
Authored: Tue Nov 3 09:42:20 2015 -0800
Committer: Gwen Shapira <cshapi@gmail.com>
Committed: Tue Nov 3 09:42:20 2015 -0800

----------------------------------------------------------------------
 docs/security.html | 43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/840e9589/docs/security.html
----------------------------------------------------------------------
diff --git a/docs/security.html b/docs/security.html
index 2776ad4..c2a9dfb 100644
--- a/docs/security.html
+++ b/docs/security.html
@@ -21,6 +21,7 @@ In release 0.9.0.0, the Kafka community added a number of features that,
used ei
     <li>Authenticating clients (Producers and consumers) connections to brokers, using
either SSL or SASL (Kerberos)</li>
     <li>Authorizing read / write operations by clients</li>
     <li>Encryption of data sent between brokers and clients, or between brokers, using
SSL</li>
+    <li>Authenticate brokers connecting to ZooKeeper</li>
     <li>Security is optional - non-secured clusters are supported, as well as a mix
of authenticated, unauthenticated, encrypted and non-encrypted clients.</li>
     <li>Authorization is pluggable and supports integration with external authorization
services</li>
 </ol>
@@ -72,7 +73,7 @@ The guides below explain how to configure and use the security features
in both
             <u>Important notes:</u>
             <ol>
                 <li>KafkaServer is a section name in JAAS file used by KafkaServer/Broker.
This section tells Kafka Server which principal to use and which keytab this principal is
stored. It allows Kafka Server to login using the keytab specified in this section.</li>
-                <li>Client section is used to authenticate a SASL connection with zookeeper.
It also allows a broker to set SASL ACL on zookeeper nodes which locks these nodes down so
that only kafka broker can modify. It is important to have the same principal name across
all the brokers.</li>
+                <li>Client section is used to authenticate a SASL connection with zookeeper.
It also allows a broker to set SASL ACL on zookeeper nodes which locks these nodes down so
that only kafka broker can modify. It is necessary to have the same principal name across
all the brokers. If you want to use a section name other than Client, then you need to set
the system property <tt>zookeeper.sasl.client</tt> to the appropriate name (<i>e.g.</i>,
<tt>-Dzookeeper.sasl.client=ZkClient</tt>).</li>
                 <li>KafkaClient section here describes how the clients like producer
and consumer can connect to the Kafka Broker. Here we specified "useTicketCache=true" not
a keytab this allows user to do kinit and run a kafka-console-consumer or kafka-console-producer
to connect to broker. For a long running process one should create KafkaClient section similar
to KafkaServer.</li>
                 <li>In KafkaServer and KafkaClient sections we've "serviceName" this
should match principal name with which kafka broker is running. In the above example principal="kafka/kafka1.hostname.com@DOMAIN.com"
so we've "kafka" which is matching the principalName.</li>
             </ol>
@@ -114,4 +115,42 @@ The guides below explain how to configure and use the security features
in both
         </ol></li>
 </ol>
 
-<h3><a id="security_authz">7.4 Authorization and ACLs</a></h3>
\ No newline at end of file
+<h3><a id="security_authz">7.4 Authorization and ACLs</a></h3>
+<h3><a id="zk_authz">7.5 ZooKeeper Authentication</a></h3>
+<h4><a id="zk_authz_new">7.5.1 New clusters</a></h4>
+To enable ZooKeeper authentication on brokers, there are two necessary steps:
+<ol>
+	<li> Create a JAAS login file and set the appropriate system property to point to
it as described above</li>
+	<li> Set the configuration property <tt>zookeeper.set.acl</tt> in each
broker to true</li>
+</ol>
+
+The metadata stored in ZooKeeper is such that only brokers will be able to modify the corresponding
znodes, but znodes are world readable. The rationale behind this decision is that the data
stored in ZooKeeper is not sensitive, but inappropriate manipulation of znodes can cause cluster
disruption.
+
+<h4><a id="zk_authz_migration">7.5.2 Migrating clusters</a></h4>
+If you are running a version of Kafka that does not support security of simply with security
disabled, and you want to make the cluster secure, then you need to execute the following
steps to enable ZooKeeper authentication with minimal disruption to your operations:
+<ol>
+	<li>Perform a rolling restart setting the JAAS login file, which enables brokers to
authenticate. At the end of the rolling restart, brokers are able to manipulate znodes with
strict ACLs, but they will not create znodes with those ACLs</li>
+	<li>Perform a second rolling restart of brokers, this time setting the configuration
parameter <tt>zookeeper.set.acl</tt> to true, which enables the use of secure
ACLs when creating znodes</li>
+	<li>Execute the ZkSecurityMigrator tool. To execute the tool, there is this script:
<tt>./bin/zookeeper-security-migration.sh</tt> with <tt>zookeeper.acl</tt>
set to secure. This tool traverses the corresponding sub-trees changing the ACLs of the znodes</li>
+</ol>
+<p>It is also possible to turn off authentication in a secure cluster. To do it, follow
these steps:</p>
+<ol>
+	<li>Perform a rolling restart of brokers setting the JAAS login file, which enables
brokers to authenticate, but setting <tt>zookeeper.set.acl</tt> to false. At the
end of the rolling restart, brokers stop creating znodes with secure ACLs, but are still able
to authenticate and manipulate all znodes</li>
+	<li>Execute the ZkSecurityMigrator tool. To execute the tool, run this script <tt>./bin/zookeeper-security-migration.sh</tt>
with <tt>zookeeper.acl</tt> set to unsecure. This tool traverses the corresponding
sub-trees changing the ACLs of the znodes</li>
+	<li>Perform a second rolling restart of brokers, this time omitting the system property
that sets the JAAS login file</li>
+</ol>
+Here is an example of how to run the migration tool:
+<pre>
+./bin/zookeeper-security-migration --zookeeper.acl=secure --zookeeper.connection=localhost:2181
+</pre>
+<p>Run this to see the full list of parameters:</p>
+<pre>
+./bin/zookeeper-security-migration --help
+</pre>
+<h4><a id="zk_authz_new">7.5.3 Migrating the ZooKeeper ensemble</a></h4>
+It is also necessary to enable authentication on the ZooKeeper ensemble. To do it, we need
to perform a rolling restart of the server and set a few properties. Please refer to the ZooKeeper
documentation for more detail:
+<ol>
+	<li><a href="http://zookeeper.apache.org/doc/r3.4.6/zookeeperProgrammers.html#sc_ZooKeeperAccessControl">Apache
ZooKeeper documentation</a></li>
+	<li><a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zookeeper+and+SASL">Apache
ZooKeeper wiki</a></li>
+	<li><a href="http://www.cloudera.com/content/www/en-us/documentation/cdh/5-1-x/CDH5-Security-Guide/cdh5sg_zookeeper_security.html">Cloudera
ZooKeeper security configuration</a></li>
+</ol>
\ No newline at end of file


Mime
View raw message