kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jun...@apache.org
Subject kafka git commit: KAFKA-3517; Add documentation for SASL/PLAIN
Date Tue, 03 May 2016 16:49:39 GMT
Repository: kafka
Updated Branches:
  refs/heads/0.10.0 3cfe73a2c -> 2f39fb0d1


KAFKA-3517; Add documentation for SASL/PLAIN

Documentation corresponding to KIP-43 - SASL/PLAIN and multiple mechanism support.

Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Magnus Edenhill <apache_m@edenhill.se>, Jun Rao <junrao@gmail.com>

Closes #1232 from rajinisivaram/KAFKA-3517

(cherry picked from commit abac51dc0bf0d5bdc1d5414787da3191e00adcab)
Signed-off-by: Jun Rao <junrao@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/2f39fb0d
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/2f39fb0d
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/2f39fb0d

Branch: refs/heads/0.10.0
Commit: 2f39fb0d151301018339c8e6dfe61507f3f85b1b
Parents: 3cfe73a
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Authored: Tue May 3 09:49:23 2016 -0700
Committer: Jun Rao <junrao@gmail.com>
Committed: Tue May 3 09:49:34 2016 -0700

----------------------------------------------------------------------
 docs/protocol.html |  15 ++
 docs/security.html | 372 ++++++++++++++++++++++++++++++++++++------------
 2 files changed, 293 insertions(+), 94 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/2f39fb0d/docs/protocol.html
----------------------------------------------------------------------
diff --git a/docs/protocol.html b/docs/protocol.html
index cb359f1..c26f16b 100644
--- a/docs/protocol.html
+++ b/docs/protocol.html
@@ -114,6 +114,21 @@
 
 <p>Currently all versions are baselined at 0, as we evolve these APIs we will indicate
the format for each version individually.</p>
 
+<h5><a id="sasl_handshake" href="#sasl_handshake">SASL Authentication Sequence</a></h5>
+<p>The following sequence is used for SASL authentication:
+<ol>
+  <li>Kafka <code>ApiVersionsRequest</code> may be sent by the client to
obtain the version ranges of requests supported by the broker. This is optional.</li>
+  <li>Kafka <code>SaslHandshakeRequest</code> containing the SASL mechanism
for authentication is sent by the client. If the requested mechanism is not enabled
+    in the server, the server responds with the list of supported mechanisms and closes the
client connection. If the mechanism is enabled
+    in the server, the server sends a successful response and continues with SASL authentication.
+  <li>The actual SASL authentication is now performed. A series of SASL client and
server tokens corresponding to the mechanism are sent as opaque
+    packets. These packets contain a 32-bit size followed by the token as defined by the
protocol for the SASL mechanism.
+  <li>If authentication succeeds, subsequent packets are handled as Kafka API requests.
Otherwise, the client connection is closed.
+</ol>
+<p>For interoperability with 0.9.0.x clients, the first packet received by the server
is handled as a SASL/GSSAPI client token if it is not a valid
+Kafka request. SASL/GSSAPI authentication is performed starting with this packet, skipping
the first two steps above.</p>
+
+
 <h4><a id="protocol_details" href="#protocol_details">The Protocol</a></h4>
 
 <h5><a id="protocol_types" href="#protocol_types">Protocol Primitive Types</a></h5>

http://git-wip-us.apache.org/repos/asf/kafka/blob/2f39fb0d/docs/security.html
----------------------------------------------------------------------
diff --git a/docs/security.html b/docs/security.html
index a2e7816..3e5085b 100644
--- a/docs/security.html
+++ b/docs/security.html
@@ -18,7 +18,8 @@
 <h3><a id="security_overview" href="#security_overview">7.1 Security Overview</a></h3>
 In release 0.9.0.0, the Kafka community added a number of features that, used either separately
or together, increases security in a Kafka cluster. These features are considered to be of
beta quality. The following security measures are currently supported:
 <ol>
-    <li>Authentication of connections to brokers from clients (producers and consumers),
other brokers and tools, using either SSL or SASL (Kerberos)</li>
+    <li>Authentication of connections to brokers from clients (producers and consumers),
other brokers and tools, using either SSL or SASL (Kerberos).
+    SASL/PLAIN can also be used from release 0.10.0.0 onwards.</li>
     <li>Authentication of connections from brokers to ZooKeeper</li>
     <li>Encryption of data transferred between brokers and clients, between brokers,
or between brokers and tools using SSL (Note that there is a performance degradation when
SSL is enabled, the magnitude of which depends on the CPU type and the JVM implementation.)</li>
     <li>Authorization of read / write operations by clients</li>
@@ -182,22 +183,95 @@ Apache Kafka allows clients to connect over SSL. By default SSL is disabled
but
 <h3><a id="security_sasl" href="#security_sasl">7.3 Authentication using SASL</a></h3>
 
 <ol>
-    <li><h4><a id="security_sasl_prereq" href="#security_sasl_prereq">Prerequisites</a></h4>
+  <li><h4><a id="security_sasl_brokerconfig"
+    href="#security_sasl_brokerconfig">SASL configuration for Kafka brokers</a></h4>
     <ol>
-        <li><b>Kerberos</b><br>
-        If your organization is already using a Kerberos server (for example, by using Active
Directory), there is no need to install a new server just for Kafka. Otherwise you will need
to install one, your Linux vendor likely has packages for Kerberos and a short guide on how
to install and configure it (<a href="https://help.ubuntu.com/community/Kerberos">Ubuntu</a>,
<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/installing-kerberos.html">Redhat</a>).
Note that if you are using Oracle Java, you will need to download JCE policy files for your
Java version and copy them to $JAVA_HOME/jre/lib/security.</li>
-        <li><b>Create Kerberos Principals</b><br>
-        If you are using the organization's Kerberos or Active Directory server, ask your
Kerberos administrator for a principal for each Kafka broker in your cluster and for every
operating system user that will access Kafka with Kerberos authentication (via clients and
tools).</br>
-        If you have installed your own Kerberos, you will need to create these principals
yourself using the following commands:
+      <li>Select one or more supported mechanisms to enable in the broker. <tt>GSSAPI</tt>
+        and <tt>PLAIN</tt> are the mechanisms currently supported in Kafka.</li>
+      <li>Add a JAAS config file for the selected mechanisms as described in the examples
+        for setting up <a href="#security_sasl_kerberos_brokerconfig">GSSAPI (Kerberos)</a>
+        or <a href="#security_sasl_plain_brokerconfig">PLAIN</a>.</li>
+      <li>Pass the JAAS config file location as JVM parameter to each Kafka broker.
+        For example:
+        <pre>    -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf</pre></li>
+      <li>Configure a SASL port in server.properties, by adding at least one of
+        SASL_PLAINTEXT or SASL_SSL to the <i>listeners</i> parameter, which
+        contains one or more comma-separated values:
+        <pre>    listeners=SASL_PLAINTEXT://host.name:port</pre>
+        If SASL_SSL is used, then <a href="#security_ssl">SSL must also be
+        configured</a>. If you are only configuring a SASL port (or if you want
+        the Kafka brokers to authenticate each other using SASL) then make sure
+        you set the same SASL protocol for inter-broker communication:
+        <pre>    security.inter.broker.protocol=SASL_PLAINTEXT (or SASL_SSL)</pre></li>
+      <li>Enable one or more SASL mechanisms in server.properties:
+          <pre>    sasl.enabled.mechanisms=GSSAPI (,PLAIN)</pre></li>
+      <li>Configure the SASL mechanism for inter-broker communication in server.properties
+        if using SASL for inter-broker communication:
+        <pre>    sasl.mechanism.inter.broker.protocol=GSSAPI (or PLAIN)</pre></li>
+      <li>Follow the steps in <a href="#security_sasl_kerberos_brokerconfig">GSSAPI
(Kerberos)</a>
+        or <a href="#security_sasl_plain_brokerconfig">PLAIN</a> to configure
SASL
+        for the enabled mechanisms. To enable multiple mechanisms in the broker, follow
+        the steps <a href="#security_sasl_multimechanism">here</a>.</li>
+      <u><a id="security_sasl_brokernotes" href="#security_sasl_brokernotes">Important
notes:</a></u>
+      <ol>
+        <li><tt>KafkaServer</tt> is the section name in the JAAS file used
by each
+          KafkaServer/Broker. This section provides SASL configuration options
+          for the broker including any SASL client connections made by the broker
+          for inter-broker communication.</li>
+        <li><tt>Client</tt> section is used to authenticate a SASL connection
with
+          zookeeper. It also allows the brokers to set SASL ACL on zookeeper
+          nodes which locks these nodes down so that only the brokers can
+          modify it. It is necessary to have the same principal name across all
+          brokers. If you want to use a section name other than Client, set the
+          system property <tt>zookeeper.sasl.client</tt> to the appropriate
+          name (<i>e.g.</i>, <tt>-Dzookeeper.sasl.client=ZkClient</tt>).</li>
+        <li>ZooKeeper uses "zookeeper" as the service name by default. If you
+          want to change this, set the system property
+          <tt>zookeeper.sasl.client.username</tt> to the appropriate name
+          (<i>e.g.</i>, <tt>-Dzookeeper.sasl.client.username=zk</tt>).</li>
+      </ol>
+    </ol>
+  </li>
+  <li><h4><a id="security_sasl_clientconfig"
+    href="#security_sasl_clientconfig">SASL configuration for Kafka clients</a></h4>
+    SASL authentication is only supported for the new Java Kafka producer and
+    consumer, the older API is not supported. To configure SASL authentication
+    on the clients:
+    <ol>
+      <li>Select a SASL mechanism for authentication.</li>
+      <li>Add a JAAS config file for the selected mechanism as described in the examples
+        for setting up <a href="#security_sasl_kerberos_clientconfig">GSSAPI (Kerberos)</a>
+        or <a href="#security_sasl_plain_clientconfig">PLAIN</a>. <tt>KafkaClient</tt>
is the
+        section name in the JAAS file used by Kafka clients.</li>
+      <li>Pass the JAAS config file location as JVM parameter to each client JVM. For
example:
+        <pre>    -Djava.security.auth.login.config=/etc/kafka/kafka_client_jaas.conf</pre></li>
+      <li>Configure the following properties in producer.properties or
+        consumer.properties:
+        <pre>    security.protocol=SASL_PLAINTEXT (or SASL_SSL)
+    sasl.mechanism=GSSAPI (or PLAIN)</pre></li>
+      <li>Follow the steps in <a href="#security_sasl_kerberos_clientconfig">GSSAPI
(Kerberos)</a>
+        or <a href="#security_sasl_plain_clientconfig">PLAIN</a> to configure
SASL
+        for the selected mechanism.</li>
+    </ol>
+  </li>
+  <li><h4><a id="security_sasl_kerberos" href="#security_sasl_kerberos">Authentication
using SASL/Kerberos</a></h4>
+    <ol>
+      <li><h5><a id="security_sasl_kerberos_prereq" href="#security_sasl_kerberos_prereq">Prerequisites</a></h5>
+      <ol>
+          <li><b>Kerberos</b><br>
+          If your organization is already using a Kerberos server (for example, by using
Active Directory), there is no need to install a new server just for Kafka. Otherwise you
will need to install one, your Linux vendor likely has packages for Kerberos and a short guide
on how to install and configure it (<a href="https://help.ubuntu.com/community/Kerberos">Ubuntu</a>,
<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/installing-kerberos.html">Redhat</a>).
Note that if you are using Oracle Java, you will need to download JCE policy files for your
Java version and copy them to $JAVA_HOME/jre/lib/security.</li>
+          <li><b>Create Kerberos Principals</b><br>
+          If you are using the organization's Kerberos or Active Directory server, ask your
Kerberos administrator for a principal for each Kafka broker in your cluster and for every
operating system user that will access Kafka with Kerberos authentication (via clients and
tools).</br>
+          If you have installed your own Kerberos, you will need to create these principals
yourself using the following commands:
             <pre>
     sudo /usr/sbin/kadmin.local -q 'addprinc -randkey kafka/{hostname}@{REALM}'
     sudo /usr/sbin/kadmin.local -q "ktadd -k /etc/security/keytabs/{keytabname}.keytab kafka/{hostname}@{REALM}"</pre></li>
-        <li><b>Make sure all hosts can be reachable using hostnames</b>
- it is a Kerberos requirement that all your hosts can be resolved with their FQDNs.</li>
-    </ol>
-    <li><h4><a id="security_sasl_brokerconfig" href="#security_sasl_brokerconfig">Configuring
Kafka Brokers</a></h4>
-    <ol>
-        <li>Add a suitably modified JAAS file similar to the one below to each Kafka
broker's config directory, let's call it kafka_server_jaas.conf for this example (note that
each broker should have its own keytab):
-        <pre>
+          <li><b>Make sure all hosts can be reachable using hostnames</b>
- it is a Kerberos requirement that all your hosts can be resolved with their FQDNs.</li>
+      </ol>
+      <li><h5><a id="security_sasl_kerberos_brokerconfig" href="#security_sasl_kerberos_brokerconfig">Configuring
Kafka Brokers</a></h5>
+      <ol>
+          <li>Add a suitably modified JAAS file similar to the one below to each Kafka
broker's config directory, let's call it kafka_server_jaas.conf for this example (note that
each broker should have its own keytab):
+          <pre>
     KafkaServer {
         com.sun.security.auth.module.Krb5LoginModule required
         useKeyTab=true
@@ -215,40 +289,31 @@ Apache Kafka allows clients to connect over SSL. By default SSL is disabled
but
        principal="kafka/kafka1.hostname.com@EXAMPLE.COM";
     };</pre>
 
-        </li>
-        <li>Pass the JAAS and optionally the krb5 file locations as JVM parameters
to each Kafka broker (see <a href="https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html">here</a>
for more details):
-            <pre>
-    -Djava.security.krb5.conf=/etc/kafka/krb5.conf
+          </li>
+          <tt>KafkaServer</tt> section in the JAAS file tells the broker which
principal to use and the location of the keytab where this principal is stored. It
+          allows the broker to login using the keytab specified in this section. See <a
href="#security_sasl_brokernotes">notes</a> for more details on Zookeeper SASL configuration.
+          <li>Pass the JAAS and optionally the krb5 file locations as JVM parameters
to each Kafka broker (see <a href="https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html">here</a>
for more details): 
+            <pre>    -Djava.security.krb5.conf=/etc/kafka/krb5.conf
     -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf</pre>
-        </li>
-        <li>Make sure the keytabs configured in the JAAS file are readable by the operating
system user who is starting kafka broker.</li>
-        <li>Configure a SASL port in server.properties, by adding at least one of SASL_PLAINTEXT
or SASL_SSL to the <i>listeners</i> parameter, which contains one or more comma-separated
values:
-        <pre>
-    listeners=SASL_PLAINTEXT://host.name:port</pre>
-        If SASL_SSL is used, then <a href="#security_ssl">SSL must also be configured</a>.
-        If you are only configuring a SASL port (or if you want the Kafka brokers to authenticate
each other using SASL) then make sure you set the same SASL protocol for inter-broker communication:
-        <pre>
-    security.inter.broker.protocol=SASL_PLAINTEXT (or SASL_SSL)</pre></li>
-
-        We must also configure the service name in server.properties, which should match
the principal name of the kafka brokers. In the above example, principal is "kafka/kafka1.hostname.com@EXAMPLE.com",
so:
-        <pre>
-    sasl.kerberos.service.name=kafka</pre>
-
-        <u>Important notes:</u>
-        <ol>
-            <li>KafkaServer is a section name in JAAS file used by each KafkaServer/Broker.
This section tells the broker which principal to use and the location of the keytab where
this principal is stored. It allows the broker to login using the keytab specified in this
section.</li>
-            <li>Client section is used to authenticate a SASL connection with zookeeper.
It also allows the brokers to set SASL ACL on zookeeper nodes which locks these nodes down
so that only the brokers can modify it. It is necessary to have the same principal name across
all brokers. If you want to use a section name other than Client, set the system property
<tt>zookeeper.sasl.client</tt> to the appropriate name (<i>e.g.</i>,
<tt>-Dzookeeper.sasl.client=ZkClient</tt>).</li>
-            <li>ZooKeeper uses "zookeeper" as the service name by default. If you want
to change this, set the system property <tt>zookeeper.sasl.client.username</tt>
to the appropriate name (<i>e.g.</i>, <tt>-Dzookeeper.sasl.client.username=zk</tt>).</li>
-        </ol>
-
-    </ol>
-    <li><h4><a id="security_sasl_clientconfig" href="#security_sasl_clientconfig">Configuring
Kafka Clients</a></h4>
-        SASL authentication is only supported for the new kafka producer and consumer, the
older API is not supported. To configure SASL authentication on the clients:
-        <ol>
-            <li>
-                Clients (producers, consumers, connect workers, etc) will authenticate to
the cluster with their own principal (usually with the same name as the user running the client),
so obtain or create these principals as needed. Then create a JAAS file for each principal.
-                The KafkaClient section describes how the clients like producer and consumer
can connect to the Kafka Broker. The following is an example configuration for a client using
a keytab (recommended for long-running processes):
-            <pre>
+          </li>
+          <li>Make sure the keytabs configured in the JAAS file are readable by the
operating system user who is starting kafka broker.</li>
+          <li>Configure SASL port and SASL mechanisms in server.properties as described
<a href="#security_sasl_brokerconfig">here</a>.</pre> For example:
+          <pre>    listeners=SASL_PLAINTEXT://host.name:port
+    security.inter.broker.protocol=SASL_PLAINTEXT
+    sasl.mechanism.inter.broker.protocol=GSSAPI
+    sasl.enabled.mechanisms=GSSAPI
+          </pre>
+          </li>We must also configure the service name in server.properties, which
should match the principal name of the kafka brokers. In the above example, principal is "kafka/kafka1.hostname.com@EXAMPLE.com",
so: 
+          <pre>    sasl.kerberos.service.name=kafka</pre>
+
+      </ol></li>
+      <li><h5><a id="security_sasl_kerberos_clientconfig" href="#security_kerberos_sasl_clientconfig">Configuring
Kafka Clients</a></h5>
+          To configure SASL authentication on the clients:
+          <ol>
+              <li>
+                  Clients (producers, consumers, connect workers, etc) will authenticate
to the cluster with their own principal (usually with the same name as the user running the
client), so obtain or create these principals as needed. Then create a JAAS file for each
principal.
+                  The KafkaClient section describes how the clients like producer and consumer
can connect to the Kafka Broker. The following is an example configuration for a client using
a keytab (recommended for long-running processes):
+              <pre>
     KafkaClient {
         com.sun.security.auth.module.Krb5LoginModule required
         useKeyTab=true
@@ -257,93 +322,212 @@ Apache Kafka allows clients to connect over SSL. By default SSL is
disabled but
         principal="kafka-client-1@EXAMPLE.COM";
     };</pre>
 
-            For command-line utilities like kafka-console-consumer or kafka-console-producer,
kinit can be used along with "useTicketCache=true" as in:
-            <pre>
+              For command-line utilities like kafka-console-consumer or kafka-console-producer,
kinit can be used along with "useTicketCache=true" as in:
+              <pre>
     KafkaClient {
         com.sun.security.auth.module.Krb5LoginModule required
         useTicketCache=true;
     };</pre>
-            </li>
-            <li>Pass the JAAS and optionally krb5 file locations as JVM parameters
to each client JVM (see <a href="https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html">here</a>
for more details):
-            <pre>
-    -Djava.security.krb5.conf=/etc/kafka/krb5.conf
+              </li>
+              <li>Pass the JAAS and optionally krb5 file locations as JVM parameters
to each client JVM (see <a href="https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html">here</a>
for more details): 
+              <pre>    -Djava.security.krb5.conf=/etc/kafka/krb5.conf
     -Djava.security.auth.login.config=/etc/kafka/kafka_client_jaas.conf</pre></li>
-            <li>Make sure the keytabs configured in the kafka_client_jaas.conf are
readable by the operating system user who is starting kafka client.</li>
-            <li>Configure the following properties in producer.properties or consumer.properties:
-                <pre>
-    security.protocol=SASL_PLAINTEXT (or SASL_SSL)
-    sasl.kerberos.service.name=kafka</pre>
-            </li>
-        </ol></li>
-
-    <li><h4><a id="security_rolling_upgrade" href="#security_rolling_upgrade">Incorporating
Security Features in a Running Cluster</a></h4>
-        You can secure a running cluster via one or more of the supported protocols discussed
previously. This is done in phases:
-        <p></p>
+              <li>Make sure the keytabs configured in the kafka_client_jaas.conf are
readable by the operating system user who is starting kafka client.</li>
+              <li>Configure the following properties in producer.properties or consumer.properties:

+              <pre>    security.protocol=SASL_PLAINTEXT (or SASL_SSL)
+    sasl.mechanism=GSSAPI
+    sasl.kerberos.service.name=kafka</pre></li>
+          </ol>
+      </li>
+    </ol>
+  </li>
+      
+  <li><h4><a id="security_sasl_plain" href="#security_sasl_plain">Authentication
using SASL/PLAIN</a></h4>
+    <p>SASL/PLAIN is a simple username/password authentication mechanism that is typically
used with TLS for encryption to implement secure authentication.
+       Kafka supports a default implementation for SASL/PLAIN which can be extended for production
use as described <a href="#security_sasl_plain_production">here</a>.</p>
+       The username is used as the authenticated <code>Principal</code> for configuration
of ACLs etc.
+    <ol>
+      <li><h5><a id="security_sasl_plain_brokerconfig" href="#security_sasl_plain_brokerconfig">Configuring
Kafka Brokers</a></h5>
+        <ol>
+          <li>Add a suitably modified JAAS file similar to the one below to each Kafka
broker's config directory, let's call it kafka_server_jaas.conf for this example:
+            <pre>
+    KafkaServer {
+        org.apache.kafka.common.security.plain.PlainLoginModule required
+        username="admin"
+        password="admin-secret"
+        user_admin="admin-secret"
+        user_alice="alice-secret";
+    };</pre>
+            This configuration defines two users (<i>admin</i> and <i>alice</i>).
The properties <tt>username</tt> and <tt>password</tt>
+            in the <tt>KafkaServer</tt> section are used by the broker to initiate
connections to other brokers. In this example,
+            <i>admin</i> is the user for inter-broker communication. The set
of properties <tt>user_<i>userName</i></tt> defines
+            the passwords for all users that connect to the broker and the broker validates
all client connections including
+            those from other brokers using these properties.</li>
+          <li>Pass the JAAS config file location as JVM parameter to each Kafka broker:
+              <pre>    -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf</pre></li>
+          <li>Configure SASL port and SASL mechanisms in server.properties as described
<a href="#security_sasl_brokerconfig">here</a>.</pre> For example:
+            <pre>    listeners=SASL_SSL://host.name:port
+    security.inter.broker.protocol=SASL_SSL
+    sasl.mechanism.inter.broker.protocol=PLAIN
+    sasl.enabled.mechanisms=PLAIN</pre></li>
+        </ol>
+      </li>
+
+      <li><h5><a id="security_sasl_plain_clientconfig" href="#security_sasl_plain_clientconfig">Configuring
Kafka Clients</a></h5>
+        To configure SASL authentication on the clients:
+        <ol>
+          <li>The <tt>KafkaClient</tt> section describes how the clients
like producer and consumer can connect to the Kafka Broker.
+          The following is an example configuration for a client for the PLAIN mechanism:
+            <pre>
+    KafkaClient {
+        org.apache.kafka.common.security.plain.PlainLoginModule required
+        username="alice"
+        password="alice-secret";
+    };</pre>
+            The properties <tt>username</tt> and <tt>password</tt>
in the <tt>KafkaClient</tt> section are used by clients to configure
+            the user for client connections. In this example, clients connect to the broker
as user <i>alice</i>.
+          </li>
+          <li>Pass the JAAS config file location as JVM parameter to each client JVM:
+            <pre>    -Djava.security.auth.login.config=/etc/kafka/kafka_client_jaas.conf</pre></li>
+          <li>Configure the following properties in producer.properties or consumer.properties:
+            <pre>    security.protocol=SASL_SSL
+    sasl.mechanism=PLAIN</pre></li>
+        </ol>
+      </li>
+      <li><h5><a id="security_sasl_plain_production" href="#security_sasl_plain_production">Use
of SASL/PLAIN in production</a></h5>
         <ul>
-            <li>Incrementally bounce the cluster nodes to open additional secured port(s).</li>
-            <li>Restart clients using the secured rather than PLAINTEXT port (assuming
you are securing the client-broker connection).</li>
-            <li>Incrementally bounce the cluster again to enable broker-to-broker security
(if this is required)</li>
-            <li>A final incremental bounce to close the PLAINTEXT port.</li>
+          <li>SASL/PLAIN should be used only with SSL as transport layer to ensure
that clear passwords are not transmitted on the wire without encryption.</li>
+          <li>The default implementation of SASL/PLAIN in Kafka specifies usernames
and passwords in the JAAS configuration file as shown
+            <a href="#security_sasl_plain_brokerconfig">here</a>. To avoid storing
passwords on disk, you can plugin your own implementation of
+            <code>javax.security.auth.spi.LoginModule</code> that provides usernames
and passwords from an external source. The login module implementation should
+            provide username as the public credential and password as the private credential
of the <code>Subject</code>. The default implementation
+            <code>org.apache.kafka.common.security.plain.PlainLoginModule</code>
can be used as an example.</li>
+          <li>In production systems, external authentication servers may implement
password authentication. Kafka brokers can be integrated with these servers by adding
+            your own implementation of <code>javax.security.sasl.SaslServer</code>.
The default implementation included in Kafka in the package
+            <code>org.apache.kafka.common.security.plain</code> can be used as
an example to get started.
+            <ul>
+              <li>New providers must be installed and registered in the JVM. Providers
can be installed by adding provider classes to
+              the normal <tt>CLASSPATH</tt> or bundled as a jar file and added
to <tt><i>JAVA_HOME</i>/lib/ext</tt>.</li>
+              <li>Providers can be registered statically by adding a provider to the
security properties file
+              <tt><i>JAVA_HOME</i>/lib/security/java.security</tt>.
+              <pre>    security.provider.n=providerClassName</pre>
+              where <i>providerClassName</i> is the fully qualified name of the
new provider and <i>n</i> is the preference order with
+              lower numbers indicating higher preference.</li>
+              <li>Alternatively, you can register providers dynamically at runtime
by invoking <code>Security.addProvider</code> at the beginning of the client
+              application or in a static initializer in the login module. For example:
+              <pre>    Security.addProvider(new PlainSaslServerProvider());</pre></li>
+              <li>For more details, see <a href="http://docs.oracle.com/javase/8/docs/technotes/guides/security/crypto/CryptoSpec.html">JCA
Reference</a>.</li>
+            </ul>
+          </li>
         </ul>
-        <p></p>
-        The specific steps for configuring SSL and SASL are described in sections <a href="#security_ssl">7.2</a>
and <a href="#security_sasl">7.3</a>.
-        Follow these steps to enable security for your desired protocol(s).
-        <p></p>
-        The security implementation lets you configure different protocols for both broker-client
and broker-broker communication.
-        These must be enabled in separate bounces. A PLAINTEXT port must be left open throughout
so brokers and/or clients can continue to communicate.
-        <p></p>
-
-        When performing an incremental bounce stop the brokers cleanly via a SIGTERM. It's
also good practice to wait for restarted replicas to return to the ISR list before moving
onto the next node.
-        <p></p>
-        As an example, say we wish to encrypt both broker-client and broker-broker communication
with SSL. In the first incremental bounce, a SSL port is opened on each node:
+      </li>
+    </ol>
+  </li>
+  <li><h4><a id="security_sasl_multimechanism" href="#security_sasl_multimechanism">Enabling
multiple SASL mechanisms in a broker</a></h4>
+    <ol>
+      <li>Specify configuration for the login modules of all enabled mechanisms in
the <tt>KafkaServer</tt> section of the JAAS config file. For example:
         <pre>
+    KafkaServer {
+        com.sun.security.auth.module.Krb5LoginModule required
+        useKeyTab=true
+        storeKey=true
+        keyTab="/etc/security/keytabs/kafka_server.keytab"
+        principal="kafka/kafka1.hostname.com@EXAMPLE.COM";
+
+        org.apache.kafka.common.security.plain.PlainLoginModule required
+        username="admin"
+        password="admin-secret"
+        user_admin="admin-secret"
+        user_alice="alice-secret";
+    };</pre></li>
+      <li>Enable the SASL mechanisms in server.properties: <pre>    sasl.enabled.mechanisms=GSSAPI,PLAIN</pre></li>
+      <li>Specify the SASL security protocol and mechanism for inter-broker communication
in server.properties if required:
+        <pre>    security.inter.broker.protocol=SASL_PLAINTEXT (or SASL_SSL)
+    sasl.mechanism.inter.broker.protocol=GSSAPI (or PLAIN)</pre></li>
+      <li>Follow the mechanism-specific steps in <a href="#security_sasl_kerberos_brokerconfig">GSSAPI
(Kerberos)</a>
+          and <a href="#security_sasl_plain_brokerconfig">PLAIN</a> to configure
SASL for the enabled mechanisms.</li>
+    </ol>
+  </li>
+  <li><h4><a id="security_rolling_upgrade" href="#security_rolling_upgrade">Incorporating
Security Features in a Running Cluster</a></h4>
+          You can secure a running cluster via one or more of the supported protocols discussed
previously. This is done in phases:
+          <p></p>
+          <ul>
+              <li>Incrementally bounce the cluster nodes to open additional secured
port(s).</li>
+              <li>Restart clients using the secured rather than PLAINTEXT port (assuming
you are securing the client-broker connection).</li>
+              <li>Incrementally bounce the cluster again to enable broker-to-broker
security (if this is required)</li>
+            <li>A final incremental bounce to close the PLAINTEXT port.</li>
+          </ul>
+          <p></p>
+          The specific steps for configuring SSL and SASL are described in sections <a
href="#security_ssl">7.2</a> and <a href="#security_sasl">7.3</a>.
+          Follow these steps to enable security for your desired protocol(s).
+          <p></p>
+          The security implementation lets you configure different protocols for both broker-client
and broker-broker communication.
+          These must be enabled in separate bounces. A PLAINTEXT port must be left open throughout
so brokers and/or clients can continue to communicate.
+          <p></p>
+
+          When performing an incremental bounce stop the brokers cleanly via a SIGTERM. It's
also good practice to wait for restarted replicas to return to the ISR list before moving
onto the next node.
+          <p></p>
+          As an example, say we wish to encrypt both broker-client and broker-broker communication
with SSL. In the first incremental bounce, a SSL port is opened on each node:
+          <pre>
          listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092</pre>
 
-        We then restart the clients, changing their config to point at the newly opened,
secured port:
+          We then restart the clients, changing their config to point at the newly opened,
secured port:
 
-        <pre>
+          <pre>
         bootstrap.servers = [broker1:9092,...]
         security.protocol = SSL
         ...etc</pre>
 
-        In the second incremental server bounce we instruct Kafka to use SSL as the broker-broker
protocol (which will use the same SSL port):
+          In the second incremental server bounce we instruct Kafka to use SSL as the broker-broker
protocol (which will use the same SSL port):
 
-        <pre>
+          <pre>
         listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092
         security.inter.broker.protocol=SSL</pre>
 
-        In the final bounce we secure the cluster by closing the PLAINTEXT port:
+          In the final bounce we secure the cluster by closing the PLAINTEXT port:
 
-        <pre>
+          <pre>
         listeners=SSL://broker1:9092
         security.inter.broker.protocol=SSL</pre>
 
-        Alternatively we might choose to open multiple ports so that different protocols
can be used for broker-broker and broker-client communication. Say we wished to use SSL encryption
throughout (i.e. for broker-broker and broker-client communication) but we'd like to add SASL
authentication to the broker-client connection also. We would achieve this by opening two
additional ports during the first bounce:
+          Alternatively we might choose to open multiple ports so that different protocols
can be used for broker-broker and broker-client communication. Say we wished to use SSL encryption
throughout (i.e. for broker-broker and broker-client communication) but we'd like to add SASL
authentication to the broker-client connection also. We would achieve this by opening two
additional ports during the first bounce:
 
-        <pre>
+          <pre>
         listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092,SASL_SSL://broker1:9093</pre>
 
-        We would then restart the clients, changing their config to point at the newly opened,
SASL & SSL secured port:
+          We would then restart the clients, changing their config to point at the newly
opened, SASL & SSL secured port:
 
-        <pre>
+          <pre>
         bootstrap.servers = [broker1:9093,...]
         security.protocol = SASL_SSL
         ...etc</pre>
 
-        The second server bounce would switch the cluster to use encrypted broker-broker
communication via the SSL port we previously opened on port 9092:
+          The second server bounce would switch the cluster to use encrypted broker-broker
communication via the SSL port we previously opened on port 9092:
 
-        <pre>
+          <pre>
         listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092,SASL_SSL://broker1:9093
         security.inter.broker.protocol=SSL</pre>
 
-        The final bounce secures the cluster by closing the PLAINTEXT port.
+          The final bounce secures the cluster by closing the PLAINTEXT port.
 
-        <pre>
+          <pre>
        listeners=SSL://broker1:9092,SASL_SSL://broker1:9093
        security.inter.broker.protocol=SSL</pre>
 
-        ZooKeeper can be secured independently of the Kafka cluster. The steps for doing
this are covered in section <a href="#zk_authz_migration">7.5.2</a>.
-    </li>
+          ZooKeeper can be secured independently of the Kafka cluster. The steps for doing
this are covered in section <a href="#zk_authz_migration">7.5.2</a>.
+  </li>
+  <li><h4><a id="saslmechanism_rolling_upgrade" href="#saslmechanism_rolling_upgrade">Modifying
SASL mechanism in a Running Cluster</a></h4>
+    <p>SASL mechanism can be modified in a running cluster using the following sequence:</p>
+    <ol>
+      <li>Enable new SASL mechanism by adding the mechanism to <tt>sasl.enabled.mechanisms</tt>
in server.properties for each broker. Update JAAS config file to include both
+        mechanisms as described <a href="#security_sasl_multimechanism">here</a>.
Incrementally bounce the cluster nodes.</li>
+      <li>Restart clients using the new mechanism.</li>
+      <li>To change the mechanism of inter-broker communication (if this is required),
set <tt>sasl.mechanism.inter.broker.protocol</tt> in server.properties to the
new mechanism and
+        incrementally bounce the cluster again.</li>
+      <li>To remove old mechanism (if this is required), remove the old mechanism from
<tt>sasl.enabled.mechanisms</tt> in server.properties and remove the entries for
the
+        old mechanism from JAAS config file. Incrementally bounce the cluster again.</li>
+    </ol>
+  </li>
 </ol>
 
 <h3><a id="security_authz" href="#security_authz">7.4 Authorization and ACLs</a></h3>


Mime
View raw message