kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maniku...@apache.org
Subject [kafka] branch trunk updated: KAFKA-7617: Add authorization primitives to security page
Date Fri, 30 Nov 2018 09:12:58 GMT
This is an automated email from the ASF dual-hosted git repository.

manikumar pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/trunk by this push:
     new eca6620  KAFKA-7617: Add authorization primitives to security page
eca6620 is described below

commit eca66207c8dc81ca139975a25ce22c6f465fabba
Author: Viktor Somogyi-Vass <viktorsomogyi@gmail.com>
AuthorDate: Fri Nov 30 14:42:34 2018 +0530

    KAFKA-7617: Add authorization primitives to security page
    
    This is a security page improvement that adds documentation about Kafka authorization
primitives to the security page.
    
    Author: Viktor Somogyi-Vass <viktorsomogyi@gmail.com>
    Author: Viktor Somogyi <viktorsomogyi@gmail.com>
    Author: Manikumar Reddy <manikumar.reddy@gmail.com>
    
    Reviewers: Stanislav Kozlovski <stanislav_kozlovski@outlook.com>, Srinivas <srinivas96alluri@gmail.com>,
Manikumar Reddy <manikumar.reddy@gmail.com>, Mickael Maison <mickael.maison@gmail.com>
    
    Closes #5906 from viktorsomogyi/security-page-improvement
---
 docs/security.html | 444 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 444 insertions(+)

diff --git a/docs/security.html b/docs/security.html
index 5f6d0ac..28c506c 100644
--- a/docs/security.html
+++ b/docs/security.html
@@ -1258,6 +1258,450 @@
 
     </ul>
 
+    <h4><a id="security_authz_primitives" href="#security_authz_primitives">Authorization
Primitives</a></h4>
+    <p>Protocol calls are usually performing some operations on certain resources in
Kafka. It is required to know the
+        operations and resources to set up effective protection. In this section we'll list
these operations and
+        resources, then list the combination of these with the protocols to see the valid
scenarios.</p>
+    <h5><a id="operations_in_kafka" href="#operations_in_kafka">Operations in
Kafka</a></h5>
+    <p>There are a few operation primitives that can be used to build up privileges.
These can be matched up with
+        certain resources to allow specific protocol calls for a given user. These are:</p>
+    <ul>
+        <li>Read</li>
+        <li>Write</li>
+        <li>Create</li>
+        <li>Delete</li>
+        <li>Alter</li>
+        <li>Describe</li>
+        <li>ClusterAction</li>
+        <li>DescribeConfigs</li>
+        <li>AlterConfigs</li>
+        <li>IdempotentWrite</li>
+        <li>All</li>
+    </ul>
+    <h5><a id="resources_in_kafka" href="#resources_in_kafka">Resources in Kafka</a></h5>
+    <p>The operations above can be applied on certain resources which are described
below.</p>
+    <ul>
+        <li><b>Topic:</b> this simply represents a Topic. All protocol
calls that are acting on topics (such as reading,
+            writing them) require the corresponding privilege to be added. If there is an
authorization error with a
+            topic resource, then a TOPIC_AUTHORIZATION_FAILED (error code: 29) will be returned.</li>
+        <li><b>Group:</b> this represents the consumer groups in the brokers.
All protocol calls that are working with
+            consumer groups, like joining a group must have privileges with the group in
subject. If the privilege is not
+            given then a GROUP_AUTHORIZATION_FAILED (error code: 30) will be returned in
the protocol response.</li>
+        <li><b>Cluster:</b> this resource represents the cluster. Operations
that are affecting the whole cluster, like
+            controlled shutdown are protected by privileges on the Cluster resource. If there
is an authorization problem
+            on a cluster resource, then a CLUSTER_AUTHORIZATION_FAILED (error code: 31) will
be returned.</li>
+        <li><b>TransactionalId:</b> this resource represents actions related
to transactions, such as committing.
+            If any error occurs, then a TRANSACTIONAL_ID_AUTHORIZATION_FAILED (error code:
53) will be returned by brokers.</li>
+        <li><b>DelegationToken:</b> this represents the delegation tokens
in the cluster. Actions, such as describing
+            delegation tokens could be protected by a privilege on the DelegationToken resource.
Since these objects have
+            a little special behavior in Kafka it is recommended to read
+            <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-DescribeDelegationTokenRequest">KIP-48</a>
+            and the related upstream documentation at <a href="#security_delegation_token">Authentication
using Delegation Tokens</a>.</li>
+    </ul>
+    <h5><a id="operations_resources_and_protocols" href="#operations_resources_and_protocols">Operations
and Resources on Protocols</a></h5>
+    <p>In the below table we'll list the valid operations on resources that are executed
by the Kafka API protocols.</p>
+    <table class="data-table">
+        <thead>
+        <tr>
+            <th>Protocol (API key)</th>
+            <th>Operation</th>
+            <th>Resource</th>
+            <th>Note</th>
+        </tr>
+        </thead>
+        <tbody>
+        <tr>
+            <td>PRODUCE (0)</td>
+            <td>Write</td>
+            <td>TransactionalId</td>
+            <td>An transactional producer which has its transactional.id set requires
this privilege.</td>
+        </tr>
+        <tr>
+            <td>PRODUCE (0)</td>
+            <td>IdempotentWrite</td>
+            <td>Cluster</td>
+            <td>An idempotent produce action requires this privilege.</td>
+        </tr>
+        <tr>
+            <td>PRODUCE (0)</td>
+            <td>Write</td>
+            <td>Topic</td>
+            <td>This applies to a normal produce action.</td>
+        </tr>
+        <tr>
+            <td>FETCH (1)</td>
+            <td>ClusterAction</td>
+            <td>Cluster</td>
+            <td>A follower must have ClusterAction on the Cluster resource in order
to fetch partition data.</td>
+        </tr>
+        <tr>
+            <td>FETCH (1)</td>
+            <td>Read</td>
+            <td>Topic</td>
+            <td>Regular Kafka consumers need READ permission on each partition they
are fetching.</td>
+        </tr>
+        <tr>
+            <td>LIST_OFFSETS (2)</td>
+            <td>Describe</td>
+            <td>Topic</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>METADATA (3)</td>
+            <td>Describe</td>
+            <td>Topic</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>METADATA (3)</td>
+            <td>Create</td>
+            <td>Cluster</td>
+            <td>If topic auto-creation is enabled, then the broker-side API will check
for the existence of a Cluster
+            level privilege. If it's found then it'll allow creating the topic, otherwise
it'll iterate through the
+            Topic level privileges (see the next one).</td>
+        </tr>
+        <tr>
+            <td>METADATA (3)</td>
+            <td>Create</td>
+            <td>Topic</td>
+            <td>This authorizes auto topic creation if enabled but the given user doesn't
have a cluster level
+                permission (above).</td>
+        </tr>
+        <tr>
+            <td>LEADER_AND_ISR (4)</td>
+            <td>ClusterAction</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>STOP_REPLICA (5)</td>
+            <td>ClusterAction</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>UPDATE_METADATA (6)</td>
+            <td>ClusterAction</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>CONTROLLED_SHUTDOWN (7)</td>
+            <td>ClusterAction</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>OFFSET_COMMIT (8)</td>
+            <td>Read</td>
+            <td>Group</td>
+            <td>An offset can only be committed if it's authorized to the given group
and the topic too (see below).
+                Group access is checked first, then Topic access.</td>
+        </tr>
+        <tr>
+            <td>OFFSET_COMMIT (8)</td>
+            <td>Read</td>
+            <td>Topic</td>
+            <td>Since offset commit is part of the consuming process, it needs privileges
for the read action.</td>
+        </tr>
+        <tr>
+            <td>OFFSET_FETCH (9)</td>
+            <td>Describe</td>
+            <td>Group</td>
+            <td>Similarly to OFFSET_COMMIT, the application must have privileges on
group and topic level too to be able
+                to fetch. However in this case it requires describe access instead of read.
Group access is checked first,
+                then Topic access.</td>
+        </tr>
+        <tr>
+            <td>OFFSET_FETCH (9)</td>
+            <td>Describe</td>
+            <td>Topic</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>FIND_COORDINATOR (10)</td>
+            <td>Describe</td>
+            <td>Group</td>
+            <td>The FIND_COORDINATOR request can be of "Group" type in which case it
is looking for consumergroup
+                coordinators. This privilege would represent the Group mode.</td>
+        </tr>
+        <tr>
+            <td>FIND_COORDINATOR (10)</td>
+            <td>Describe</td>
+            <td>TransactionalId</td>
+            <td>This applies only on transactional producers and checked when a producer
tries to find the transaction
+                coordinator.</td>
+        </tr>
+        <tr>
+            <td>JOIN_GROUP (11)</td>
+            <td>Read</td>
+            <td>Group</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>HEARTBEAT (12)</td>
+            <td>Read</td>
+            <td>Group</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>LEAVE_GROUP (13)</td>
+            <td>Read</td>
+            <td>Group</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>SYNC_GROUP (14)</td>
+            <td>Read</td>
+            <td>Group</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>DESCRIBE_GROUPS (15)</td>
+            <td>Describe</td>
+            <td>Group</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>LIST_GROUPS (16)</td>
+            <td>Describe</td>
+            <td>Cluster</td>
+            <td>When the broker checks to authorize a list_groups request it first
checks for this cluster
+                level authorization. If none found then it proceeds to check the groups individually.
This operation
+                doesn't return CLUSTER_AUTHORIZATION_FAILED.</td>
+        </tr>
+        <tr>
+            <td>LIST_GROUPS (16)</td>
+            <td>Describe</td>
+            <td>Group</td>
+            <td>If none of the groups are authorized, then just an empty response will
be sent back instead
+                of an error. This operation doesn't return CLUSTER_AUTHORIZATION_FAILED.
This is applicable from the
+                2.1 release.</td>
+        </tr>
+        <tr>
+            <td>SASL_HANDSHAKE (17)</td>
+            <td></td>
+            <td></td>
+            <td>The SASL handshake is part of the authentication process and therefore
it's not possible to
+                apply any kind of authorization here.</td>
+        </tr>
+        <tr>
+            <td>API_VERSIONS (18)</td>
+            <td></td>
+            <td></td>
+            <td>The API_VERSIONS request is part of the Kafka protocol handshake and
happens on connection
+                and before any authentication. Therefore it's not possible to control this
with authorization.</td>
+        </tr>
+        <tr>
+            <td>CREATE_TOPICS (19)</td>
+            <td>Create</td>
+            <td>Cluster</td>
+            <td>If there is no cluster level authorization then it won't return CLUSTER_AUTHORIZATION_FAILED
but
+                fall back to use topic level, which is just below. That'll throw error if
there is a problem.</td>
+        </tr>
+        <tr>
+            <td>CREATE_TOPICS (19)</td>
+            <td>Create</td>
+            <td>Topic</td>
+            <td>This is applicable from the 2.0 release.</td>
+        </tr>
+        <tr>
+            <td>DELETE_TOPICS (20)</td>
+            <td>Delete</td>
+            <td>Topic</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>DELETE_RECORDS (21)</td>
+            <td>Delete</td>
+            <td>Topic</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>INIT_PRODUCER_ID (22)</td>
+            <td>Write</td>
+            <td>TransactionalId</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>INIT_PRODUCER_ID (22)</td>
+            <td>IdempotentWrite</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>OFFSET_FOR_LEADER_EPOCH (23)</td>
+            <td>ClusterAction</td>
+            <td>Cluster</td>
+            <td>If there is no cluster level privilege for this operation, then it'll
check for topic level one.</td>
+        </tr>
+        <tr>
+            <td>OFFSET_FOR_LEADER_EPOCH (23)</td>
+            <td>Describe</td>
+            <td>Topic</td>
+            <td>This is applicable from the 2.1 release.</td>
+        </tr>
+        <tr>
+            <td>ADD_PARTITIONS_TO_TXN (24)</td>
+            <td>Write</td>
+            <td>TransactionalId</td>
+            <td>This API is only applicable to transactional requests. It first checks
for the Write action on the
+                TransactionalId resource, then it checks the Topic in subject (below).</td>
+        </tr>
+        <tr>
+            <td>ADD_PARTITIONS_TO_TXN (24)</td>
+            <td>Write</td>
+            <td>Topic</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>ADD_OFFSETS_TO_TXN (25)</td>
+            <td>Write</td>
+            <td>TransactionalId</td>
+            <td>Similarly to ADD_PARTITIONS_TO_TXN this is only applicable to transactional
request. It first checks
+                for Write action on the TransactionalId resource, then it checks whether
it can Read on the given group
+                (below).</td>
+        </tr>
+        <tr>
+            <td>ADD_OFFSETS_TO_TXN (25)</td>
+            <td>Read</td>
+            <td>Group</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>END_TXN (26)</td>
+            <td>Write</td>
+            <td>TransactionalId</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>WRITE_TXN_MARKERS (27)</td>
+            <td>ClusterAction</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>TXN_OFFSET_COMMIT (28)</td>
+            <td>Write</td>
+            <td>TransactionalId</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>TXN_OFFSET_COMMIT (28)</td>
+            <td>Read</td>
+            <td>Group</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>TXN_OFFSET_COMMIT (28)</td>
+            <td>Read</td>
+            <td>Topic</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>DESCRIBE_ACLS (29)</td>
+            <td>Describe</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>CREATE_ACLS (30)</td>
+            <td>Alter</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>DELETE_ACLS (31)</td>
+            <td>Alter</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>DESCRIBE_CONFIGS (32)</td>
+            <td>DescribeConfigs</td>
+            <td>Cluster</td>
+            <td>If broker configs are requested, then the broker will check cluster
level privileges.</td>
+        </tr>
+        <tr>
+            <td>DESCRIBE_CONFIGS (32)</td>
+            <td>DescribeConfigs</td>
+            <td>Topic</td>
+            <td>If topic configs are requested, then the broker will check topic level
privileges.</td>
+        </tr>
+        <tr>
+            <td>ALTER_CONFIGS (33)</td>
+            <td>AlterConfigs</td>
+            <td>Cluster</td>
+            <td>If broker configs are altered, then the broker will check cluster level
privileges.</td>
+        </tr>
+        <tr>
+            <td>ALTER_CONFIGS (33)</td>
+            <td>AlterConfigs</td>
+            <td>Topic</td>
+            <td>If topic configs are altered, then the broker will check topic level
privileges.</td>
+        </tr>
+        <tr>
+            <td>ALTER_REPLICA_LOG_DIRS (34)</td>
+            <td>Alter</td>
+            <td>Cluster</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>DESCRIBE_LOG_DIRS (35)</td>
+            <td>Describe</td>
+            <td>Cluster</td>
+            <td>An empty response will be returned on authorization failure.</td>
+        </tr>
+        <tr>
+            <td>SASL_AUTHENTICATE (36)</td>
+            <td></td>
+            <td></td>
+            <td>SASL_AUTHENTICATE is part of the authentication process and therefore
it's not possible to
+                apply any kind of authorization here.</td>
+        </tr>
+        <tr>
+            <td>CREATE_PARTITIONS (37)</td>
+            <td>Alter</td>
+            <td>Topic</td>
+            <td></td>
+        </tr>
+        <tr>
+            <td>CREATE_DELEGATION_TOKEN (38)</td>
+            <td></td>
+            <td></td>
+            <td>Creating delegation tokens has special rules, for this please see the
+                <a id="security_delegation_token" href="#security_delegation_token">Authentication
using Delegation Tokens</a> section.</td>
+        </tr>
+        <tr>
+            <td>RENEW_DELEGATION_TOKEN (39)</td>
+            <td></td>
+            <td></td>
+            <td>Renewing delegation tokens has special rules, for this please see the
+                <a id="security_delegation_token" href="#security_delegation_token">Authentication
using Delegation Tokens</a> section.</td>
+        </tr>
+        <tr>
+            <td>EXPIRE_DELEGATION_TOKEN (40)</td>
+            <td></td>
+            <td></td>
+            <td>Expiring delegation tokens has special rules, for this please see the
+                <a id="security_delegation_token" href="#security_delegation_token">Authentication
using Delegation Tokens</a> section.</td>
+        </tr>
+        <tr>
+            <td>DESCRIBE_DELEGATION_TOKEN (41)</td>
+            <td>Describe</td>
+            <td>DelegationToken</td>
+            <td>Describing delegation tokens has special rules, for this please see
the
+                <a id="security_delegation_token" href="#security_delegation_token">Authentication
using Delegation Tokens</a> section.</td>
+        </tr>
+        <tr>
+            <td>DELETE_GROUPS (42)</td>
+            <td>Delete</td>
+            <td>Group</td>
+            <td></td>
+        </tr>
+        </tbody>
+    </table>
+
     <h3><a id="security_rolling_upgrade" href="#security_rolling_upgrade">7.5
Incorporating Security Features in a Running Cluster</a></h3>
         You can secure a running cluster via one or more of the supported protocols discussed
previously. This is done in phases:
         <p></p>


Mime
View raw message