kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ij...@apache.org
Subject kafka-site git commit: Updates to config documentation from 0.11.0 branch
Date Wed, 28 Jun 2017 09:53:40 GMT
Repository: kafka-site
Updated Branches:
  refs/heads/asf-site 6699e22d4 -> 0f672e80e


Updates to config documentation from 0.11.0 branch


Project: http://git-wip-us.apache.org/repos/asf/kafka-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka-site/commit/0f672e80
Tree: http://git-wip-us.apache.org/repos/asf/kafka-site/tree/0f672e80
Diff: http://git-wip-us.apache.org/repos/asf/kafka-site/diff/0f672e80

Branch: refs/heads/asf-site
Commit: 0f672e80e4085f91fd906a59511d004ff2f08a72
Parents: 6699e22
Author: Ismael Juma <ismael@juma.me.uk>
Authored: Wed Jun 28 10:52:45 2017 +0100
Committer: Ismael Juma <ismael@juma.me.uk>
Committed: Wed Jun 28 10:52:45 2017 +0100

----------------------------------------------------------------------
 0110/generated/consumer_config.html | 4 ++--
 0110/generated/kafka_config.html    | 6 +++---
 0110/generated/producer_config.html | 4 ++--
 0110/generated/streams_config.html  | 2 +-
 0110/generated/topic_config.html    | 6 +++---
 5 files changed, 11 insertions(+), 11 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka-site/blob/0f672e80/0110/generated/consumer_config.html
----------------------------------------------------------------------
diff --git a/0110/generated/consumer_config.html b/0110/generated/consumer_config.html
index 83e7b44..20cbce8 100644
--- a/0110/generated/consumer_config.html
+++ b/0110/generated/consumer_config.html
@@ -20,7 +20,7 @@
 <tr>
 <td>heartbeat.interval.ms</td><td>The expected time between heartbeats
to the consumer coordinator when using Kafka's group management facilities. Heartbeats are
used to ensure that the consumer's session stays active and to facilitate rebalancing when
new consumers join or leave the group. The value must be set lower than <code>session.timeout.ms</code>,
but typically should be set no higher than 1/3 of that value. It can be adjusted even lower
to control the expected time for normal rebalances.</td><td>int</td><td>3000</td><td></td><td>high</td></tr>
 <tr>
-<td>max.partition.fetch.bytes</td><td>The maximum amount of data per-partition
the server will return. If the first message in the first non-empty partition of the fetch
is larger than this limit, the message will still be returned to ensure that the consumer
can make progress. The maximum message size accepted by the broker is defined via <code>message.max.bytes</code>
(broker config) or <code>max.message.bytes</code> (topic config). See fetch.max.bytes
for limiting the consumer request size.</td><td>int</td><td>1048576</td><td>[0,...]</td><td>high</td></tr>
+<td>max.partition.fetch.bytes</td><td>The maximum amount of data per-partition
the server will return. Records are fetched in batches by the consumer. If the first record
batch in the first non-empty partition of the fetch is larger than this limit, the batch will
still be returned to ensure that the consumer can make progress. The maximum record batch
size accepted by the broker is defined via <code>message.max.bytes</code> (broker
config) or <code>max.message.bytes</code> (topic config). See fetch.max.bytes
for limiting the consumer request size.</td><td>int</td><td>1048576</td><td>[0,...]</td><td>high</td></tr>
 <tr>
 <td>session.timeout.ms</td><td>The timeout used to detect consumer failures
when using Kafka's group management facility. The consumer sends periodic heartbeats to indicate
its liveness to the broker. If no heartbeats are received by the broker before the expiration
of this session timeout, then the broker will remove this consumer from the group and initiate
a rebalance. Note that the value must be in the allowable range as configured in the broker
configuration by <code>group.min.session.timeout.ms</code> and <code>group.max.session.timeout.ms</code>.</td><td>int</td><td>10000</td><td></td><td>high</td></tr>
 <tr>
@@ -42,7 +42,7 @@
 <tr>
 <td>exclude.internal.topics</td><td>Whether records from internal topics
(such as offsets) should be exposed to the consumer. If set to <code>true</code>
the only way to receive records from an internal topic is subscribing to it.</td><td>boolean</td><td>true</td><td></td><td>medium</td></tr>
 <tr>
-<td>fetch.max.bytes</td><td>The maximum amount of data the server should
return for a fetch request. This is not an absolute maximum, if the first message in the first
non-empty partition of the fetch is larger than this value, the message will still be returned
to ensure that the consumer can make progress. The maximum message size accepted by the broker
is defined via <code>message.max.bytes</code> (broker config) or <code>max.message.bytes</code>
(topic config). Note that the consumer performs multiple fetches in parallel.</td><td>int</td><td>52428800</td><td>[0,...]</td><td>medium</td></tr>
+<td>fetch.max.bytes</td><td>The maximum amount of data the server should
return for a fetch request. Records are fetched in batches by the consumer, and if the first
record batch in the first non-empty partition of the fetch is larger than this value, the
record batch will still be returned to ensure that the consumer can make progress. As such,
this is not a absolute maximum.The maximum record batch size accepted by the broker is defined
via <code>message.max.bytes</code> (broker config) or <code>max.message.bytes</code>
(topic config). Note that the consumer performs multiple fetches in parallel.</td><td>int</td><td>52428800</td><td>[0,...]</td><td>medium</td></tr>
 <tr>
 <td>isolation.level</td><td><p>Controls how to read messages written
transactionally. If set to <code>read_committed</code>, consumer.poll() will only
return transactional messages which have been committed. If set to <code>read_uncommitted</code>'
(the default), consumer.poll() will return all messages, even transactional messages which
have been aborted. Non-transactional messages will be returned unconditionally in either mode.</p>
<p>Messages will always be returned in offset order. Hence, in  <code>read_committed</code>
mode, consumer.poll() will only return messages up to the last stable offset (LSO), which
is the one less than the offset of the first open transaction. In particular any messages
appearing after messages belonging to ongoing transactions will be withheld until the relevant
transaction has been completed. As a result, <code>read_committed</code> consumers
will not be able to read up to the high watermark when there are in flight transactions.</p><p>
Further, whe
 n in <code>read_committed</mode> the seekToEnd method will return the LSO</td><td>string</td><td>read_uncommitted</td><td>[read_committed,
read_uncommitted]</td><td>medium</td></tr>
 <tr>

http://git-wip-us.apache.org/repos/asf/kafka-site/blob/0f672e80/0110/generated/kafka_config.html
----------------------------------------------------------------------
diff --git a/0110/generated/kafka_config.html b/0110/generated/kafka_config.html
index 7046ca3..8ea1021 100644
--- a/0110/generated/kafka_config.html
+++ b/0110/generated/kafka_config.html
@@ -79,7 +79,7 @@ hostname of broker. If this is set, it will only bind to this address. If
this i
 <tr>
 <td>log.segment.delete.delay.ms</td><td>The amount of time to wait before
deleting a file from the filesystem</td><td>long</td><td>60000</td><td>[0,...]</td><td>high</td></tr>
 <tr>
-<td>message.max.bytes</td><td>The maximum message size that the server
can receive. Note that this limit also applies to the total size of a compressed batch of
messages (when compression is enabled). Additionally, in versions 0.11 and later, all messages
are written as batches and this setting applies to the total size of the batch.</td><td>int</td><td>1000012</td><td>[0,...]</td><td>high</td></tr>
+<td>message.max.bytes</td><td><p>The largest record batch size allowed
by Kafka. If this is increased and there are consumers older than 0.10.2, the consumers' fetch
size must also be increased so that the they can fetch record batches this large.</p><p>In
the latest message format version, records are always grouped into batches for efficiency.
In previous message format versions, uncompressed records are not grouped into batches and
this limit only applies to asingle record in that case.</p><p>This can be set
per topic with the topic level <code>max.message.bytes</code> config.</p></td><td>int</td><td>1000012</td><td>[0,...]</td><td>high</td></tr>
 <tr>
 <td>min.insync.replicas</td><td>When a producer sets acks to "all" (or
"-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge
a write for the write to be considered successful. If this minimum cannot be met, then the
producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend).<br>When
used together, min.insync.replicas and acks allow you to enforce greater durability guarantees.
A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas
to 2, and produce with acks of "all". This will ensure that the producer raises an exception
if a majority of replicas do not receive a write.</td><td>int</td><td>1</td><td>[1,...]</td><td>high</td></tr>
 <tr>
@@ -242,9 +242,9 @@ the port to listen and accept connections on</td><td>int</td><td>9092</td><td></
 <tr>
 <td>replica.fetch.backoff.ms</td><td>The amount of time to sleep when fetch
partition error occurs.</td><td>int</td><td>1000</td><td>[0,...]</td><td>medium</td></tr>
 <tr>
-<td>replica.fetch.max.bytes</td><td>The number of bytes of messages to
attempt to fetch for each partition. This is not an absolute maximum, if the first message
in the first non-empty partition of the fetch is larger than this value, the message will
still be returned to ensure that progress can be made. The maximum message size accepted by
the broker is defined via <code>message.max.bytes</code> (broker config) or <code>max.message.bytes</code>
(topic config).</td><td>int</td><td>1048576</td><td>[0,...]</td><td>medium</td></tr>
+<td>replica.fetch.max.bytes</td><td>The number of bytes of messages to
attempt to fetch for each partition. This is not an absolute maximum, if the first record
batch in the first non-empty partition of the fetch is larger than this value, the record
batch will still be returned to ensure that progress can be made. The maximum record batch
size accepted by the broker is defined via <code>message.max.bytes</code> (broker
config) or <code>max.message.bytes</code> (topic config).</td><td>int</td><td>1048576</td><td>[0,...]</td><td>medium</td></tr>
 <tr>
-<td>replica.fetch.response.max.bytes</td><td>Maximum bytes expected for
the entire fetch response. This is not an absolute maximum, if the first message in the first
non-empty partition of the fetch is larger than this value, the message will still be returned
to ensure that progress can be made. The maximum message size accepted by the broker is defined
via <code>message.max.bytes</code> (broker config) or <code>max.message.bytes</code>
(topic config).</td><td>int</td><td>10485760</td><td>[0,...]</td><td>medium</td></tr>
+<td>replica.fetch.response.max.bytes</td><td>Maximum bytes expected for
the entire fetch response. Records are fetched in batches, and if the first record batch in
the first non-empty partition of the fetch is larger than this value, the record batch will
still be returned to ensure that progress can be made. As such, this is not an absolute maximum.
The maximum record batch size accepted by the broker is defined via <code>message.max.bytes</code>
(broker config) or <code>max.message.bytes</code> (topic config).</td><td>int</td><td>10485760</td><td>[0,...]</td><td>medium</td></tr>
 <tr>
 <td>reserved.broker.max.id</td><td>Max number that can be used for a broker.id</td><td>int</td><td>1000</td><td>[0,...]</td><td>medium</td></tr>
 <tr>

http://git-wip-us.apache.org/repos/asf/kafka-site/blob/0f672e80/0110/generated/producer_config.html
----------------------------------------------------------------------
diff --git a/0110/generated/producer_config.html b/0110/generated/producer_config.html
index 215d1c0..819dfbc 100644
--- a/0110/generated/producer_config.html
+++ b/0110/generated/producer_config.html
@@ -42,7 +42,7 @@
 <tr>
 <td>max.block.ms</td><td>The configuration controls how long <code>KafkaProducer.send()</code>
and <code>KafkaProducer.partitionsFor()</code> will block.These methods can be
blocked either because the buffer is full or metadata unavailable.Blocking in the user-supplied
serializers or partitioner will not be counted against this timeout.</td><td>long</td><td>60000</td><td>[0,...]</td><td>medium</td></tr>
 <tr>
-<td>max.request.size</td><td>The maximum size of a request in bytes. This
is also effectively a cap on the maximum record size. Note that the server has its own cap
on record size which may be different from this. This setting will limit the number of record
batches the producer will send in a single request to avoid sending huge requests.</td><td>int</td><td>1048576</td><td>[0,...]</td><td>medium</td></tr>
+<td>max.request.size</td><td>The maximum size of a request in bytes. This
setting will limit the number of record batches the producer will send in a single request
to avoid sending huge requests. This is also effectively a cap on the maximum record batch
size. Note that the server has its own cap on record batch size which may be different from
this.</td><td>int</td><td>1048576</td><td>[0,...]</td><td>medium</td></tr>
 <tr>
 <td>partitioner.class</td><td>Partitioner class that implements the <code>Partitioner</code>
interface.</td><td>class</td><td>org.apache.kafka.clients.producer.internals.DefaultPartitioner</td><td></td><td>medium</td></tr>
 <tr>
@@ -112,5 +112,5 @@
 <tr>
 <td>transaction.timeout.ms</td><td>The maximum amount of time in ms that
the transaction coordinator will wait for a transaction status update from the producer before
proactively aborting the ongoing transaction.If this value is larger than the max.transaction.timeout.ms
setting in the broker, the request will fail with a `InvalidTransactionTimeout` error.</td><td>int</td><td>60000</td><td></td><td>low</td></tr>
 <tr>
-<td>transactional.id</td><td>The TransactionalId to use for transactional
delivery. This enables reliability semantics which span multiple producer sessions since it
allows the client to guarantee that transactions using the same TransactionalId have been
completed prior to starting any new transactions. If no TransactionalId is provided, then
the producer is limited to idempotent delivery. Note that enable.idempotence must be enabled
if a TransactionalId is configured. The default is empty, which means transactions cannot
be used.</td><td>string</td><td>null</td><td>org.apache.kafka.common.config.ConfigDef$NonEmptyString@6dffeaea</td><td>low</td></tr>
+<td>transactional.id</td><td>The TransactionalId to use for transactional
delivery. This enables reliability semantics which span multiple producer sessions since it
allows the client to guarantee that transactions using the same TransactionalId have been
completed prior to starting any new transactions. If no TransactionalId is provided, then
the producer is limited to idempotent delivery. Note that enable.idempotence must be enabled
if a TransactionalId is configured. The default is empty, which means transactions cannot
be used.</td><td>string</td><td>null</td><td>org.apache.kafka.common.config.ConfigDef$NonEmptyString@4fca772d</td><td>low</td></tr>
 </tbody></table>

http://git-wip-us.apache.org/repos/asf/kafka-site/blob/0f672e80/0110/generated/streams_config.html
----------------------------------------------------------------------
diff --git a/0110/generated/streams_config.html b/0110/generated/streams_config.html
index 53e229b..dad8110 100644
--- a/0110/generated/streams_config.html
+++ b/0110/generated/streams_config.html
@@ -80,5 +80,5 @@
 <tr>
 <td>windowstore.changelog.additional.retention.ms</td><td>Added to a windows
maintainMs to ensure data is not deleted from the log prematurely. Allows for clock drift.
Default is 1 day</td><td>long</td><td>86400000</td><td></td><td>low</td></tr>
 <tr>
-<td>zookeeper.connect</td><td>Zookeeper connect string for Kafka topics
management.</td><td>string</td><td>""</td><td></td><td>low</td></tr>
+<td>zookeeper.connect</td><td>Zookeeper connect string for Kafka topics
management. This config is deprecated and will be ignored as Streams API does not use Zookeeper
anymore.</td><td>string</td><td>""</td><td></td><td>low</td></tr>
 </tbody></table>

http://git-wip-us.apache.org/repos/asf/kafka-site/blob/0f672e80/0110/generated/topic_config.html
----------------------------------------------------------------------
diff --git a/0110/generated/topic_config.html b/0110/generated/topic_config.html
index 1653db7..ebf03a1 100644
--- a/0110/generated/topic_config.html
+++ b/0110/generated/topic_config.html
@@ -21,13 +21,13 @@
 <tr>
 <td>flush.ms</td><td>This setting allows specifying a time interval at
which we will force an fsync of data written to the log. For example if this was set to 1000
we would fsync after 1000 ms had passed. In general we recommend you not set this and use
replication for durability and allow the operating system's background flush capabilities
as it is more efficient.</td><td>long</td><td>9223372036854775807</td><td>[0,...]</td><td>log.flush.interval.ms</td><td>medium</td></tr>
 <tr>
-<td>follower.replication.throttled.replicas</td><td>A list of replicas
for which log replication should be throttled on the follower side. The list should describe
a set of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively
the wildcard '*' can be used to throttle all replicas for this topic.</td><td>list</td><td>""</td><td>kafka.server.ThrottledReplicaListValidator$@10e76514</td><td>follower.replication.throttled.replicas</td><td>medium</td></tr>
+<td>follower.replication.throttled.replicas</td><td>A list of replicas
for which log replication should be throttled on the follower side. The list should describe
a set of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively
the wildcard '*' can be used to throttle all replicas for this topic.</td><td>list</td><td>""</td><td>kafka.server.ThrottledReplicaListValidator$@52feb982</td><td>follower.replication.throttled.replicas</td><td>medium</td></tr>
 <tr>
 <td>index.interval.bytes</td><td>This setting controls how frequently Kafka
adds an index entry to it's offset index. The default setting ensures that we index a message
roughly every 4096 bytes. More indexing allows reads to jump closer to the exact position
in the log but makes the index larger. You probably don't need to change this.</td><td>int</td><td>4096</td><td>[0,...]</td><td>log.index.interval.bytes</td><td>medium</td></tr>
 <tr>
-<td>leader.replication.throttled.replicas</td><td>A list of replicas for
which log replication should be throttled on the leader side. The list should describe a set
of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively
the wildcard '*' can be used to throttle all replicas for this topic.</td><td>list</td><td>""</td><td>kafka.server.ThrottledReplicaListValidator$@10e76514</td><td>leader.replication.throttled.replicas</td><td>medium</td></tr>
+<td>leader.replication.throttled.replicas</td><td>A list of replicas for
which log replication should be throttled on the leader side. The list should describe a set
of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively
the wildcard '*' can be used to throttle all replicas for this topic.</td><td>list</td><td>""</td><td>kafka.server.ThrottledReplicaListValidator$@52feb982</td><td>leader.replication.throttled.replicas</td><td>medium</td></tr>
 <tr>
-<td>max.message.bytes</td><td>This is largest message size Kafka will allow
to be appended. Note that if you increase this size you must also increase your consumer's
fetch size so they can fetch messages this large.</td><td>int</td><td>1000012</td><td>[0,...]</td><td>message.max.bytes</td><td>medium</td></tr>
+<td>max.message.bytes</td><td><p>The largest record batch size allowed
by Kafka. If this is increased and there are consumers older than 0.10.2, the consumers' fetch
size must also be increased so that the they can fetch record batches this large.</p><p>In
the latest message format version, records are always grouped into batches for efficiency.
In previous message format versions, uncompressed records are not grouped into batches and
this limit only applies to asingle record in that case.</p></td><td>int</td><td>1000012</td><td>[0,...]</td><td>message.max.bytes</td><td>medium</td></tr>
 <tr>
 <td>message.format.version</td><td>Specify the message format version the
broker will use to append messages to the logs. The value should be a valid ApiVersion. Some
examples are: 0.8.2, 0.9.0.0, 0.10.0, check ApiVersion for more details. By setting a particular
message format version, the user is certifying that all the existing messages on disk are
smaller or equal than the specified version. Setting this value incorrectly will cause consumers
with older versions to break as they will receive messages with a format that they don't understand.</td><td>string</td><td>0.11.0-IV2</td><td></td><td>log.message.format.version</td><td>medium</td></tr>
 <tr>


Mime
View raw message