kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mj...@apache.org
Subject [kafka-site] branch asf-site updated: KAFKA-8913 : Document topic based configs & ISR settings for Streams (#231)
Date Sat, 21 Sep 2019 16:59:13 GMT
This is an automated email from the ASF dual-hosted git repository.

mjsax pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/kafka-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new 2423a37  KAFKA-8913 : Document topic based configs & ISR settings for Streams
(#231)
2423a37 is described below

commit 2423a37678cc05e5b9560555f5ea9f8c4eedd3d3
Author: vinoth chandar <vinothchandar@users.noreply.github.com>
AuthorDate: Sat Sep 21 09:59:08 2019 -0700

    KAFKA-8913 : Document topic based configs & ISR settings for Streams (#231)
    
    - Documented how to set topic based configs
    - Documented how to achieve MIN_IN_SYNC_REPLICAS_CONFIG = 2 for reliable apps
    - cf https://github.com/apache/kafka/pull/7346
    
    Reviewer: Matthias J. Sax <matthias@confluent.io>
---
 20/streams/developer-guide/config-streams.html | 11 +++++++++++
 21/streams/developer-guide/config-streams.html | 11 +++++++++++
 22/streams/developer-guide/config-streams.html | 11 +++++++++++
 23/streams/developer-guide/config-streams.html | 11 +++++++++++
 4 files changed, 44 insertions(+)

diff --git a/20/streams/developer-guide/config-streams.html b/20/streams/developer-guide/config-streams.html
index b58661c..48f5bcd 100644
--- a/20/streams/developer-guide/config-streams.html
+++ b/20/streams/developer-guide/config-streams.html
@@ -611,6 +611,16 @@
           </div>
           <p> Same applied to <code class="docutils literal"><span class="pre">main.consumer.</span></code>
and <code class="docutils literal"><span class="pre">main.consumer.</span></code>,
if you only want to specify one consumer type config.</p>
           </div>
+
+          <p> Additionally, to configure the internal repartition/changelog topics,
you could use the <code class="docutils literal"><span class="pre">topic.</span></code>
prefix, followed by any of the standard topic configs.</p>
+          <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">Properties</span> <span class="n">streamsSettings</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span
class="o">();</span>
+<span class="c1">// Override default for both changelog and repartition topics</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="s">&quot;topic.PARAMETER_NAME&quot;</span><span
class="o">,</span> <span class="s">&quot;topic-value&quot;</span><span
class="o">);</span>
+<span class="c1">// alternatively, you can use</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">topicPrefix</span><span class="o">(</span><span
class="s">&quot;PARAMETER_NAME&quot;</span><span class="o">),</span>
<span class="s">&quot;topic-value&quot;</span><span class="o">);</span>
+</pre></div>
+          </div>
+
         </div>
         <div class="section" id="default-values">
           <h4><a class="toc-backref" href="#id18">Default Values</a><a
class="headerlink" href="#default-values" title="Permalink to this headline"></a></h4>
@@ -762,6 +772,7 @@
             <div>See the <a class="reference internal" href="#replication-factor-parm"><span
class="std std-ref">description here</span></a>.</div></blockquote>
           <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">Properties</span> <span class="n">streamsSettings</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span
class="o">();</span>
 <span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">REPLICATION_FACTOR_CONFIG</span><span
class="o">,</span> <span class="mi">3</span><span class="o">);</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">topicPrefix</span><span class="o">(</span><span
class="n">TopicConfig</span><span class="o">.</span><span class="na">MIN_IN_SYNC_REPLICAS_CONFIG</span><span
class="o">),</span> <span class="mi">2</span><span class="o">);</span>
 <span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">producerPrefix</span><span class="o">(</span><span
class="n">ProducerConfig</span><span class="o">.</span><span class="na">ACKS_CONFIG</span><span
class="o">),</span> <span class="s">&quot;all&quot;</span><span
class="o">);</span>
 </pre></div>
           </div>
diff --git a/21/streams/developer-guide/config-streams.html b/21/streams/developer-guide/config-streams.html
index d143752..c8604c6 100644
--- a/21/streams/developer-guide/config-streams.html
+++ b/21/streams/developer-guide/config-streams.html
@@ -616,6 +616,16 @@
           </div>
           <p> Same applied to <code class="docutils literal"><span class="pre">main.consumer.</span></code>
and <code class="docutils literal"><span class="pre">main.consumer.</span></code>,
if you only want to specify one consumer type config.</p>
           </div>
+
+          <p> Additionally, to configure the internal repartition/changelog topics,
you could use the <code class="docutils literal"><span class="pre">topic.</span></code>
prefix, followed by any of the standard topic configs.</p>
+          <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">Properties</span> <span class="n">streamsSettings</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span
class="o">();</span>
+<span class="c1">// Override default for both changelog and repartition topics</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="s">&quot;topic.PARAMETER_NAME&quot;</span><span
class="o">,</span> <span class="s">&quot;topic-value&quot;</span><span
class="o">);</span>
+<span class="c1">// alternatively, you can use</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">topicPrefix</span><span class="o">(</span><span
class="s">&quot;PARAMETER_NAME&quot;</span><span class="o">),</span>
<span class="s">&quot;topic-value&quot;</span><span class="o">);</span>
+</pre></div>
+          </div>
+
         </div>
         <div class="section" id="default-values">
           <h4><a class="toc-backref" href="#id18">Default Values</a><a
class="headerlink" href="#default-values" title="Permalink to this headline"></a></h4>
@@ -763,6 +773,7 @@
             <div>See the <a class="reference internal" href="#replication-factor-parm"><span
class="std std-ref">description here</span></a>.</div></blockquote>
           <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">Properties</span> <span class="n">streamsSettings</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span
class="o">();</span>
 <span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">REPLICATION_FACTOR_CONFIG</span><span
class="o">,</span> <span class="mi">3</span><span class="o">);</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">topicPrefix</span><span class="o">(</span><span
class="n">TopicConfig</span><span class="o">.</span><span class="na">MIN_IN_SYNC_REPLICAS_CONFIG</span><span
class="o">),</span> <span class="mi">2</span><span class="o">);</span>
 <span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">producerPrefix</span><span class="o">(</span><span
class="n">ProducerConfig</span><span class="o">.</span><span class="na">ACKS_CONFIG</span><span
class="o">),</span> <span class="s">&quot;all&quot;</span><span
class="o">);</span>
 </pre></div>
           </div>
diff --git a/22/streams/developer-guide/config-streams.html b/22/streams/developer-guide/config-streams.html
index d143752..c8604c6 100644
--- a/22/streams/developer-guide/config-streams.html
+++ b/22/streams/developer-guide/config-streams.html
@@ -616,6 +616,16 @@
           </div>
           <p> Same applied to <code class="docutils literal"><span class="pre">main.consumer.</span></code>
and <code class="docutils literal"><span class="pre">main.consumer.</span></code>,
if you only want to specify one consumer type config.</p>
           </div>
+
+          <p> Additionally, to configure the internal repartition/changelog topics,
you could use the <code class="docutils literal"><span class="pre">topic.</span></code>
prefix, followed by any of the standard topic configs.</p>
+          <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">Properties</span> <span class="n">streamsSettings</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span
class="o">();</span>
+<span class="c1">// Override default for both changelog and repartition topics</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="s">&quot;topic.PARAMETER_NAME&quot;</span><span
class="o">,</span> <span class="s">&quot;topic-value&quot;</span><span
class="o">);</span>
+<span class="c1">// alternatively, you can use</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">topicPrefix</span><span class="o">(</span><span
class="s">&quot;PARAMETER_NAME&quot;</span><span class="o">),</span>
<span class="s">&quot;topic-value&quot;</span><span class="o">);</span>
+</pre></div>
+          </div>
+
         </div>
         <div class="section" id="default-values">
           <h4><a class="toc-backref" href="#id18">Default Values</a><a
class="headerlink" href="#default-values" title="Permalink to this headline"></a></h4>
@@ -763,6 +773,7 @@
             <div>See the <a class="reference internal" href="#replication-factor-parm"><span
class="std std-ref">description here</span></a>.</div></blockquote>
           <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">Properties</span> <span class="n">streamsSettings</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span
class="o">();</span>
 <span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">REPLICATION_FACTOR_CONFIG</span><span
class="o">,</span> <span class="mi">3</span><span class="o">);</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">topicPrefix</span><span class="o">(</span><span
class="n">TopicConfig</span><span class="o">.</span><span class="na">MIN_IN_SYNC_REPLICAS_CONFIG</span><span
class="o">),</span> <span class="mi">2</span><span class="o">);</span>
 <span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">producerPrefix</span><span class="o">(</span><span
class="n">ProducerConfig</span><span class="o">.</span><span class="na">ACKS_CONFIG</span><span
class="o">),</span> <span class="s">&quot;all&quot;</span><span
class="o">);</span>
 </pre></div>
           </div>
diff --git a/23/streams/developer-guide/config-streams.html b/23/streams/developer-guide/config-streams.html
index 895cb72..922c6c9 100644
--- a/23/streams/developer-guide/config-streams.html
+++ b/23/streams/developer-guide/config-streams.html
@@ -616,6 +616,16 @@
           </div>
           <p> Same applied to <code class="docutils literal"><span class="pre">main.consumer.</span></code>
and <code class="docutils literal"><span class="pre">main.consumer.</span></code>,
if you only want to specify one consumer type config.</p>
           </div>
+
+          <p> Additionally, to configure the internal repartition/changelog topics,
you could use the <code class="docutils literal"><span class="pre">topic.</span></code>
prefix, followed by any of the standard topic configs.</p>
+          <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">Properties</span> <span class="n">streamsSettings</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span
class="o">();</span>
+<span class="c1">// Override default for both changelog and repartition topics</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="s">&quot;topic.PARAMETER_NAME&quot;</span><span
class="o">,</span> <span class="s">&quot;topic-value&quot;</span><span
class="o">);</span>
+<span class="c1">// alternatively, you can use</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">topicPrefix</span><span class="o">(</span><span
class="s">&quot;PARAMETER_NAME&quot;</span><span class="o">),</span>
<span class="s">&quot;topic-value&quot;</span><span class="o">);</span>
+</pre></div>
+          </div>
+
         </div>
         <div class="section" id="default-values">
           <h4><a class="toc-backref" href="#id18">Default Values</a><a
class="headerlink" href="#default-values" title="Permalink to this headline"></a></h4>
@@ -778,6 +788,7 @@
             <div>See the <a class="reference internal" href="#replication-factor-parm"><span
class="std std-ref">description here</span></a>.</div></blockquote>
           <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">Properties</span> <span class="n">streamsSettings</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span
class="o">();</span>
 <span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">REPLICATION_FACTOR_CONFIG</span><span
class="o">,</span> <span class="mi">3</span><span class="o">);</span>
+<span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">topicPrefix</span><span class="o">(</span><span
class="n">TopicConfig</span><span class="o">.</span><span class="na">MIN_IN_SYNC_REPLICAS_CONFIG</span><span
class="o">),</span> <span class="mi">2</span><span class="o">);</span>
 <span class="n">streamsSettings</span><span class="o">.</span><span
class="na">put</span><span class="o">(</span><span class="n">StreamsConfig</span><span
class="o">.</span><span class="na">producerPrefix</span><span class="o">(</span><span
class="n">ProducerConfig</span><span class="o">.</span><span class="na">ACKS_CONFIG</span><span
class="o">),</span> <span class="s">&quot;all&quot;</span><span
class="o">);</span>
 </pre></div>
           </div>


Mime
View raw message