kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guozh...@apache.org
Subject [kafka] branch trunk updated: HOTFIX: Fix broken links (#5676)
Date Thu, 04 Oct 2018 01:28:42 GMT
This is an automated email from the ASF dual-hosted git repository.

guozhang pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/trunk by this push:
     new 862e09f  HOTFIX: Fix broken links (#5676)
862e09f is described below

commit 862e09f994e329a4f56f597ab002db9946fa9523
Author: Bill Bejeck <bbejeck@gmail.com>
AuthorDate: Wed Oct 3 21:27:15 2018 -0400

    HOTFIX: Fix broken links (#5676)
    
    Reviewers: Joel Hamill <11722533+joel-hamill@users.noreply.github.com>, Guozhang
Wang <wangguoz@gmail.com>
---
 docs/streams/core-concepts.html                 |  48 ++++++++
 docs/streams/developer-guide/dsl-api.html       | 147 ++++++++++++++++++++----
 docs/streams/developer-guide/processor-api.html |   4 +-
 docs/streams/upgrade-guide.html                 |   2 +-
 4 files changed, 178 insertions(+), 23 deletions(-)

diff --git a/docs/streams/core-concepts.html b/docs/streams/core-concepts.html
index 015fbb4..bffaa8a 100644
--- a/docs/streams/core-concepts.html
+++ b/docs/streams/core-concepts.html
@@ -131,6 +131,54 @@
 	Note, that the describe default behavior can be changed in the Processor API by assigning
timestamps to output records explicitly when calling <code>#forward()</code>.
     </p>
 
+    <h3><a id="streams_concepts_aggregations" href="#streams_concepts_aggregations">Aggregations</a></h3>
+    <p>
+        An <strong>aggregation</strong> operation takes one input stream or table,
and yields a new table by combining multiple input records into a single output record. Examples
of aggregations are computing counts or sum.
+    </p>
+
+    <p>
+        In the <code>Kafka Streams DSL</code>, an input stream of an <code>aggregation</code>
can be a KStream or a KTable, but the output stream will always be a KTable. This allows Kafka
Streams to update an aggregate value upon the late arrival of further records after the value
was produced and emitted. When such late arrival happens, the aggregating KStream or KTable
emits a new aggregate value. Because the output is a KTable, the new value is considered to
overwrite the old value w [...]
+    </p>
+
+    <h3> <a id="streams_concepts_windowing" href="#streams_concepts_windowing">Windowing</a></h3>
+    <p>
+        Windowing lets you control how to <em>group records that have the same key</em>
for stateful operations such as <code>aggregations</code> or <code>joins</code>
into so-called <em>windows</em>. Windows are tracked per record key.
+    </p>
+    <p>
+        <code>Windowing operations</code> are available in the <code>Kafka
Streams DSL</code>. When working with windows, you can specify a <strong>retention
period</strong> for the window. This retention period controls how long Kafka Streams
will wait for <strong>out-of-order</strong> or <strong>late-arriving</strong>
data records for a given window. If a record arrives after the retention period of a window
has passed, the record is discarded and will not be processed in that window.
+    </p>
+    <p>
+        Late-arriving records are always possible in the real world and should be properly
accounted for in your applications. It depends on the effective <code>time semantics
</code> how late records are handled. In the case of processing-time, the semantics
are &quot;when the record is being processed&quot;, which means that the notion of
late records is not applicable as, by definition, no record can be late. Hence, late-arriving
records can only be considered as such (i.e. as arrivin [...]
+    </p>
+
+    <h3><a id="streams_concepts_duality" href="#streams-concepts-duality">Duality
of Streams and Tables</a></h3>
+    <p>
+        When implementing stream processing use cases in practice, you typically need both
<strong>streams</strong> and also <strong>databases</strong>.
+        An example use case that is very common in practice is an e-commerce application
that enriches an incoming <em>stream</em> of customer
+        transactions with the latest customer information from a <em>database table</em>.
In other words, streams are everywhere, but databases are everywhere, too.
+    </p>
+
+    <p>
+        Any stream processing technology must therefore provide <strong>first-class
support for streams and tables</strong>.
+        Kafka's Streams API provides such functionality through its core abstractions for
+        <code class="interpreted-text" data-role="ref">streams &lt;streams_concepts_kstream&gt;</code>
and
+        <code class="interpreted-text" data-role="ref">tables &lt;streams_concepts_ktable&gt;</code>,
which we will talk about in a minute.
+        Now, an interesting observation is that there is actually a <strong>close relationship
between streams and tables</strong>,
+        the so-called stream-table duality.
+        And Kafka exploits this duality in many ways: for example, to make your applications
+        <code class="interpreted-text" data-role="ref">elastic &lt;streams_developer-guide_execution-scaling&gt;</code>,
+        to support <code class="interpreted-text" data-role="ref">fault-tolerant stateful
processing &lt;streams_developer-guide_state-store_fault-tolerance&gt;</code>,
+        or to run <code class="interpreted-text" data-role="ref">interactive queries
&lt;streams_concepts_interactive-queries&gt;</code>
+        against your application's latest processing results. And, beyond its internal usage,
the Kafka Streams API
+        also allows developers to exploit this duality in their own applications.
+    </p>
+
+    <p>
+        Before we discuss concepts such as <code class="interpreted-text" data-role="ref">aggregations
&lt;streams_concepts_aggregations&gt;</code>
+        in Kafka Streams we must first introduce <strong>tables</strong> in more
detail, and talk about the aforementioned stream-table duality.
+        Essentially, this duality means that a stream can be viewed as a table, and a table
can be viewed as a stream.
+    </p>
+
     <h3><a id="streams_state" href="#streams_state">States</a></h3>
 
     <p>
diff --git a/docs/streams/developer-guide/dsl-api.html b/docs/streams/developer-guide/dsl-api.html
index 01931c8..1622702 100644
--- a/docs/streams/developer-guide/dsl-api.html
+++ b/docs/streams/developer-guide/dsl-api.html
@@ -79,9 +79,9 @@
             <h2><a class="toc-backref" href="#id7">Overview</a><a class="headerlink"
href="#overview" title="Permalink to this headline"></a></h2>
             <p>In comparison to the <a class="reference internal" href="processor-api.html#streams-developer-guide-processor-api"><span
class="std std-ref">Processor API</span></a>, only the DSL supports:</p>
             <ul class="simple">
-                <li>Built-in abstractions for <a class="reference internal" href="../concepts.html#streams-concepts-duality"><span
class="std std-ref">streams and tables</span></a> in the form of
-                    <a class="reference internal" href="../concepts.html#streams-concepts-kstream"><span
class="std std-ref">KStream</span></a>, <a class="reference internal" href="../concepts.html#streams-concepts-ktable"><span
class="std std-ref">KTable</span></a>, and
-                    <a class="reference internal" href="../concepts.html#streams-concepts-globalktable"><span
class="std std-ref">GlobalKTable</span></a>.  Having first-class support for
streams and tables is crucial
+                <li>Built-in abstractions for <a class="reference internal" href="../core-concepts.html#streams_concepts_duality"><span
class="std std-ref">streams and tables</span></a> in the form of
+                    <a class="reference internal" href="#streams_concepts_kstream"><span
class="std std-ref">KStream</span></a>, <a class="reference internal" href="#streams_concepts_ktable"><span
class="std std-ref">KTable</span></a>, and
+                    <a class="reference internal" href="#streams_concepts_globalktable"><span
class="std std-ref">GlobalKTable</span></a>.  Having first-class support for
streams and tables is crucial
                     because, in practice, most use cases require not just either streams
or databases/tables, but a combination of both.
                     For example, if your use case is to create a customer 360-degree view
that is updated in real-time, what your
                     application will be doing is transforming many input <em>streams</em>
of customer-related events into an output <em>table</em>
@@ -93,7 +93,7 @@
                     <a class="reference internal" href="#streams-developer-guide-dsl-joins"><span
class="std std-ref">joins</span></a> (e.g. <code class="docutils literal"><span
class="pre">leftJoin</span></code>), and
                     <a class="reference internal" href="#streams-developer-guide-dsl-windowing"><span
class="std std-ref">windowing</span></a> (e.g. <a class="reference internal"
href="#windowing-session"><span class="std std-ref">session windows</span></a>).</li>
             </ul>
-            <p>With the DSL, you can define <a class="reference internal" href="../concepts.html#streams-concepts-processor-topology"><span
class="std std-ref">processor topologies</span></a> (i.e., the logical
+            <p>With the DSL, you can define <a class="reference internal" href="../core-concepts.html#streams_topology"><span
class="std std-ref">processor topologies</span></a> (i.e., the logical
                 processing plan) in your application. The steps to accomplish this are:</p>
             <ol class="arabic simple">
                 <li>Specify <a class="reference internal" href="#streams-developer-guide-dsl-sources"><span
class="std std-ref">one or more input streams that are read from Kafka topics</span></a>.</li>
@@ -104,6 +104,113 @@
                 action). A step-by-step guide for writing a stream processing application
using the DSL is provided below.</p>
             <p>For a complete list of available API functionality, see also the <a
href="../../../javadoc/org/apache/kafka/streams/package-summary.html">Streams</a>
API docs.</p>
         </div>
+
+        <div class="section" id="dsl-core-constructs-overview">
+            <h4><a id="streams_concepts_kstream" href="#streams_concepts_kstream">KStream</a></h4>
+
+            <p>
+                Only the <strong>Kafka Streams DSL</strong> has the notion of
a <code>KStream</code>.
+            </p>
+
+            <p>
+                A <strong>KStream</strong> is an abstraction of a <strong>record
stream</strong>, where each data record represents a self-contained datum in the unbounded
data set. Using the table analogy, data records in a record stream are always interpreted
as an &quot;INSERT&quot; -- think: adding more entries to an append-only ledger --
because no record replaces an existing row with the same key. Examples are a credit card transaction,
a page view event, or a server log entry.
+            </p>
+
+            <p>
+                To illustrate, let's imagine the following two data records are being sent
to the stream:
+            </p>
+
+            <div class="sourcecode">
+                <p>(&quot;alice&quot;, 1) --&gt; (&quot;alice&quot;,
3)</p>
+            </div>
+
+            <p>
+                If your stream processing application were to sum the values per user, it
would return <code>4</code> for <code>alice</code>. Why? Because the
second data record would not be considered an update of the previous record. Compare this
behavior of KStream to <code>KTable</code> below,
+                which would return <code>3</code> for <code>alice</code>.
+            </p>
+
+            <h4><a id="streams_concepts_ktable" href="#streams_concepts_ktable">KTable</a></h4>
+
+            <p>
+                Only the <strong>Kafka Streams DSL</strong> has the notion of
a <code>KTable</code>.
+            </p>
+
+            <p>
+                A <strong>KTable</strong> is an abstraction of a <strong>changelog
stream</strong>, where each data record represents an update. More precisely, the value
in a data record is interpreted as an &quot;UPDATE&quot; of the last value for the
same record key, if any (if a corresponding key doesn't exist yet, the update will be considered
an INSERT). Using the table analogy, a data record in a changelog stream is interpreted as
an UPSERT aka INSERT/UPDATE because any existing r [...]
+            </p>
+            <p>
+                To illustrate, let's imagine the following two data records are being sent
to the stream:
+            </p>
+
+            <div class="sourcecode">
+                <p>
+                    (&quot;alice&quot;, 1) --&gt; (&quot;alice&quot;,
3)
+                </p>
+            </div>
+
+            <p>
+                If your stream processing application were to sum the values per user, it
would return <code>3</code> for <code>alice</code>. Why? Because the
second data record would be considered an update of the previous record.
+            </p>
+
+            <p>
+                <strong>Effects of Kafka's log compaction:</strong> Another way
of thinking about KStream and KTable is as follows: If you were to store a KTable into a Kafka
topic, you'd probably want to enable Kafka's <a href="http://kafka.apache.org/documentation.html#compaction">log
compaction</a> feature, e.g. to save storage space.
+            </p>
+
+            <p>
+                However, it would not be safe to enable log compaction in the case of a KStream
because, as soon as log compaction would begin purging older data records of the same key,
it would break the semantics of the data. To pick up the illustration example again, you'd
suddenly get a <code>3</code> for <code>alice</code> instead of a
<code>4</code> because log compaction would have removed the <code>(&quot;alice&quot;,
1)</code> data record. Hence log compaction is perfectly safe [...]
+            </p>
+
+            <p>
+                We have already seen an example of a changelog stream in the section <strong>streams_concepts_duality</strong>.
Another example are change data capture (CDC) records in the changelog of a relational database,
representing which row in a database table was inserted, updated, or deleted.
+            </p>
+
+            <p>
+                KTable also provides an ability to look up <em>current</em> values
of data records by keys. This table-lookup functionality is available through <strong>join
operations</strong> (see also <strong>Joining</strong> in the Developer
Guide) as well as through <strong>Interactive Queries</strong>.
+            </p>
+
+            <h4><a id="streams_concepts_globalktable" href="#streams_concepts_globalktable">GlobalKTable</a></h4>
+
+            <p>Only the <strong>Kafka Streams DSL</strong> has the notion
of a <strong>GlobalKTable</strong>.</p>
+
+            <p>
+                Like a <strong>KTable</strong>, a <strong>GlobalKTable</strong>
is an abstraction of a <strong>changelog stream</strong>, where each data record
represents an update.
+            </p>
+
+            <p>
+                A GlobalKTable differs from a KTable in the data that they are being populated
with, i.e. which data from the underlying Kafka topic is being read into the respective table.
Slightly simplified, imagine you have an input topic with 5 partitions. In your application,
you want to read this topic into a table. Also, you want to run your application across 5
application instances for <strong>maximum parallelism</strong>.
+            </p>
+
+            <ul>
+                <li>
+                    If you read the input topic into a <strong>KTable</strong>,
then the &quot;local&quot; KTable instance of each application instance will be populated
with data <strong>from only 1 partition</strong> of the topic's 5 partitions.
+                </li>
+
+                <li>
+                    If you read the input topic into a <strong>GlobalKTable</strong>,
then the local GlobalKTable instance of each application instance will be populated with data
<strong>from all partitions of the topic</strong>.
+                </li>
+            </ul>
+
+            <p>
+                GlobalKTable provides the ability to look up <em>current</em>
values of data records by keys. This table-lookup functionality is available through <code
class="interpreted-text">join operations</code>.
+            </p>
+            <p>Benefits of global tables:</p>
+
+            <ul>
+                <li>
+                    More convenient and/or efficient <strong>joins</strong>:
Notably, global tables allow you to perform star joins, they support &quot;foreign-key&quot;
lookups (i.e., you can lookup data in the table not just by record key, but also by data in
the record values), and they are more efficient when chaining multiple joins. Also, when joining
against a global table, the input data does not need to be <strong>co-partitioned</strong>.
+                </li>
+                <li>
+                    Can be used to &quot;broadcast&quot; information to all the running
instances of your application.
+                </li>
+            </ul>
+
+            <p>Downsides of global tables:</p>
+            <ul>
+                <li>Increased local storage consumption compared to the (partitioned)
KTable because the entire topic is tracked.</li>
+                <li>Increased network and Kafka broker load compared to the (partitioned)
KTable because the entire topic is read.</li>
+            </ul>
+
+        </div>
         <div class="section" id="creating-source-streams-from-kafka">
             <span id="streams-developer-guide-dsl-sources"></span><h2><a
class="toc-backref" href="#id8">Creating source streams from Kafka</a><a class="headerlink"
href="#creating-source-streams-from-kafka" title="Permalink to this headline"></a></h2>
             <p>You can easily read data from Kafka topics into your application.  The
following operations are supported.</p>
@@ -123,8 +230,8 @@
                         <li><em>input topics</em> &rarr; KStream</li>
                     </ul>
                 </td>
-                    <td><p class="first">Creates a <a class="reference internal"
href="../concepts.html#streams-concepts-kstream"><span class="std std-ref">KStream</span></a>
from the specified Kafka input topics and interprets the data
-                        as a <a class="reference internal" href="../concepts.html#streams-concepts-kstream"><span
class="std std-ref">record stream</span></a>.
+                    <td><p class="first">Creates a <a class="reference internal"
href="#streams_concepts_kstream"><span class="std std-ref">KStream</span></a>
from the specified Kafka input topics and interprets the data
+                        as a <a class="reference internal" href="#streams_concepts_kstream"><span
class="std std-ref">record stream</span></a>.
                         A <code class="docutils literal"><span class="pre">KStream</span></code>
represents a <em>partitioned</em> record stream.
                         <a class="reference external" href="../../../javadoc/org/apache/kafka/streams/StreamsBuilder.html#stream(java.lang.String)">(details)</a></p>
                         <p>In the case of a KStream, the local KStream instance of
every application instance will
@@ -157,7 +264,7 @@
                         <li><em>input topic</em> &rarr; KTable</li>
                     </ul>
                 </td>
-                    <td><p class="first">Reads the specified Kafka input topic
into a <a class="reference internal" href="../concepts.html#streams-concepts-ktable"><span
class="std std-ref">KTable</span></a>.  The topic is
+                    <td><p class="first">Reads the specified Kafka input topic
into a <a class="reference internal" href="#streams_concepts_ktable"><span class="std
std-ref">KTable</span></a>.  The topic is
                         interpreted as a changelog stream, where records with the same key
are interpreted as UPSERT aka INSERT/UPDATE
                         (when the record value is not <code class="docutils literal"><span
class="pre">null</span></code>) or as DELETE (when the value is <code class="docutils
literal"><span class="pre">null</span></code>) for that key.
                         <a class="reference external" href="../../../javadoc/org/apache/kafka/streams/StreamsBuilder.html#table-java.lang.String(java.lang.String)">(details)</a></p>
@@ -182,7 +289,7 @@
                         <li><em>input topic</em> &rarr; GlobalKTable</li>
                     </ul>
                 </td>
-                    <td><p class="first">Reads the specified Kafka input topic
into a <a class="reference internal" href="../concepts.html#streams-concepts-globalktable"><span
class="std std-ref">GlobalKTable</span></a>.  The topic is
+                    <td><p class="first">Reads the specified Kafka input topic
into a <a class="reference internal" href="#streams_concepts_globalktable"><span
class="std std-ref">GlobalKTable</span></a>.  The topic is
                         interpreted as a changelog stream, where records with the same key
are interpreted as UPSERT aka INSERT/UPDATE
                         (when the record value is not <code class="docutils literal"><span
class="pre">null</span></code>) or as DELETE (when the value is <code class="docutils
literal"><span class="pre">null</span></code>) for that key.
                         <a class="reference external" href="../../../javadoc/org/apache/kafka/streams/StreamsBuilder.html#globalTable-java.lang.String(java.lang.String)">(details)</a></p>
@@ -225,7 +332,7 @@
             <p>Some KStream transformations may generate one or more KStream objects,
for example:
                 - <code class="docutils literal"><span class="pre">filter</span></code>
and <code class="docutils literal"><span class="pre">map</span></code>
on a KStream will generate another KStream
                 - <code class="docutils literal"><span class="pre">branch</span></code>
on KStream can generate multiple KStreams</p>
-            <p>Some others may generate a KTable object, for example an aggregation
of a KStream also yields a KTable. This allows Kafka Streams to continuously update the computed
value upon arrivals of <a class="reference internal" href="../concepts.html#streams-concepts-aggregations"><span
class="std std-ref">late records</span></a> after it
+            <p>Some others may generate a KTable object, for example an aggregation
of a KStream also yields a KTable. This allows Kafka Streams to continuously update the computed
value upon arrivals of <a class="reference internal" href="../core-concepts.html#streams_concepts_aggregations"><span
class="std std-ref">late records</span></a> after it
                 has already been produced to the downstream transformation operators.</p>
             <p>All KTable transformation operations can only generate another KTable.
However, the Kafka Streams DSL does provide a special function
                 that converts a KTable representation into a KStream. All of these transformation
methods can be chained together to compose
@@ -1485,7 +1592,7 @@
                         prices, inventory, customer information) in order to enrich a new
data record (e.g. customer transaction) with context
                         information.  That is, scenarios where you need to perform table
lookups at very large scale and with a low processing
                         latency.  Here, a popular pattern is to make the information in the
databases available in Kafka through so-called
-                        <em>change data capture</em> in combination with <a
class="reference internal" href="../../connect/index.html#kafka-connect"><span class="std
std-ref">Kafka&#8217;s Connect API</span></a>, and then implementing
+                        <em>change data capture</em> in combination with <a
class="reference internal" href="../../#connect"><span class="std std-ref">Kafka&#8217;s
Connect API</span></a>, and then implementing
                         applications that leverage the Streams API to perform very fast and
efficient local joins
                         of such tables and streams, rather than requiring the application
to make a query to a remote database over the network
                         for each record.  In this example, the KTable concept in Kafka Streams
would enable you to track the latest state
@@ -1546,7 +1653,7 @@
                             <strong>It is the responsibility of the user to ensure
data co-partitioning when joining</strong>.</p>
                         <div class="admonition tip">
                             <p><b>Tip</b></p>
-                            <p class="last">If possible, consider using <a class="reference
internal" href="../concepts.html#streams-concepts-globalktable"><span class="std std-ref">global
tables</span></a> (<code class="docutils literal"><span class="pre">GlobalKTable</span></code>)
for joining because they do not require data co-partitioning.</p>
+                            <p class="last">If possible, consider using <a class="reference
internal" href="#streams_concepts_globalktable"><span class="std std-ref">global
tables</span></a> (<code class="docutils literal"><span class="pre">GlobalKTable</span></code>)
for joining because they do not require data co-partitioning.</p>
                         </div>
                         <p>The requirements for data co-partitioning are:</p>
                         <ul class="simple">
@@ -1554,7 +1661,7 @@
                             <li>All applications that <em>write</em> to
the input topics must have the <strong>same partitioning strategy</strong> so
that records with
                                 the same key are delivered to same partition number.  In
other words, the keyspace of the input data must be
                                 distributed across partitions in the same manner.
-                                This means that, for example, applications that use Kafka&#8217;s
<a class="reference internal" href="../../clients/index.html#kafka-clients"><span
class="std std-ref">Java Producer API</span></a> must use the
+                                This means that, for example, applications that use Kafka&#8217;s
<a class="reference internal" href="../../#producerapi"><span class="std std-ref">Java
Producer API</span></a> must use the
                                 same partitioner (cf. the producer setting <code class="docutils
literal"><span class="pre">&quot;partitioner.class&quot;</span></code>
aka <code class="docutils literal"><span class="pre">ProducerConfig.PARTITIONER_CLASS_CONFIG</span></code>),
                                 and applications that use the Kafka&#8217;s Streams API
must use the same <code class="docutils literal"><span class="pre">StreamPartitioner</span></code>
for operations such as
                                 <code class="docutils literal"><span class="pre">KStream#to()</span></code>.
 The good news is that, if you happen to use the default partitioner-related settings across
all
@@ -1958,7 +2065,7 @@
                         <span id="streams-developer-guide-dsl-joins-ktable-ktable"></span><h5><a
class="toc-backref" href="#id16">KTable-KTable Join</a><a class="headerlink" href="#ktable-ktable-join"
title="Permalink to this headline"></a></h5>
                         <p>KTable-KTable joins are always <em>non-windowed</em>
joins.  They are designed to be consistent with their counterparts in
                             relational databases.  The changelog streams of both KTables
are materialized into local state stores to represent the
-                            latest snapshot of their <a class="reference internal" href="../concepts.html#streams-concepts-ktable"><span
class="std std-ref">table duals</span></a>.
+                            latest snapshot of their <a class="reference internal" href="#streams_concepts_ktable"><span
class="std std-ref">table duals</span></a>.
                             The join result is a new KTable that represents the changelog
stream of the join operation.</p>
                         <p>Join output records are effectively created as follows,
leveraging the user-supplied <code class="docutils literal"><span class="pre">ValueJoiner</span></code>:</p>
                         <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="n">KeyValue</span><span class="o">&lt;</span><span class="n">K</span><span
class="o">,</span> <span class="n">LV</span><span class="o">&gt;</span>
<span class="n">leftRecord</span> <span class="o">=</span> <span
class="o">...;</span>
@@ -2523,13 +2630,13 @@
                     <div class="section" id="kstream-globalktable-join">
                         <span id="streams-developer-guide-dsl-joins-kstream-globalktable"></span><h5><a
class="toc-backref" href="#id18">KStream-GlobalKTable Join</a><a class="headerlink"
href="#kstream-globalktable-join" title="Permalink to this headline"></a></h5>
                         <p>KStream-GlobalKTable joins are always <em>non-windowed</em>
joins.  They allow you to perform <em>table lookups</em> against a
-                            <a class="reference internal" href="../concepts.html#streams-concepts-globalktable"><span
class="std std-ref">GlobalKTable</span></a> (entire changelog stream) upon
receiving a new record from the
+                            <a class="reference internal" href="#streams_concepts_globalktable"><span
class="std std-ref">GlobalKTable</span></a> (entire changelog stream) upon
receiving a new record from the
                             KStream (record stream).  An example use case would be &#8220;star
queries&#8221; or &#8220;star joins&#8221;, where you would enrich a stream
                             of user activities (KStream) with the latest user profile information
(GlobalKTable) and further context information
                             (further GlobalKTables).</p>
                         <p>At a high-level, KStream-GlobalKTable joins are very similar
to
                             <a class="reference internal" href="#streams-developer-guide-dsl-joins-kstream-ktable"><span
class="std std-ref">KStream-KTable joins</span></a>.  However, global tables
provide you
-                            with much more flexibility at the <a class="reference internal"
href="../concepts.html#streams-concepts-globalktable"><span class="std std-ref">some
expense</span></a> when compared to partitioned
+                            with much more flexibility at the <a class="reference internal"
href="#streams_concepts_globalktable"><span class="std std-ref">some expense</span></a>
when compared to partitioned
                             tables:</p>
                         <ul class="simple">
                             <li>They do not require <a class="reference internal"
href="#streams-developer-guide-dsl-joins-co-partitioning"><span class="std std-ref">data
co-partitioning</span></a>.</li>
@@ -2695,7 +2802,7 @@
                         defined window boundary.  In aggregating operations, a windowing
state store is used to store the latest aggregation
                         results per window.
                         Old records in the state store are purged after the specified
-                        <a class="reference internal" href="../concepts.html#streams-concepts-windowing"><span
class="std std-ref">window retention period</span></a>.
+                        <a class="reference internal" href="../core-concepts.html#streams_concepts_windowing"><span
class="std std-ref">window retention period</span></a>.
                         Kafka Streams guarantees to keep a window for at least this specified
time; the default value is one day and can be
                         changed via <code class="docutils literal"><span class="pre">Windows#until()</span></code>
and <code class="docutils literal"><span class="pre">SessionWindows#until()</span></code>.</p>
                     <p>The DSL supports the following types of windows:</p>
@@ -2875,7 +2982,7 @@ t=5 (blue), which lead to a merge of sessions and an extension of a
session, res
                     <li><strong>Combining ease-of-use with full flexibility where
it&#8217;s needed:</strong> Even though you generally prefer to use
                         the expressiveness of the DSL, there are certain steps in your processing
that require more flexibility and
                         tinkering than the DSL provides.  For example, only the Processor
API provides access to a
-                        <a class="reference internal" href="../faq.html#streams-faq-processing-record-metadata"><span
class="std std-ref">record&#8217;s metadata</span></a> such as its topic,
partition, and offset information.
+                        record&#8217;s metadata such as its topic, partition, and offset
information.
                         However, you don&#8217;t want to switch completely to the Processor
API just because of that.</li>
                     <li><strong>Migrating from other tools:</strong> You
are migrating from other stream processing technologies that provide an
                         imperative API, and migrating some of your legacy code to the Processor
API was faster and/or easier than to
@@ -2901,7 +3008,7 @@ t=5 (blue), which lead to a merge of sessions and an extension of a
session, res
                             <code class="docutils literal"><span class="pre">process()</span></code>
allows you to leverage the <a class="reference internal" href="processor-api.html#streams-developer-guide-processor-api"><span
class="std std-ref">Processor API</span></a> from the DSL.
                             (<a class="reference external" href="../../../javadoc/org/apache/kafka/streams/kstream/KStream.html#process-org.apache.kafka.streams.processor.ProcessorSupplier-java.lang.String...-">details</a>)</p>
                             <p>This is essentially equivalent to adding the <code
class="docutils literal"><span class="pre">Processor</span></code> via
<code class="docutils literal"><span class="pre">Topology#addProcessor()</span></code>
to your
-                                <a class="reference internal" href="../concepts.html#streams-concepts-processor-topology"><span
class="std std-ref">processor topology</span></a>.</p>
+                                <a class="reference internal" href="../core-concepts.html#streams_topology"><span
class="std std-ref">processor topology</span></a>.</p>
                             <p class="last">An example is available in the
                                 <a class="reference external" href="../../../javadoc/org/apache/kafka/streams/kstream/KStream.html#process-org.apache.kafka.streams.processor.ProcessorSupplier-java.lang.String...-">javadocs</a>.</p>
                         </td>
@@ -2921,7 +3028,7 @@ t=5 (blue), which lead to a merge of sessions and an extension of a
session, res
                                 Applying a grouping or a join after <code class="docutils
literal"><span class="pre">transform</span></code> will result in re-partitioning
of the records.
                                 If possible use <code class="docutils literal"><span
class="pre">transformValues</span></code> instead, which will not cause data
re-partitioning.</p>
                             <p><code class="docutils literal"><span class="pre">transform</span></code>
is essentially equivalent to adding the <code class="docutils literal"><span class="pre">Transformer</span></code>
via <code class="docutils literal"><span class="pre">Topology#addProcessor()</span></code>
to your
-                                <a class="reference internal" href="../concepts.html#streams-concepts-processor-topology"><span
class="std std-ref">processor topology</span></a>.</p>
+                                <a class="reference internal" href="../core-concepts.html#streams_topology"><span
class="std std-ref">processor topology</span></a>.</p>
                             <p class="last">An example is available in the
                                 <a class="reference external" href="../../../javadoc/org/apache/kafka/streams/kstream/KStream.html#transform-org.apache.kafka.streams.kstream.TransformerSupplier-java.lang.String...-">javadocs</a>.
                                </p>
@@ -2940,7 +3047,7 @@ t=5 (blue), which lead to a merge of sessions and an extension of a
session, res
                                 The <code class="docutils literal"><span class="pre">ValueTransformer</span></code>
may return <code class="docutils literal"><span class="pre">null</span></code>
as the new value for a record.</p>
                             <p><code class="docutils literal"><span class="pre">transformValues</span></code>
is preferable to <code class="docutils literal"><span class="pre">transform</span></code>
because it will not cause data re-partitioning.</p>
                             <p><code class="docutils literal"><span class="pre">transformValues</span></code>
is essentially equivalent to adding the <code class="docutils literal"><span class="pre">ValueTransformer</span></code>
via <code class="docutils literal"><span class="pre">Topology#addProcessor()</span></code>
to your
-                                <a class="reference internal" href="../concepts.html#streams-concepts-processor-topology"><span
class="std std-ref">processor topology</span></a>.</p>
+                                <a class="reference internal" href="../core-concepts.html#streams_topology"><span
class="std std-ref">processor topology</span></a>.</p>
                             <p class="last">An example is available in the
                                 <a class="reference external" href="../../../javadoc/org/apache/kafka/streams/kstream/KStream.html#transformValues-org.apache.kafka.streams.kstream.ValueTransformerSupplier-java.lang.String...-">javadocs</a>.</p>
                         </td>
diff --git a/docs/streams/developer-guide/processor-api.html b/docs/streams/developer-guide/processor-api.html
index ae43360..0c83154 100644
--- a/docs/streams/developer-guide/processor-api.html
+++ b/docs/streams/developer-guide/processor-api.html
@@ -70,7 +70,7 @@
         </div>
         <div class="section" id="defining-a-stream-processor">
             <span id="streams-developer-guide-stream-processor"></span><h2><a
class="toc-backref" href="#id2">Defining a Stream Processor</a><a class="headerlink"
href="#defining-a-stream-processor" title="Permalink to this headline"></a></h2>
-            <p>A <a class="reference internal" href="../concepts.html#streams-concepts"><span
class="std std-ref">stream processor</span></a> is a node in the processor
topology that represents a single processing step.
+            <p>A <a class="reference internal" href="../core-concepts.html#streams_processor_node"><span
class="std std-ref">stream processor</span></a> is a node in the processor
topology that represents a single processing step.
                 With the Processor API, you can define arbitrary stream processors that processes
one received record at a time, and connect
                 these processors with their associated state stores to compose the processor
topology.</p>
             <p>You can define a customized stream processor by implementing the <code
class="docutils literal"><span class="pre">Processor</span></code> interface,
which provides the <code class="docutils literal"><span class="pre">process()</span></code>
API method.
@@ -91,7 +91,7 @@
 	       Note, that <code class="docutils literal"><span class="pre">#forward()</span></code>
also allows to change the default behavior by passing a custom timestamp for the output record.</p>
             <p>Specifically, <code class="docutils literal"><span class="pre">ProcessorContext#schedule()</span></code>
accepts a user <code class="docutils literal"><span class="pre">Punctuator</span></code>
callback interface, which triggers its <code class="docutils literal"><span class="pre">punctuate()</span></code>
                 API method periodically based on the <code class="docutils literal"><span
class="pre">PunctuationType</span></code>. The <code class="docutils literal"><span
class="pre">PunctuationType</span></code> determines what notion of time is
used
-                for the punctuation scheduling: either <a class="reference internal" href="../concepts.html#streams-concepts-time"><span
class="std std-ref">stream-time</span></a> or wall-clock-time (by default,
stream-time
+                for the punctuation scheduling: either <a class="reference internal" href="../core-concepts.html#streams_time"><span
class="std std-ref">stream-time</span></a> or wall-clock-time (by default,
stream-time
                 is configured to represent event-time via <code class="docutils literal"><span
class="pre">TimestampExtractor</span></code>). When stream-time is used, <code
class="docutils literal"><span class="pre">punctuate()</span></code>
is triggered purely
                 by data because stream-time is determined (and advanced forward) by the timestamps
derived from the input data. When there
                 is no new input data arriving, stream-time is not advanced and thus <code
class="docutils literal"><span class="pre">punctuate()</span></code>
is not called.</p>
diff --git a/docs/streams/upgrade-guide.html b/docs/streams/upgrade-guide.html
index a4e08ba..0309a53 100644
--- a/docs/streams/upgrade-guide.html
+++ b/docs/streams/upgrade-guide.html
@@ -147,7 +147,7 @@
         put operation metrics would now be
         <code>kafka.streams:type=stream-rocksdb-window-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),rocksdb-window-state-id=([-.\w]+)</code>.
         Users need to update their metrics collecting and reporting systems for their time
and session windowed stores accordingly.
-        For more details, please read the <a href="/{{version}}/documentation/ops.html#kafka_streams_store_monitoring">State
Store Metrics</a> section.
+        For more details, please read the <a href="/{{version}}/documentation/#kafka_streams_store_monitoring">State
Store Metrics</a> section.
     </p>
 
     <p>


Mime
View raw message