kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mj...@apache.org
Subject [kafka] branch trunk updated: MINOR: document timestamped state stores (#8920)
Date Tue, 07 Jul 2020 01:10:00 GMT
This is an automated email from the ASF dual-hosted git repository.

mjsax pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/trunk by this push:
     new 11a8549  MINOR: document timestamped state stores (#8920)
11a8549 is described below

commit 11a8549479431826e64e80e49f1c5e41be8a7fd5
Author: Jim Galasyn <jim.galasyn@confluent.io>
AuthorDate: Mon Jul 6 18:09:24 2020 -0700

    MINOR: document timestamped state stores (#8920)
    
    Reviewers: Matthias J. Sax <matthias@confluent.io>
---
 docs/streams/developer-guide/processor-api.html | 28 +++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/docs/streams/developer-guide/processor-api.html b/docs/streams/developer-guide/processor-api.html
index e7c6eac..d314c7d 100644
--- a/docs/streams/developer-guide/processor-api.html
+++ b/docs/streams/developer-guide/processor-api.html
@@ -49,6 +49,7 @@
                         <li><a class="reference internal" href="#defining-and-creating-a-state-store"
id="id4">Defining and creating a State Store</a></li>
                         <li><a class="reference internal" href="#fault-tolerant-state-stores"
id="id5">Fault-tolerant State Stores</a></li>
                         <li><a class="reference internal" href="#enable-or-disable-fault-tolerance-of-state-stores-store-changelogs"
id="id6">Enable or Disable Fault Tolerance of State Stores (Store Changelogs)</a></li>
+                        <li><a class="reference internal" href="#timestamped-state-stores"
id="id11">Timestamped State Stores</a></li>
                         <li><a class="reference internal" href="#implementing-custom-state-stores"
id="id7">Implementing Custom State Stores</a></li>
                     </ul>
                 </li>
@@ -229,6 +230,10 @@
                                 <a class="reference internal" href="config-streams.html#streams-developer-guide-rocksdb-config"><span
class="std std-ref">RocksDB configuration</span></a>.</li>
                             <li>Available <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentKeyValueStore-java.lang.String-">store
variants</a>:
                                 time window key-value store, session window key-value store.</li>
+                            <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentTimestampedKeyValueStore-java.lang.String-">persistentTimestampedKeyValueStore</a>
+                                when you need a  persistent key-(value/timestamp) store that
supports put/get/delete and range queries.</li>
+                            <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentTimestampedWindowStore-java.lang.String-java.time.Duration-java.time.Duration-boolean-">persistentTimestampedWindowStore</a>
+                                when you need a persistent windowedKey-(value/timestamp)
store.</li>
                         </ul>
                             <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="c1">// Creating a persistent key-value store:</span>
 <span class="c1">// here, we create a `KeyValueStore&lt;String, Long&gt;` named
&quot;persistent-counts&quot;.</span>
@@ -260,6 +265,10 @@
                                 in-between app instance restarts.</li>
                             <li>Available <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#inMemoryKeyValueStore-java.lang.String-">store
variants</a>:
                                 time window key-value store, session window key-value store.</li>
+                            <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedKeyValueStore.html">TimestampedKeyValueStore</a>
+                                when you need a key-(value/timestamp) store that supports
put/get/delete and range queries.</li>
+                            <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedWindowStore.html">TimestampedWindowStore</a>
+                                when you need to store windowedKey-(value/timestamp) pairs.</li>
                         </ul>
                             <div class="highlight-java"><div class="highlight"><pre><span></span><span
class="c1">// Creating an in-memory key-value store:</span>
 <span class="c1">// here, we create a `KeyValueStore&lt;String, Long&gt;` named
&quot;inmemory-counts&quot;.</span>
@@ -343,6 +352,25 @@
 </pre></div>
                 </div>
             </div>
+            <div class="section" id="timestamped-state-stores">
+                <span id="streams-developer-guide-state-store-timestamps"></span><h3><a
class="toc-backref" href="#id11">Timestamped State Stores</a><a class="headerlink"
href="#timestamped-state-stores" title="Permalink to this headline"></a></h3>
+                <p>KTables always store timestamps by default.
+                    A timestamped state store improves stream processing semantics and enables
+                    handling out-of-order data in source KTables, detecting out-of-order
joins and aggregations,
+                    and getting the timestamp of the latest update in an Interactive Query.</p>
+                <p>You can query timestamped state stores both with and without a timestamp.</p>
+                <b>Upgrade note:</b> All users upgrade with a single rolling
bounce per instance.
+                <ul class="first simple">
+                    <li>For Processor API users, nothing changes in existing applications,
and you
+                    have the option of using the timestamped stores.</li>        
+                    <li>For DSL operators, store data is upgraded lazily in the background.</li>
+                    <li>No upgrade happens if you provide a custom XxxBytesStoreSupplier,
but you can opt-in
+                    by implementing the <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedBytesStore.html">TimestampedBytesStore</a>
+                    interface. In this case, the old format is retained, and Streams uses
a proxy store
+                    that removes/adds timestamps on read/write.</li>
+                </ul>
+                </p>
+            </div>
             <div class="section" id="implementing-custom-state-stores">
                 <span id="streams-developer-guide-state-store-custom"></span><h3><a
class="toc-backref" href="#id7">Implementing Custom State Stores</a><a class="headerlink"
href="#implementing-custom-state-stores" title="Permalink to this headline"></a></h3>
                 <p>You can use the <a class="reference internal" href="#streams-developer-guide-state-store-defining"><span
class="std std-ref">built-in state store types</span></a> or  implement your
own.


Mime
View raw message