kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guozh...@apache.org
Subject [kafka] branch 3.0 updated: KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
Date Wed, 18 Aug 2021 00:06:48 GMT
This is an automated email from the ASF dual-hosted git repository.

guozhang pushed a commit to branch 3.0
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/3.0 by this push:
     new 3f2c833  KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
3f2c833 is described below

commit 3f2c833cda512d5e7332f41e4c66e544114ae4d5
Author: Guozhang Wang <wangguoz@gmail.com>
AuthorDate: Tue Aug 17 17:03:11 2021 -0700

    KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
    
    Reviewers: Luke Chen <showuon@gmail.com>, Anna Sophie Blee-Goldman <ableegoldman@apache.org>,
Bruno Cadonna <cadonna@confluent.io>
---
 docs/streams/upgrade-guide.html | 33 +++++++++++++++++++--------------
 1 file changed, 19 insertions(+), 14 deletions(-)

diff --git a/docs/streams/upgrade-guide.html b/docs/streams/upgrade-guide.html
index eb3b125..ecc8cb8 100644
--- a/docs/streams/upgrade-guide.html
+++ b/docs/streams/upgrade-guide.html
@@ -52,6 +52,22 @@
         <li> restart all new ({{fullDotVersion}}) application instances </li>
     </ul>
 
+    <h3 class="anchor-heading"><a id="streams_notable_changes" class="anchor-link"></a><a
href="#streams_notable_changes">Notable compatibility changes in past releases</a></h3>
+    <p>
+        Downgrading from 3.0.x or newer version to 2.8.x or older version needs special attention:
+        Since 3.0.0 release, Kafka Streams uses a newer RocksDB version whose on-disk format
changed.
+        This means that old versioned RocksDB would not be able to recognize the bytes written
by that newer versioned RocksDB,
+        and hence it is harder to downgrade Kafka Streams with version 3.0.0 or newer to
older versions in-flight.
+        Users need to wipe out the local RocksDB state stores written by the new versioned
Kafka Streams before swapping in the
+        older versioned Kafka Streams bytecode, which would then restore the state stores
with the old on-disk format from the
+        changelogs.
+    </p>
+
+    <p>
+        Kafka Streams does not support running multiple instances of the same application
as different processes on the same physical state directory. Starting in 2.8.0 (as well as
2.7.1 and 2.6.2),
+        this restriction will be enforced. If you wish to run more than one instance of Kafka
Streams, you must configure them with different values for <code>state.dir</code>.
+    </p>
+
     <p>
         Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version
2. This can be configured
         by setting <code>"processing.guarantee"</code> to <code>"exactly_once_v2"</code>
for
@@ -69,6 +85,8 @@
         Afterward, you can downgrade your application to a pre-2.6.x version.
     </p>
 
+    <p>Since 2.6.0 release, Kafka Streams depends on a RocksDB version that requires
MacOS 10.14 or higher.</p>
+
     <p>
         To run a Kafka Streams application version 2.2.1, 2.3.0, or higher a broker version
0.11.0 or higher is required
         and the on-disk message format must be 0.11 or higher.
@@ -77,10 +95,8 @@
         For Kafka Streams 0.10.0, broker version 0.10.0 or higher is required.
     </p>
 
-    <p>Since 2.6.0 release, Kafka Streams depends on a RocksDB version that requires
MacOS 10.14 or higher.</p>
-
     <p>
-        Another important thing to keep in mind: in deprecated <code>KStreamBuilder</code>
class, when a <code>KTable</code> is created from a source topic via <code>KStreamBuilder.table()</code>,
its materialized state store
+        In deprecated <code>KStreamBuilder</code> class, when a <code>KTable</code>
is created from a source topic via <code>KStreamBuilder.table()</code>, its materialized
state store
         will reuse the source topic as its changelog topic for restoring, and will disable
logging to avoid appending new updates to the source topic; in the <code>StreamsBuilder</code>
class introduced in 1.0, this behavior was changed
         accidentally: we still reuse the source topic as the changelog topic for restoring,
but will also create a separate changelog topic to append the update records from source topic
to. In the 2.0 release, we have fixed this issue and now users
         can choose whether or not to reuse the source topic based on the <code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code>:
if you are upgrading from the old <code>KStreamBuilder</code> class and hence
you need to change your code to use
@@ -89,11 +105,6 @@
         More details about the new config <code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code>
can be found in <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-295%3A+Add+Streams+Configuration+Allowing+for+Optional+Topology+Optimization">KIP-295</a>.
     </p>
 
-    <p>
-      Note: Kafka Streams does not support running multiple instances of the same application
as different processes on the same physical state directory. Starting in 2.8.0 (as well as
2.7.1 and 2.6.2),
-      this restriction will be enforced. If you wish to run more than one instance of Kafka
Streams, you must configure them with different values for <code>state.dir</code>.
-    </p>
-
     <h3><a id="streams_api_changes_300" href="#streams_api_changes_300">Streams
API changes in 3.0.0</a></h3>
     <p>
       We improved the semantics of
@@ -543,12 +554,6 @@
         For more details please read <a href="https://issues.apache.org/jira/browse/KAFKA-8215">KAFKA-8215</a>.
     </p>
 
-    <h3 class="anchor-heading"><a id="streams_notable_changes_221" class="anchor-link"></a><a
href="#streams_notable_changes_221">Notable changes in Kafka Streams 2.2.1</a></h3>
-    <p>
-        As of Kafka Streams 2.2.1 a message format 0.11 or higher is required;
-        this implies that brokers must be on version 0.11.0 or higher.
-    </p>
-
     <h3 class="anchor-heading"><a id="streams_api_changes_220" class="anchor-link"></a><a
href="#streams_api_changes_220">Streams API changes in 2.2.0</a></h3>
     <p>
         We've simplified the <code>KafkaStreams#state</code> transition diagram
during the starting up phase a bit in 2.2.0: in older versions the state will transit from
<code>CREATED</code> to <code>RUNNING</code>, and then to <code>REBALANCING</code>
to get the first

Mime
View raw message