kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bbej...@apache.org
Subject [kafka] branch 2.3 updated: [MINOR] Improve docs for Global Store operations (#6803)
Date Thu, 30 May 2019 11:23:32 GMT
This is an automated email from the ASF dual-hosted git repository.

bbejeck pushed a commit to branch 2.3
in repository https://gitbox.apache.org/repos/asf/kafka.git

The following commit(s) were added to refs/heads/2.3 by this push:
     new d5d68f6  [MINOR] Improve docs for Global Store operations (#6803)
d5d68f6 is described below

commit d5d68f6934482597b4eb0ab250eb625efe6b4720
Author: A. Sophie Blee-Goldman <ableegoldman@gmail.com>
AuthorDate: Thu May 30 04:12:27 2019 -0700

    [MINOR] Improve docs for Global Store operations (#6803)
    A lot of confusion seems to have arisen from the StreamBuilder#addGlobalStore(...ProcessorSupplier)
method. Users have assumed they can safely use this to transform records before populating
their global state store; unfortunately this results in corrupted data as on restore the records
are read directly from the source topic changelog, bypassing their custom processor.
    We should probably provide a means to do this at some point but for the time being we
should clarify the proper use of #addGlobalStore as it currently functions
    Reviewers:  Matthias J. Sax <mjsax@apache.org>, Bruno Cadonna <bruno@confluent.io>
 streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java b/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java
index abb5aa1..609bce5 100644
--- a/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java
+++ b/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java
@@ -507,7 +507,9 @@ public class StreamsBuilder {
      * of the input topic.
      * <p>
      * The provided {@link ProcessorSupplier} will be used to create an {@link ProcessorNode}
that will receive all
-     * records forwarded from the {@link SourceNode}.
+     * records forwarded from the {@link SourceNode}. NOTE: you should not use the {@code
Processor} to insert transformed records into
+     * the global state store. This store uses the source topic as changelog and during restore
will insert records directly
+     * from the source.
      * This {@link ProcessorNode} should be used to keep the {@link StateStore} up-to-date.
      * The default {@link TimestampExtractor} as specified in the {@link StreamsConfig config}
is used.
      * <p>

View raw message