kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vvcep...@apache.org
Subject [kafka] 01/02: MINOR: Fix Streams EOS system tests by adding clean-up of state dir (#7693)
Date Mon, 01 Jun 2020 19:29:08 GMT
This is an automated email from the ASF dual-hosted git repository.

vvcephei pushed a commit to branch 2.0
in repository https://gitbox.apache.org/repos/asf/kafka.git

commit 8bd7715b858d01cf0fbeba55a28102e5f3b9ed84
Author: Bruno Cadonna <bruno@confluent.io>
AuthorDate: Thu Nov 21 16:32:31 2019 +0100

    MINOR: Fix Streams EOS system tests by adding clean-up of state dir (#7693)
    
    Recently, system tests test_rebalance_[simple|complex] failed
    repeatedly with a verfication error. The cause was most probably
    the missing clean-up of a state directory of one of the processors.
    
    A node is cleaned up when a service on that node is started and when
    a test is torn down.
    
    If the clean-up flag clean_node_enabled of a EOS Streams service is
    unset, the clean-up of the node is skipped.
    
    The clean-up flag of processor1 in the EOS tests should stay set before
    its first start, so that the node is cleaned before the service is started.
    Afterwards for the multiple restarts of processor1 the cleans-up flag should
    be unset to re-use the local state.
    
    After the multiple restarts are done, the clean-up flag of processor1 should
    again be set to trigger node clean-up during the test teardown.
    
    A dirty node can lead to test failures when tests from Streams EOS tests are
    scheduled on the same node, because the state store would not start empty
    since it reads the local state that was not cleaned up.
    
    Reviewers: Matthias J. Sax <mjsax@apache.org>, Andrew Choi <andchoi@linkedin.com>,
Bill Bejeck <bbejeck@gmail.com>
---
 tests/kafkatest/tests/streams/streams_eos_test.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/kafkatest/tests/streams/streams_eos_test.py b/tests/kafkatest/tests/streams/streams_eos_test.py
index 7e5cc26..c56489d 100644
--- a/tests/kafkatest/tests/streams/streams_eos_test.py
+++ b/tests/kafkatest/tests/streams/streams_eos_test.py
@@ -60,9 +60,8 @@ class StreamsEosTest(KafkaTest):
 
         self.driver.start()
 
-        processor1.clean_node_enabled = False
-
         self.add_streams(processor1)
+        processor1.clean_node_enabled = False
         self.add_streams2(processor1, processor2)
         self.add_streams3(processor1, processor2, processor3)
         self.stop_streams3(processor2, processor3, processor1)
@@ -70,6 +69,7 @@ class StreamsEosTest(KafkaTest):
         self.stop_streams3(processor1, processor3, processor2)
         self.stop_streams2(processor1, processor3)
         self.stop_streams(processor1)
+        processor1.clean_node_enabled = True
 
         self.driver.stop()
 
@@ -100,9 +100,8 @@ class StreamsEosTest(KafkaTest):
 
         self.driver.start()
 
-        processor1.clean_node_enabled = False
-
         self.add_streams(processor1)
+        processor1.clean_node_enabled = False
         self.add_streams2(processor1, processor2)
         self.add_streams3(processor1, processor2, processor3)
         self.abort_streams(processor2, processor3, processor1)
@@ -112,6 +111,7 @@ class StreamsEosTest(KafkaTest):
         self.abort_streams(processor1, processor3, processor2)
         self.stop_streams2(processor1, processor3)
         self.stop_streams(processor1)
+        processor1.clean_node_enabled = True
 
         self.driver.stop()
 


Mime
View raw message