From commits-return-14634-apmail-kafka-commits-archive=kafka.apache.org@kafka.apache.org Mon Jun 1 20:50:37 2020 Return-Path: X-Original-To: apmail-kafka-commits-archive@www.apache.org Delivered-To: apmail-kafka-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by minotaur.apache.org (Postfix) with SMTP id 2442919C50 for ; Mon, 1 Jun 2020 20:50:37 +0000 (UTC) Received: (qmail 17143 invoked by uid 500); 1 Jun 2020 20:50:36 -0000 Delivered-To: apmail-kafka-commits-archive@kafka.apache.org Received: (qmail 17115 invoked by uid 500); 1 Jun 2020 20:50:36 -0000 Mailing-List: contact commits-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list commits@kafka.apache.org Received: (qmail 17106 invoked by uid 99); 1 Jun 2020 20:50:36 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2020 20:50:36 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id 1BEAA81F29; Mon, 1 Jun 2020 20:50:36 +0000 (UTC) Date: Mon, 01 Jun 2020 20:50:27 +0000 To: "commits@kafka.apache.org" Subject: [kafka] 01/03: MINOR: Fix Streams EOS system tests by adding clean-up of state dir (#7693) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit From: vvcephei@apache.org In-Reply-To: <159104462064.3265.569239084949955451@gitbox.apache.org> References: <159104462064.3265.569239084949955451@gitbox.apache.org> X-Git-Host: gitbox.apache.org X-Git-Repo: kafka X-Git-Refname: refs/heads/1.0 X-Git-Reftype: branch X-Git-Rev: d9f9011063aeefd6706c4c851dbb54216d39c8a5 X-Git-NotificationType: diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated Message-Id: <20200601205036.1BEAA81F29@gitbox.apache.org> This is an automated email from the ASF dual-hosted git repository. vvcephei pushed a commit to branch 1.0 in repository https://gitbox.apache.org/repos/asf/kafka.git commit d9f9011063aeefd6706c4c851dbb54216d39c8a5 Author: Bruno Cadonna AuthorDate: Thu Nov 21 16:32:31 2019 +0100 MINOR: Fix Streams EOS system tests by adding clean-up of state dir (#7693) Recently, system tests test_rebalance_[simple|complex] failed repeatedly with a verfication error. The cause was most probably the missing clean-up of a state directory of one of the processors. A node is cleaned up when a service on that node is started and when a test is torn down. If the clean-up flag clean_node_enabled of a EOS Streams service is unset, the clean-up of the node is skipped. The clean-up flag of processor1 in the EOS tests should stay set before its first start, so that the node is cleaned before the service is started. Afterwards for the multiple restarts of processor1 the cleans-up flag should be unset to re-use the local state. After the multiple restarts are done, the clean-up flag of processor1 should again be set to trigger node clean-up during the test teardown. A dirty node can lead to test failures when tests from Streams EOS tests are scheduled on the same node, because the state store would not start empty since it reads the local state that was not cleaned up. Reviewers: Matthias J. Sax , Andrew Choi , Bill Bejeck --- tests/kafkatest/tests/streams/streams_eos_test.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/kafkatest/tests/streams/streams_eos_test.py b/tests/kafkatest/tests/streams/streams_eos_test.py index d6ac600..0bb2eb7 100644 --- a/tests/kafkatest/tests/streams/streams_eos_test.py +++ b/tests/kafkatest/tests/streams/streams_eos_test.py @@ -60,9 +60,8 @@ class StreamsEosTest(KafkaTest): self.driver.start() - processor1.clean_node_enabled = False - self.add_streams(processor1) + processor1.clean_node_enabled = False self.add_streams2(processor1, processor2) self.add_streams3(processor1, processor2, processor3) self.stop_streams3(processor2, processor3, processor1) @@ -70,6 +69,7 @@ class StreamsEosTest(KafkaTest): self.stop_streams3(processor1, processor3, processor2) self.stop_streams2(processor1, processor3) self.stop_streams(processor1) + processor1.clean_node_enabled = True self.driver.stop() @@ -100,9 +100,8 @@ class StreamsEosTest(KafkaTest): self.driver.start() - processor1.clean_node_enabled = False - self.add_streams(processor1) + processor1.clean_node_enabled = False self.add_streams2(processor1, processor2) self.add_streams3(processor1, processor2, processor3) self.abort_streams(processor2, processor3, processor1) @@ -112,6 +111,7 @@ class StreamsEosTest(KafkaTest): self.abort_streams(processor1, processor3, processor2) self.stop_streams2(processor1, processor3) self.stop_streams(processor1) + processor1.clean_node_enabled = True self.driver.stop()