kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rsiva...@apache.org
Subject [kafka] branch trunk updated: KAFKA-6415; Use WARN log level for Metadata in system test
Date Wed, 03 Jan 2018 19:18:23 GMT
This is an automated email from the ASF dual-hosted git repository.

rsivaram pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/trunk by this push:
     new 3de9103  KAFKA-6415; Use WARN log level for Metadata in system test
3de9103 is described below

commit 3de910319e56d81c50597f05459d35752345907c
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
AuthorDate: Wed Jan 3 19:17:53 2018 +0000

    KAFKA-6415; Use WARN log level for Metadata in system test
    
    When a log entry is appended to a Kafka topic using `KafkaLog4jAppender`, the producer.send
operation may block waiting for metadata. This can result in deadlocks in a couple of scenarios
if a log entry from the producer network thread is also at a log level that results in the
entry being appended to a Kafka topic.
    1. Producer's network thread will attempt to send data to a Kafka topic and this is unsafe
since producer.send may block waiting for metadata, causing a deadlock since the thread will
not process the metadata request/response.
    2. `KafkaLog4jAppender#append` is invoked while holding the lock of the logger. So the
thread waiting for metadata in the initial send will be holding the logger lock. If the producer
network thread has.a log entry that needs to be appended, it will attempt to acquire the logger
lock and deadlock.
    
    This is a temporary workaround to avoid deadlocks in system tests by setting log level
to WARN for `Metadata` in `VerifiableLog4jAppender`. The fix has been verified using the system
tests log4j_appender_test.py which started failing when the info-level log entry was introduced.
    
    Author: Rajini Sivaram <rajinisivaram@googlemail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>, Satish Duggana <satish.duggana@gmail.com>,
tedyu <yuzhihong@gmail.com>
    
    Closes #4375 from rajinisivaram/KAFKA-6415-log4jappender
---
 .../src/main/java/org/apache/kafka/tools/VerifiableLog4jAppender.java  | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tools/src/main/java/org/apache/kafka/tools/VerifiableLog4jAppender.java b/tools/src/main/java/org/apache/kafka/tools/VerifiableLog4jAppender.java
index 4dd7bee..9533f6e 100644
--- a/tools/src/main/java/org/apache/kafka/tools/VerifiableLog4jAppender.java
+++ b/tools/src/main/java/org/apache/kafka/tools/VerifiableLog4jAppender.java
@@ -204,6 +204,9 @@ public class VerifiableLog4jAppender {
                 props.setProperty("log4j.appender.KAFKA.kerb5ConfPath", res.getString("kerb5ConfPath"));
             }
             props.setProperty("log4j.logger.kafka.log4j", "INFO, KAFKA");
+            // Changing log level from INFO to WARN as a temporary workaround for KAFKA-6415.
This is to
+            // avoid deadlock in system tests when producer network thread appends to log
while updating metadata.
+            props.setProperty("log4j.logger.org.apache.kafka.clients.Metadata", "WARN, KAFKA");
 
             if (configFile != null) {
                 try {

-- 
To stop receiving notification emails like this one, please contact
['"commits@kafka.apache.org" <commits@kafka.apache.org>'].

Mime
View raw message