Community,

 

How can I suppress INFO messages  from Spark Streaming job for per job ? …….. meaning, I don’t want to change the log4j properties for the entire Spark cluster but want to suppress just the INFO messages for a specific Streaming job perhaps in the job properties file, Is that possible?

 

Or, Do I need to write the sc._jvm.Logging function inside our scala code to suppress INFO messages of RDDs?

 

Please help us, else, the Streaming job output re-direct log is so BIG with those INFO messages, our  file system is getting full. Thanks again.