spark logback 설정?

spark에서 돌아가는 app에서 logback.xml을 설정하여 사용하고 싶었습니다.

환경은!

spark 1.5.2

scala 2.10.6

그리고 아래와 같이 build.sbt에 설정! 

"ch.qos.logback"  %  "logback-classic"      % "1.1.6" 

그러나 spark conf쪽의 log4j.properties를 조정하면 영향을 받음. 배제 시켜야할 듯 어디에서? 스파크에서!

그러므로 그냥 logback 말고 log4j를 사용하는게 좋을듯!

Keep log4j: Add a bootstrap action to modify /home/hadoop/spark/conf/log4j.properties to add an appender? However, this file already contains a lot of stuff and is a symlink to hadoop conf file. I don't want to fiddle too much with that as it already contains some rootLoggers. Which appender would do best? ryantenney/log4j-redis-appender + logstash/log4j-jsonevent-layout OR pavlobaron/log4j2redis ?

Migrate to slf4j+logback: Exclude slf4j-log4j12 from spark-core, add log4j-over-slf4j ... and use a logback.xml with a com.cwbase.logback.RedisAppender? Looks like this will be problematic with dependencies. Will it hide log4j.rootLoggers already defined in log4j.properties?

위처럼 application쪽에서 아무리 해도 이미 spark쪽에는 소용 없었음.

결론 : spark 할때는 logback 바인딩이 어려우니 그냥 속편하게 log4j 사용하자! 끝~

참고 : http://stackoverflow.com/questions/31790944/best-way-to-send-apache-spark-loggin-to-redis-logstash-on-an-amazon-emr-cluster

spark + log4j 설정 방법 : 

2016/07/04 - [BigDATA/spark] - spark log4j 사용해보기!