使用 akka-http to 提供 http 服务 大致代码如下:
object SparkServer extends App with SparkServices {
implicit val system = ActorSystem("spark-akka-http")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
val bindingFuture = Http().bindAndHandle(route, "192.168.xx.xx", 8001)}
SparkServices
val sparkConf = new SparkConf().setMaster("local[8]")
val sparkSession = SparkSession.builder
.appName("SpanMsgServer")
.config(sparkConf)
.enableHiveSupport()
.getOrCreate()
使用 spark-submit, 提交 出现错误:
com.typesafe.config.ConfigException$Missing: No configuration setting found for key akka.stream
按照网上提示pom 加入
reference.conf
错误没怎么改变,修改 application.conf 添加如下http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/stream-configuration.html配置到 application.conf :
出现了com.typesafe.config.ConfigException$Missing: No configuration setting found for key debug-logging 怎么解决啊