2 lmj199582 lmj199582 于 2017.09.14 17:48 提问

SparkSql中读取hive中的表不能存在"."

val hiveDeptDF = sqlContext.read.table("emp_test.emp")
我要读取hive中emp_test中的emp表,报错不能包含“.”

Exception in thread "main" org.apache.spark.sql.AnalysisException: Specifying database name or other qualifiers are not allowed for temporary tables. If the table name has dots (.) in it, please quote the table name with backticks (`).;
at org.apache.spark.sql.catalyst.analysis.Catalog$class.getTableName(Catalog.scala:70)
at org.apache.spark.sql.catalyst.analysis.SimpleCatalog.getTableName(Catalog.scala:82)
at org.apache.spark.sql.catalyst.analysis.SimpleCatalog.lookupRelation(Catalog.scala:104)
at org.apache.spark.sql.DataFrameReader.table(DataFrameReader.scala:338)
at Hive2Rdbms$.main(Hive2Rdbms.scala:16)
at Hive2Rdbms.main(Hive2Rdbms.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

我加上反引号后,又显示找不到该表。

hive库本身没问题

3个回答

morgerton
morgerton   2017.09.14 18:15

http://www.cnblogs.com/zeppelin/p/6061887.html
第三个和你问题一样,你的反引号是不是加的不对?

iamaTraveler
iamaTraveler   2017.09.20 18:13

spark处理hive表用HiveContext

val sc = new SparkContext(conf)
val hiveContext = new HiveContext(sc)
hiveContext.setConf(getHiveConf(props.getProperty(HIVE_METASTORE_IP), props.getProperty(HIVE_METASTORE_PORT)))

wlyang666
wlyang666   2017.11.09 20:40

找不到表是元数据或者数据不在你想指定的路径,既然你说能找到hive表,那么可能是你的代码找不到元数据,看看你的hive-site.xml

Csdn user default icon
上传中...
上传图片
插入图片