sqoop中query \$CONDITIONS HUE执行失败怎么办

159次阅读

共计 2728 个字符，预计需要花费 7 分钟才能阅读完成。

这篇文章给大家介绍 sqoop 中 query \$CONDITIONS HUE 执行失败怎么办，内容非常详细，感兴趣的小伙伴们可以参考借鉴，希望对大家能有所帮助。

1. HUE sqoop 命令框中直接输入：

sqoop-import –connect jdbc:mysql://ip 地址: 端口 / 数据库 –username 用户名 –password 密码 –query select * from 表名 where 1=1 and \$CONDITIONS –split-by id –hive-table hive 表 –target-dir /apps/hive/warehouse/mydb.db/product/dt=${date} –hive-partition-key dt –hive-partition-value ${date} –hive-import –driver com.mysql.jdbc.Driver

运行会提示报错，是因为 SQL 语句被解析成 sqoop 参数

2784 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Error parsing arguments for import:
2016-09-11 15:21:46,485 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(304)) – Error parsing arguments for import:
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: *
2016-09-11 15:21:46,486 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(307)) – Unrecognized argument: *
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: from
2016-09-11 15:21:46,486 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(307)) – Unrecognized argument: from
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: product
2016-09-11 15:21:46,486 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(307)) – Unrecognized argument: p_product
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: where
2016-09-11 15:21:46,486 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(307)) – Unrecognized argument: where
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: $CONDITIONS

2. 解决方案：最好采用参数化的方式

workflow-app name= test xmlns= uri:oozie:workflow:0.5
start to= sqoop-fb61 /
kill name= Kill
message Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}] /message
/kill
action name= sqoop-fb61
sqoop xmlns= uri:oozie:sqoop-action:0.2
job-tracker ${jobTracker} /job-tracker
name-node ${nameNode} /name-node
arg import /arg
arg –connect /arg
arg quot;jdbc:mysql://IP 地址: 端口 / 数据库 quot; /arg
arg –username /arg
arg quot; 用户名 quot; /arg
arg –password /arg
arg quot; 密码 quot; /arg
arg –query /arg
arg select * from product where $CONDITIONS /arg ## 注意：这个地方没有反斜杠（\）
arg –split-by /arg
arg quot;id quot; /arg
arg –hive-table /arg
arg quot;HIVE 表名 quot; /arg
arg –target-dir /arg
arg quot;/apps/hive/warehouse/mydb.db/product/dt=${date} quot; /arg
arg –hive-partition-key /arg
arg quot;dt quot; /arg
arg –hive-partition-value /arg
arg quot;${date} quot; /arg
arg –hive-import /arg
arg –driver /arg
arg quot;com.mysql.jdbc.Driver quot; /arg
/sqoop
ok to= End /
error to= Kill /
/action
end name= End /
/workflow-app