Sqoop 导出 hive 数据到 MySQL 报错?

问答 忘尘 ⋅ 于 2020-06-28 21:21:38 ⋅ 最后回复由 忘尘 2020-07-11 15:57:51 ⋅ 2341 阅读

可以连接到mysql查到里面有哪些表

[hadoop@nn1 ~]$ sqoop list-tables --connect jdbc:mysql://127.0.0.1:3306/etl_db --username etl --password 000000
……
20/06/28 21:14:53 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
counter_info
country
tongji1
tongji2

往mysql里导数据就会出这些错误

sqoop export --connect jdbc:mysql://127.0.0.1:3306/etl_db --username etl --password 000000 --table country --export-dir /hive/warehouse/db1.db/coun

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:389)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1038)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:338)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2237)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2270)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2069)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:794)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:389)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:399)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:325)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
        at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.<init>(AsyncSqlRecordWriter.java:76)
        at org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.<init>(ExportOutputFormat.java:95)
        at org.apache.sqoop.mapreduce.ExportOutputFormat.getRecordWriter(ExportOutputFormat.java:77)
        ... 8 more
Caused by: java.net.ConnectException: 拒绝连接 (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:213)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:297)
        ... 26 more

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

20/06/28 21:19:12 INFO mapreduce.Job: Task Id : attempt_1593343815515_0009_m_000000_1, Status : FAILED
Error: java.io.IOException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
点赞
成为第一个点赞的人吧 :bowtie:
回复数量: 3
  • 苏牛
    2020-06-28 21:29:19

    这个错误可以参照讲义上对mysql参数进行配置 增加等待时间

  • 忘尘
    2020-06-28 21:38:29

    @苏牛 等待时间我改成了wait_timeout=388000 ,重启MySQL之后还是不行~

  • 忘尘
    2020-07-11 15:57:51

    localhost换成主机名或者地址就好了。

暂无评论~~
  • 请注意单词拼写,以及中英文排版,参考此页
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`, 更多语法请见这里 Markdown 语法
  • 支持表情,可用Emoji的自动补全, 在输入的时候只需要 ":" 就可以自动提示了 :metal: :point_right: 表情列表 :star: :sparkles:
  • 上传图片, 支持拖拽和剪切板黏贴上传, 格式限制 - jpg, png, gif,教程
  • 发布框支持本地存储功能,会在内容变更时保存,「提交」按钮点击时清空
Ctrl+Enter