CDH 关闭 KMS 服务后重启无法监控到 host,应用也无法监控和管理,如何恢复监控?

问答 歌唱祖国 ⋅ 于 2018-06-20 11:03:22 ⋅ 最后回复由 青牛 2018-06-21 16:53:36 ⋅ 5156 阅读

file
查看event server的日志

2018-06-19 22:01:33,380 INFO com.cloudera.enterprise.ssl.SSLFactory: Using configured truststore for verification of server certificates in HTTPS communication.
2018-06-19 22:01:33,613 WARN com.cloudera.cmf.BasicScmProxy: Exception while getting fetch configDefaults hash: none
java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:275)
        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:371)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1091)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
        at com.cloudera.cmf.BasicScmProxy.authenticate(BasicScmProxy.java:265)
        at com.cloudera.cmf.BasicScmProxy.fetch(BasicScmProxy.java:561)
        at com.cloudera.cmf.BasicScmProxy.getFragmentAndHash(BasicScmProxy.java:651)
        at com.cloudera.cmf.DescriptorAndFragments.newDescriptorAndFragments(DescriptorAndFragments.java:64)
        at com.cloudera.cmf.eventcatcher.server.EventCatcherService.main(EventCatcherService.java:100)
2018-06-19 22:01:33,620 WARN com.cloudera.cmf.eventcatcher.server.EventCatcherService: No descriptor fetched from https://ip-172-31-2-173.cn-north-1.compute.internal:7183 on after 1 tries, sleeping...
2018-06-19 22:01:33,700 WARN com.cloudera.cmf.event.publish.EventStorePublisherWithRetry: Failed to publish event: SimpleEvent{attributes={STACKTRACE=[java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:275)
        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:371)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1091)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
        at com.cloudera.cmf.BasicScmProxy.authenticate(BasicScmProxy.java:265)
        at com.cloudera.cmf.BasicScmProxy.fetch(BasicScmProxy.java:561)
        at com.cloudera.cmf.BasicScmProxy.getFragmentAndHash(BasicScmProxy.java:651)
        at com.cloudera.cmf.DescriptorAndFragments.newDescriptorAndFragments(DescriptorAndFragments.java:64)
        at com.cloudera.cmf.eventcatcher.server.EventCatcherService.main(EventCatcherService.java:100)
], EXCEPTION_TYPES=[java.net.ConnectException], ROLE=[mgmt-EVENTSERVER-e9e048cd522336b5e4f2193522ec91e5], SEVERITY=[IMPORTANT], SERVICE=[mgmt], HOST_IDS=[05c4cccc-5089-460f-9765-c2f3c0c264cd], LOG_LEVEL=[WARN], ROLE_TYPE=[EVENTSERVER], CATEGORY=[LOG_MESSAGE], SERVICE_TYPE=[MGMT], HOSTS=[ip-compute.internal], EVENTCODE=[EV_LOG_EVENT]}, content=Exception while getting fetch configDefaults hash: none, timestamp=1529460093613}
2018-06-19 22:01:35,857 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2018-06-19 22:01:35,975 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM.
2018-06-19 22:01:36,034 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2018-06-19 22:01:36,035 WARN com.cloudera.cmf.eventcatcher.server.EventCatcherService: No descriptor fetched from https://ip-compute.internal:7183 on after 2 tries, sleeping...
2018-06-19 22:01:38,103 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2018-06-19 22:01:38,194 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM.
2018-06-19 22:01:38,206 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2018-06-19 22:01:38,206 WARN com.cloudera.cmf.eventcatcher.server.EventCatcherService: No descriptor fetched from https://ip-compute.internal:7183 on after 3 tries, sleeping...
2018-06-19 22:01:40,271 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2018-06-19 22:01:40,358 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM.
2018-06-19 22:01:40,370 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2018-06-19 22:01:40,370 WARN com.cloudera.cmf.eventcatcher.server.EventCatcherService: No descriptor fetched from https://ip-.compute.internal:7183 on after 4 tries, sleeping...
2018-06-19 22:01:42,434 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2018-06-19 22:01:42,517 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM.
2018-06-19 22:01:42,526 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2018-06-19 22:01:42,526 WARN com.cloudera.cmf.eventcatcher.server.EventCatcherService: No descriptor fetched from https://ip-north-1.compute.internal:7183 on after 5 tries, sleeping...
2018-06-19 22:01:44,527 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.

请问是不是ssl证书出现问题了, 已经重启几遍集群仍无法解决
大神帮忙指点下

成为第一个点赞的人吧 :bowtie:
回复数量: 1
  • 青牛 国内首批大数据从业者,就职于金山,担任大数据团队核心研发工程师
    2018-06-21 16:53:36

    看一下你关闭的那个KMS 服务启动成功了吧?

暂无评论~~
  • 请注意单词拼写,以及中英文排版,参考此页
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`, 更多语法请见这里 Markdown 语法
  • 支持表情,可用Emoji的自动补全, 在输入的时候只需要 ":" 就可以自动提示了 :metal: :point_right: 表情列表 :star: :sparkles:
  • 上传图片, 支持拖拽和剪切板黏贴上传, 格式限制 - jpg, png, gif,教程
  • 发布框支持本地存储功能,会在内容变更时保存,「提交」按钮点击时清空
Ctrl+Enter