在远程节点上部署一些Apache Kafka实例后,我观察到了kafka-server-stop.sh
作为Kafka存档一部分的脚本问题.
默认情况下,它包含:
#!/bin/sh # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. ps ax | grep -i 'kafka\.Kafka' | grep java | grep -v grep | awk '{print $1}' | xargs kill -SIGTERM
如果我执行apache kafka而不是后台进程,这个脚本效果很好,例如:
/var/lib/kafka/bin/kafka-server-start.sh /var/lib/kafka/config/server.properties
当我将它作为后台进程执行时它也可以工作:
/var/lib/kafka/bin/kafka-server-start.sh /var/lib/kafka/config/server.properties &
但是在我的远程节点上,我使用这个python脚本执行它(使用Ansible):
#!/usr/bin/env python import argparse import os import subprocess KAFKA_PATH = "/var/lib/kafka/" def execute_command_pipe_output(command_to_call): return subprocess.Popen(command_to_call, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) def execute_command_no_output(command_to_call): with open(os.devnull, "w") as null_file: return subprocess.Popen(command_to_call, stdout=null_file, stderr=subprocess.STDOUT) def start_kafka(args): command_to_call = ["nohup"] command_to_call += [KAFKA_PATH + "bin/zookeeper-server-start.sh"] command_to_call += [KAFKA_PATH + "config/zookeeper.properties"] proc = execute_command_no_output(command_to_call) command_to_call = ["nohup"] command_to_call += [KAFKA_PATH + "bin/kafka-server-start.sh"] command_to_call += [KAFKA_PATH + "config/server.properties"] proc = execute_command_no_output(command_to_call) def stop_kafka(args): command_to_call = [KAFKA_PATH + "bin/kafka-server-stop.sh"] proc = execute_command_pipe_output(command_to_call) for line in iter(proc.stdout.readline, b''): print line, command_to_call = [KAFKA_PATH + "bin/zookeeper-server-stop.sh"] proc = execute_command_pipe_output(command_to_call) for line in iter(proc.stdout.readline, b''): print line, if __name__ == "__main__": parser = argparse.ArgumentParser(description="Starting Zookeeper and Kafka instances") parser.add_argument('action', choices=['start', 'stop'], help="action to take") args = parser.parse_args() if args.action == 'start': start_kafka(args) elif args.action == 'stop': stop_kafka(args) else: parser.print_help()
执行后
manage-kafka.py start manage-kafka.py stop
Zookeeper关闭(应该是),但Kafka仍在运行.
什么更有趣,当我调用(手动)
nohup /var/lib/kafka/bin/kafka-server-stop.sh
要么
nohup /var/lib/kafka/bin/kafka-server-stop.sh &
kafka-server-stop.sh
正确关闭Kafka实例.我怀疑这个问题可能是由某些Linux/Python引起的.