如何解决django-celery启动后迅速关闭

 更新时间:2019年10月16日 14:35:15   作者:arrowxiang  
在本篇文章里小编给大家整理的是关于django-celery启动后迅速关闭的解决方法,有需要的朋友们学习下。

日志中也没有打印什么明显的错误,只是显示连接了rabbitmq后就关闭了

[2019-09-11 06:08:45,729: INFO/Beat] beat: Starting...
[2019-09-11 06:08:45,731: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-09-11 06:08:45,732: INFO/Beat] Writing entries (0)...
[2019-09-11 06:08:45,740: INFO/MainProcess] mingle: searching for neighbors
[2019-09-11 06:08:45,771: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-09-11 06:08:45,771: INFO/Beat] Writing entries (0)...
[2019-09-11 06:08:46,748: INFO/MainProcess] mingle: all alone
[2019-09-11 06:08:46,749: INFO/MainProcess] beat: Shutting down...
[2019-09-11 06:08:46,749: INFO/Beat] Writing entries (0)...
[2019-09-11 06:08:46,749: INFO/Beat] Writing entries (0)...

查看rabbitmq的日志,这里显示rabbitmq连接成功,然后就是断开连接,(但是我还是以为是rabbitmq的问题,因为我在另外一台主机上一摸一样的环境没有问题)

=INFO REPORT==== 15-Oct-2019::02:17:54 ===
connection <0.305.0> (127.0.0.1:39786 -> 127.0.0.1:5672): user 'guest' authenticated and granted access to vhost '/'

=INFO REPORT==== 15-Oct-2019::02:17:56 ===
connection <0.310.0> (127.0.0.1:39788 -> 127.0.0.1:5672): user 'guest' authenticated and granted access to vhost '/'

=WARNING REPORT==== 15-Oct-2019::02:17:57 ===
closing AMQP connection <0.305.0> (127.0.0.1:39786 -> 127.0.0.1:5672, vhost: '/', user: 'guest'):
client unexpectedly closed TCP connection

=INFO REPORT==== 15-Oct-2019::02:17:58 ===
closing AMQP connection <0.310.0> (127.0.0.1:39788 -> 127.0.0.1:5672, vhost: '/', user: 'guest')

=INFO REPORT==== 15-Oct-2019::02:17:58 ===
closing AMQP connection <0.302.0> (127.0.0.1:39780 -> 127.0.0.1:5672, vhost: '/', user: 'guest')

在试了重装erlang和rabbitmq等服务,折腾了一天后,第二天我换了个思路,从celery源码查看

找到Shutting down...这个日志输出的位置

def stop(self, wait=False):
    info('beat: Shutting down...')
    # 这里增加一段代码
    ''''''
    import traceback
    info("start shutdown traceback ====")
    traceback.print_stack()
    ''''''
    self._is_shutdown.set()
    wait and self._is_stopped.wait() # block until shutdown done.

再次启动celery后获得以下日志

[2019-10-16 10:15:15,037: INFO/Beat] beat: Starting...
[2019-10-16 10:15:15,039: INFO/Beat] Writing entries (0)...
[2019-10-16 10:15:15,082: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:15:15,082: INFO/Beat] Writing entries (0)...
[2019-10-16 10:15:15,394: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:15:15,394: INFO/Beat] Writing entries (0)...
[2019-10-16 10:15:16,201: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:15:16,201: INFO/Beat] Writing entries (0)...
[2019-10-16 10:15:16,859: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:15:16,859: INFO/Beat] Writing entries (0)...
[2019-10-16 10:15:23,337: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-10-16 10:15:28,102: INFO/Beat] Writing entries (0)...
[2019-10-16 10:15:31,348: INFO/MainProcess] mingle: searching for neighbors
[2019-10-16 10:15:32,357: INFO/MainProcess] mingle: all alone
[2019-10-16 10:15:32,362: WARNING/MainProcess] invalid syntax (strategy.py, line 13)
[2019-10-16 10:15:33,372: INFO/MainProcess] start shutdown traceback ====
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/django/core/management/__init__.py"
, line 381, in execute_from_command_line utility.execute()
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/django/core/management/__init__.py"
, line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv)
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/djcelery/management/commands/celery
.py", line 21, in run_from_argv ['{0[0]} {0[1]}'.format(argv)] + argv[2:],
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/celery.py", line 793, in
execute_from_commandline super(CeleryCommand, self).execute_from_commandline(argv)))
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/base.py", line 311, in e
xecute_from_commandline return self.handle_argv(self.prog_name, argv[1:])
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/celery.py", line 785, in
handle_argv return self.execute(command, argv)
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/celery.py", line 717, in
execute ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/worker.py", line 179, in
run_from_argv return self(*args, **options)
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/base.py", line 274, in _
_call__ ret = self.run(*args, **kwargs)
[2019-10-16 10:15:33,374: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/worker.py", line 212, in
run state_db=self.node_format(state_db, hostname), **kwargs
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/worker/__init__.py", line 21
2, in start self.stop()
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/worker/__init__.py", line 25
4, in stop self._shutdown(warm=True)
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/worker/__init__.py", line 26
8, in _shutdown self.blueprint.stop(self, terminate=not warm)
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bootsteps.py", line 179, in
stop description=what, propagate=False,
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bootsteps.py", line 142, in
restart self.send_all(parent, method, description, propagate=propagate)
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bootsteps.py", line 155, in
send_all fun(parent, *args)
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bootsteps.py", line 378, in
stop return self.obj.stop()
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/beat.py", line 559, in stop
self.service.stop()
[2019-10-16 10:15:33,375: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/beat.py", line 499, in stop
traceback.print_stack()
[2019-10-16 10:15:33,375: INFO/MainProcess] beat: Shutting down...
[2019-10-16 10:15:33,375: INFO/Beat] Writing entries (0)...
[2019-10-16 10:15:33,376: INFO/Beat] Writing entries (0)...
[root@test operations]# [2019-10-16 10:15:51,178: INFO/Beat] Writing entries (0)...
[2019-10-16 10:16:30,352: INFO/Beat] Writing entries (0)...
[2019-10-16 10:16:46,806: INFO/Beat] Writing entries (0)...
[2019-10-16 10:18:56,064: INFO/Beat] Writing entries (0)...
[2019-10-16 10:19:35,231: INFO/Beat] Writing entries (0)...
[2019-10-16 10:19:51,689: INFO/Beat] Writing entries (0)...

可以看出在/usr/local/python3/lib/python3.7/site-packages/celery/worker/__init__.py中的start方法中执行了stop方法,在stop中打印了beat: Shutting down...字符

def start(self):
    """Starts the workers main loop."""
    '''
    修改start方法,去掉try,以便找到错误位置
    '''
    self.blueprint.start(self)
    #try:
    #  self.blueprint.start(self)
    #except WorkerTerminate:
    #  self.terminate()
    #except Exception as exc:

    #  logger.error('Unrecoverable error: %r', exc, exc_info=True)
    #  self.stop()
    #except (KeyboardInterrupt, SystemExit):
    #  self.stop()

然后再次启动celery获得以下日志

[2019-10-16 10:19:57,241: INFO/Beat] beat: Starting...
[2019-10-16 10:19:57,244: INFO/Beat] Writing entries (0)...
[2019-10-16 10:19:57,287: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:19:57,287: INFO/Beat] Writing entries (0)...
[2019-10-16 10:20:00,218: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:20:00,218: INFO/Beat] Writing entries (0)...
[2019-10-16 10:20:01,023: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:20:01,023: INFO/Beat] Writing entries (0)...
[2019-10-16 10:20:01,685: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:20:01,685: INFO/Beat] Writing entries (0)...
[2019-10-16 10:20:05,548: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-10-16 10:20:10,307: INFO/Beat] Writing entries (0)...
[2019-10-16 10:20:13,561: INFO/MainProcess] mingle: searching for neighbors
[2019-10-16 10:20:14,570: INFO/MainProcess] mingle: all alone
[2019-10-16 10:20:14,571: WARNING/MainProcess] Traceback (most recent call last):
[2019-10-16 10:20:14,571: WARNING/MainProcess] File "manage.py", line 15, in <module>
[2019-10-16 10:20:14,571: WARNING/MainProcess] execute_from_command_line(sys.argv)
[2019-10-16 10:20:14,571: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/django/core/management/__init__.py"
, line 381, in execute_from_command_line[2019-10-16 10:20:14,572: WARNING/MainProcess] utility.execute()
[2019-10-16 10:20:14,572: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/django/core/management/__init__.py"
, line 375, in execute[2019-10-16 10:20:14,572: WARNING/MainProcess] self.fetch_command(subcommand).run_from_argv(self.argv)
[2019-10-16 10:20:14,572: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/djcelery/management/commands/celery
.py", line 21, in run_from_argv[2019-10-16 10:20:14,572: WARNING/MainProcess] ['{0[0]} {0[1]}'.format(argv)] + argv[2:],
[2019-10-16 10:20:14,572: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/celery.py", line 793, in
execute_from_commandline[2019-10-16 10:20:14,573: WARNING/MainProcess] super(CeleryCommand, self).execute_from_commandline(argv)))
[2019-10-16 10:20:14,573: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/base.py", line 311, in e
xecute_from_commandline[2019-10-16 10:20:14,573: WARNING/MainProcess] return self.handle_argv(self.prog_name, argv[1:])
[2019-10-16 10:20:14,573: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/celery.py", line 785, in
handle_argv[2019-10-16 10:20:14,574: WARNING/MainProcess] return self.execute(command, argv)
[2019-10-16 10:20:14,574: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/celery.py", line 717, in
execute[2019-10-16 10:20:14,574: WARNING/MainProcess] ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
[2019-10-16 10:20:14,574: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/worker.py", line 179, in
run_from_argv[2019-10-16 10:20:14,574: WARNING/MainProcess] return self(*args, **options)
[2019-10-16 10:20:14,575: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/base.py", line 274, in _
_call__[2019-10-16 10:20:14,575: WARNING/MainProcess] ret = self.run(*args, **kwargs)
[2019-10-16 10:20:14,575: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bin/worker.py", line 212, in
run[2019-10-16 10:20:14,575: WARNING/MainProcess] state_db=self.node_format(state_db, hostname), **kwargs
[2019-10-16 10:20:14,575: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/worker/__init__.py", line 20
5, in start[2019-10-16 10:20:14,575: WARNING/MainProcess] self.blueprint.start(self)
[2019-10-16 10:20:14,575: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bootsteps.py", line 123, in
start[2019-10-16 10:20:14,576: WARNING/MainProcess] step.start(parent)
[2019-10-16 10:20:14,576: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bootsteps.py", line 374, in
start[2019-10-16 10:20:14,576: WARNING/MainProcess] return self.obj.start()
[2019-10-16 10:20:14,576: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/worker/consumer.py", line 28
0, in start[2019-10-16 10:20:14,576: WARNING/MainProcess] blueprint.start(self)
[2019-10-16 10:20:14,577: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/bootsteps.py", line 123, in
start[2019-10-16 10:20:14,577: WARNING/MainProcess] step.start(parent)
[2019-10-16 10:20:14,577: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/worker/consumer.py", line 64
2, in start[2019-10-16 10:20:14,577: WARNING/MainProcess] c.update_strategies()
[2019-10-16 10:20:14,577: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/worker/consumer.py", line 43
9, in update_strategies[2019-10-16 10:20:14,578: WARNING/MainProcess] self.strategies[name] = task.start_strategy(self.app, self)
[2019-10-16 10:20:14,578: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/app/task.py", line 448, in s
tart_strategy[2019-10-16 10:20:14,578: WARNING/MainProcess] return instantiate(self.Strategy, self, app, consumer, **kwargs)
[2019-10-16 10:20:14,578: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/utils/imports.py", line 53,
in instantiate[2019-10-16 10:20:14,578: WARNING/MainProcess] return symbol_by_name(name)(*args, **kwargs)
[2019-10-16 10:20:14,578: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/kombu/utils/__init__.py", line 96,
in symbol_by_name[2019-10-16 10:20:14,578: WARNING/MainProcess] module = imp(module_name, package=package, **kwargs)
[2019-10-16 10:20:14,579: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/importlib/__init__.py", line 127, in import_modul
e[2019-10-16 10:20:14,579: WARNING/MainProcess] return _bootstrap._gcd_import(name[level:], package, level)
[2019-10-16 10:20:14,579: WARNING/MainProcess] File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
[2019-10-16 10:20:14,579: WARNING/MainProcess] File "<frozen importlib._bootstrap>", line 983, in _find_and_load
[2019-10-16 10:20:14,579: WARNING/MainProcess] File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
[2019-10-16 10:20:14,579: WARNING/MainProcess] File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
[2019-10-16 10:20:14,579: WARNING/MainProcess] File "<frozen importlib._bootstrap_external>", line 724, in exec_module
[2019-10-16 10:20:14,579: WARNING/MainProcess] File "<frozen importlib._bootstrap_external>", line 860, in get_code
[2019-10-16 10:20:14,580: WARNING/MainProcess] File "<frozen importlib._bootstrap_external>", line 791, in source_to_code
[2019-10-16 10:20:14,580: WARNING/MainProcess] File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
[2019-10-16 10:20:14,580: WARNING/MainProcess] File "/usr/local/python3/lib/python3.7/site-packages/celery/worker/strategy.py", line 13
[2019-10-16 10:20:14,580: WARNING/MainProcess] from kombu.async.timer import to_timestamp
[2019-10-16 10:20:14,580: WARNING/MainProcess] ^
[2019-10-16 10:20:14,580: WARNING/MainProcess] SyntaxError
[2019-10-16 10:20:14,580: WARNING/MainProcess] :
[2019-10-16 10:20:14,580: WARNING/MainProcess] invalid syntax

可以看到from kombu.async.timer import to_timestamp这句导包的时候报错 在kombu中把async目录修改成了asynchronous,同时因为python3.7以后吧async变成了一个关键字,不能作为包来引用,因此报错,所以修改async为asynchronous

再次启动celery后获得以下日志

[2019-10-16 10:27:14,497: INFO/Beat] beat: Starting...
[2019-10-16 10:27:14,500: INFO/Beat] Writing entries (0)...
[2019-10-16 10:27:14,545: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:27:14,545: INFO/Beat] Writing entries (0)...
[2019-10-16 10:27:14,949: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:27:14,949: INFO/Beat] Writing entries (0)...
[2019-10-16 10:27:15,029: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:27:15,029: INFO/Beat] Writing entries (0)...
[2019-10-16 10:27:15,748: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:27:15,748: INFO/Beat] Writing entries (0)...
[2019-10-16 10:27:16,427: INFO/Beat] DatabaseScheduler: Schedule changed.
[2019-10-16 10:27:16,427: INFO/Beat] Writing entries (0)...
[2019-10-16 10:27:22,485: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-10-16 10:27:24,618: INFO/MainProcess] mingle: searching for neighbors
[2019-10-16 10:27:25,626: INFO/MainProcess] mingle: all alone
[2019-10-16 10:27:25,648: WARNING/MainProcess] /usr/local/python3/lib/python3.7/site-packages/djcelery/loaders.py:133: UserWarning: Usi
ng settings.DEBUG leads to a memory leak, never use this setting in production environments! warn('Using settings.DEBUG leads to a memory leak, never '
[2019-10-16 10:27:25,648: WARNING/MainProcess] celery@test.ylyw.com ready.

当出现celery@test.ylyw.com ready.后,就可以确定celery正常运行了

希望上面整理的内容能够帮助到大家,感谢大家的学习和对脚本之家的支持。

相关文章

  • Python实现批量合并多个txt文件并生成Excel文件

    Python实现批量合并多个txt文件并生成Excel文件

    在数据处理中,有时会面临合并多个文本文件的任务,本文将详细介绍如何使用Python批量合并多个txt文件,并将其生成为一个Excel文件,需要的可以参考下
    2023-12-12
  • 详解python eval函数的妙用

    详解python eval函数的妙用

    这篇文章主要介绍了详解python eval函数的妙用,详细介绍了python eval函数的具体用法和实例,有兴趣的可以了解一下
    2017-11-11
  • Python调用实现最小二乘法的方法详解

    Python调用实现最小二乘法的方法详解

    所谓线性最小二乘法,可以理解为是解方程的延续,区别在于,当未知量远小于方程数的时候,将得到一个无解的问题。本文主要和大家分享Python调用实现最小二乘法的方法,需要的可以参考一下
    2023-04-04
  • Python安装及Pycharm安装使用教程图解

    Python安装及Pycharm安装使用教程图解

    这篇文章主要介绍了Python安装以及Pycharm安装使用教程,本文图文并茂给大家介绍的非常详细,具有一定的参考借鉴价值,需要的朋友可以参考下
    2019-09-09
  • Python CSV 文件解析和生成方法示例

    Python CSV 文件解析和生成方法示例

    这篇文章主要为大家介绍了Python CSV 文件的解析和生成方法示例详解,有需要的朋友可以借鉴参考下,希望能够有所帮助,祝大家多多进步,早日升职加薪
    2022-07-07
  • 简述 Python 的类和对象

    简述 Python 的类和对象

    这篇文章主要介绍了Python 的类和对象的相关资料,帮助大家更好的理解和学习python,感兴趣的朋友可以了解下
    2020-08-08
  • Python 类方法和实例方法(@classmethod),静态方法(@staticmethod)原理与用法分析

    Python 类方法和实例方法(@classmethod),静态方法(@staticmethod)原理与用法分析

    这篇文章主要介绍了Python 类方法和实例方法(@classmethod),静态方法(@staticmethod),结合实例形式分析了Python 类方法和实例方法及静态方法相关原理、用法及相关操作注意事项,需要的朋友可以参考下
    2019-09-09
  • python使用多进程的实例详解

    python使用多进程的实例详解

    本文通过实例代码给大家介绍了python使用多进程的方法,非常不错,具有一定的参考借鉴价值,需要的朋友参考下吧
    2018-09-09
  • Python实现对图像加噪(高斯噪声 椒盐噪声)

    Python实现对图像加噪(高斯噪声 椒盐噪声)

    这篇文章主要介绍了展示通过Python给图像叠加不同等级的椒盐噪声和高斯噪声的代码,相应的叠加噪声的已编为对应的类,可实例化使用。感兴趣的同学可以看看
    2021-11-11
  • Python如何在for循环中同时使用两个变量与两个控制条件

    Python如何在for循环中同时使用两个变量与两个控制条件

    Python是一种广泛使用的编程语言,其提供了许多强大的方法来处理代码,Python for循环是其中一种非常有用的方法,下面这篇文章主要给大家介绍了关于Python如何在for循环中同时使用两个变量与两个控制条件的相关资料,需要的朋友可以参考下
    2024-03-03

最新评论