Celery 未连接到 Redis 代理 : Connection to broker lost

2022-11-28 22:47:48 ⋅ 9172 ⋅ 0 ⋅ 0

一、问题

我正在尝试让 Redis 作为我在 Django 上安装的 Celery 3.0.19 的代理。我看到 redis-server 正在端口 6379 上运行。当我运行一个简单的 Celery 测试时，我得到以下堆栈跟踪:

Ubuntu Lucid 10.0.4
Celery 3.0.19

celery -A tasks worker --loglevel=info

[2013-05-02 18:56:27,835: INFO/MainProcess] consumer: Connected to redis://127.0.0.1:6379/0.
[2013-05-02 18:56:27,835: ERROR/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/celery/worker/consumer.py", line 394, in start
self.reset_connection()
File "/usr/local/lib/python2.6/dist-packages/celery/worker/consumer.py", line 744, in reset_connection
self.connection, on_decode_error=self.on_decode_error,
File "/usr/local/lib/python2.6/dist-packages/celery/app/amqp.py", line 311, in __init__
**kw
File "/usr/local/lib/python2.6/dist-packages/kombu/messaging.py", line 355, in __init__
self.revive(self.channel)
File "/usr/local/lib/python2.6/dist-packages/kombu/messaging.py", line 367, in revive
self.declare()
File "/usr/local/lib/python2.6/dist-packages/kombu/messaging.py", line 377, in declare
queue.declare()
File "/usr/local/lib/python2.6/dist-packages/kombu/entity.py", line 490, in declare
self.queue_declare(nowait, passive=False)
File "/usr/local/lib/python2.6/dist-packages/kombu/entity.py", line 516, in queue_declare
nowait=nowait)
File "/usr/local/lib/python2.6/dist-packages/kombu/transport/virtual/__init__.py", line 404, in queue_declare
return queue, self._size(queue), 0
File "/usr/local/lib/python2.6/dist-packages/kombu/transport/redis.py", line 516, in _size
sizes = cmds.execute()
File "/usr/local/lib/python2.6/dist-packages/redis/client.py", line 1919, in execute
return execute(conn, stack, raise_on_error)
File "/usr/local/lib/python2.6/dist-packages/redis/client.py", line 1811, in _execute_transaction
self.parse_response(connection, '_')
File "/usr/local/lib/python2.6/dist-packages/redis/client.py", line 1882, in parse_response
self, connection, command_name, **options)
File "/usr/local/lib/python2.6/dist-packages/redis/client.py", line 387, in parse_response
response = connection.read_response()
File "/usr/local/lib/python2.6/dist-packages/redis/connection.py", line 312, in read_response
raise response
ResponseError: unknown command 'MULTI'

实际报错：

[2022-11-28 17:58:56,351: WARNING/MainProcess] Exception in thread 
[2022-11-28 17:58:56,352: WARNING/MainProcess] Thread-1
[2022-11-28 17:58:56,352: WARNING/MainProcess] :
[2022-11-28 17:58:56,352: WARNING/MainProcess] Traceback (most recent call last):
[2022-11-28 17:58:56,353: WARNING/MainProcess]   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
[2022-11-28 17:58:56,353: WARNING/MainProcess]     
[2022-11-28 17:58:56,353: WARNING/MainProcess] httplib_response = self._make_request(
[2022-11-28 17:58:56,353: WARNING/MainProcess]   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 449, in _make_request
[2022-11-28 17:58:56,353: WARNING/MainProcess]     
[2022-11-28 17:58:56,353: WARNING/MainProcess] six.raise_from(e, None)
[2022-11-28 17:58:56,353: WARNING/MainProcess]   File "<string>", line 3, in raise_from
[2022-11-28 17:58:56,354: WARNING/MainProcess]   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 444, in _make_request
[2022-11-28 17:58:56,354: WARNING/MainProcess]     
[2022-11-28 17:58:56,352: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 326, in start
    blueprint.start(self)
  File "/usr/local/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start
    step.start(parent)
  File "/usr/local/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 618, in start
    c.loop(*c.loop_args())
  File "/usr/local/lib/python3.8/site-packages/celery/worker/loops.py", line 97, in asynloop
    next(loop)
  File "/usr/local/lib/python3.8/site-packages/kombu/asynchronous/hub.py", line 362, in create_loop
    cb(*cbargs)
  File "/usr/local/lib/python3.8/site-packages/kombu/transport/redis.py", line 1326, in on_readable
    self.cycle.on_readable(fileno)
  File "/usr/local/lib/python3.8/site-packages/kombu/transport/redis.py", line 562, in on_readable
    chan.handlers[type]()
  File "/usr/local/lib/python3.8/site-packages/kombu/transport/redis.py", line 955, in _brpop_read
    dest__item = self.client.parse_response(self.client.connection,
  File "/usr/local/lib/python3.8/site-packages/redis/client.py", line 1084, in parse_response
    response = connection.read_response()
  File "/usr/local/lib/python3.8/site-packages/redis/connection.py", line 748, in read_response
    response = self._parser.read_response()
  File "/usr/local/lib/python3.8/site-packages/redis/connection.py", line 318, in read_response
    raw = self._buffer.readline()
  File "/usr/local/lib/python3.8/site-packages/redis/connection.py", line 250, in readline
    self._read_from_socket()
  File "/usr/local/lib/python3.8/site-packages/redis/connection.py", line 195, in _read_from_socket
    raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
redis.exceptions.ConnectionError: Connection closed by server.
[2022-11-28 17:58:56,351: WARNING/MainProcess] Exception in thread 
[2022-11-28 17:58:56,356: WARNING/MainProcess] Thread-2
[2022-11-28 17:58:56,356: WARNING/MainProcess] :
[2022-11-28 17:58:56,357: WARNING/MainProcess] Traceback (most recent call last):
[2022-11-28 17:58:56,356: WARNING/MainProcess] /usr/local/lib/python3.8/site-packages/celery/worker/consumer/consumer.py:361: CPendingDeprecationWarning: 
In Celery 5.1 we introduced an optional breaking change which
on connection loss cancels all currently executed tasks with late acknowledgement enabled.
These tasks cannot be acknowledged as the connection is gone, and the tasks are automatically redelivered back to the queue.
You can enable this behavior using the worker_cancel_long_running_tasks_on_connection_loss setting.
In Celery 5.1 it is set to False by default. The setting will be set to True by default in Celery 6.0.

  warnings.warn(CANCEL_TASKS_BY_DEFAULT, CPendingDeprecationWarning)

[2022-11-28 17:58:56,357: WARNING/MainProcess]   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
[2022-11-28 17:58:56,354: WARNING/MainProcess] httplib_response = conn.getresponse()
[2022-11-28 17:58:56,358: WARNING/MainProcess]   File "/usr/local/lib/python3.8/http/client.py", line 1348, in getresponse
[2022-11-28 17:58:56,357: WARNING/MainProcess]     
[2022-11-28 17:58:56,359: WARNING/MainProcess] httplib_response = self._make_request(
[2022-11-28 17:58:56,359: WARNING/MainProcess]   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 449, in _make_request
[2022-11-28 17:58:56,359: WARNING/MainProcess]     
[2022-11-28 17:58:56,362: WARNING/MainProcess] response.begin()
[2022-11-28 17:58:56,362: ERROR/MainProcess] consumer: Cannot connect to redis://:**@192.168.1.61:8801/1: Error 111 connecting to 192.168.1.61:8801. Connection refused..
Trying again in 2.00 seconds... (1/100)

解决

You need redis version >= 2.2.0.

# 修改redis配置
daemonize yes

protected-mode yes 改为 protected-mode no # 关闭保护模式

修改完成后重启redis
再启动celery 问题完美解决

相关文章：
Celery not connecting to Redis broker: Connection to broker lost
CentOS7 离线安装 Redis

为者常成，行者常至

自由转载-非商用-非衍生-保持署名（创意共享3.0许可证）