守护进程¶

Daemonization

中文

如今，大多数 Linux 发行版使用 systemd 来管理系统和用户服务的生命周期。

你可以通过输入以下命令检查你的 Linux 发行版是否使用 systemd：

$ systemctl --version
systemd 249 (v249.9-1.fc35)
+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified

如果你的输出类似于上面所示，请参考我们的 systemd 文档以获取更多指导。

然而，init.d 脚本在这些 Linux 发行版中仍然应该能正常工作，因为 systemd 提供了 systemd-sysv 兼容层，该层会自动根据我们提供的 init.d 脚本生成服务。

如果你为多个 Linux 发行版打包 Celery，并且有些不支持 systemd，或者打算支持其他 Unix 系统，你可能需要参考我们的 init.d 文档。

英文

Most Linux distributions these days use systemd for managing the lifecycle of system and user services.

You can check if your Linux distribution uses systemd by typing:

$ systemctl --version
systemd 249 (v249.9-1.fc35)
+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified

If you have output similar to the above, please refer to our systemd documentation for guidance.

However, the init.d script should still work in those Linux distributions as well since systemd provides the systemd-sysv compatibility layer which generates services automatically from the init.d scripts we provide.

If you package Celery for multiple Linux distributions and some do not support systemd or to other Unix systems as well, you may want to refer to our init.d documentation.

使用 `systemd`¶

Usage systemd

extra/systemd/

Usage:: systemctl {start|stop|restart|status} celery.service
Configuration file:: /etc/conf.d/celery

服务文件：celery.service¶

Service file: celery.service

中文

以下是一个 systemd 配置文件示例：

/etc/systemd/system/celery.service:

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=celery
Group=celery
EnvironmentFile=/etc/conf.d/celery
WorkingDirectory=/opt/celery
ExecStart=/bin/sh -c '${CELERY_BIN} -A $CELERY_APP multi start $CELERYD_NODES \
    --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} \
    --loglevel="${CELERYD_LOG_LEVEL}" $CELERYD_OPTS'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait $CELERYD_NODES \
    --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} \
    --loglevel="${CELERYD_LOG_LEVEL}"'
ExecReload=/bin/sh -c '${CELERY_BIN} -A $CELERY_APP multi restart $CELERYD_NODES \
    --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} \
    --loglevel="${CELERYD_LOG_LEVEL}" $CELERYD_OPTS'
Restart=always

[Install]
WantedBy=multi-user.target

将该文件放入 /etc/systemd/system 后，你应当运行 systemctl daemon-reload 来使 Systemd 识别该文件。每次修改该文件后也需要运行此命令。如果你希望在系统启动或重启时自动启动 celery 服务，可以运行 systemctl enable celery.service。

你也可以为 celery 服务指定额外的依赖项：例如，如果你使用 RabbitMQ 作为 broker，可以在 [Unit] 的 After= 和 Requires= 中同时指定 rabbitmq-server.service。详细选项可参考 systemd section。

要配置运行用户、用户组、以及目录切换行为，可编辑 /etc/systemd/system/celery.service 中的 User、Group 和 WorkingDirectory。

你也可以使用 systemd-tmpfiles 创建工作目录（如日志目录和 PID 文件目录）：

/etc/tmpfiles.d/celery.conf

d /run/celery 0755 celery celery - d /var/log/celery 0755 celery celery -

英文

This is an example systemd file:

/etc/systemd/system/celery.service:

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=celery
Group=celery
EnvironmentFile=/etc/conf.d/celery
WorkingDirectory=/opt/celery
ExecStart=/bin/sh -c '${CELERY_BIN} -A $CELERY_APP multi start $CELERYD_NODES \
    --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} \
    --loglevel="${CELERYD_LOG_LEVEL}" $CELERYD_OPTS'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait $CELERYD_NODES \
    --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} \
    --loglevel="${CELERYD_LOG_LEVEL}"'
ExecReload=/bin/sh -c '${CELERY_BIN} -A $CELERY_APP multi restart $CELERYD_NODES \
    --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} \
    --loglevel="${CELERYD_LOG_LEVEL}" $CELERYD_OPTS'
Restart=always

[Install]
WantedBy=multi-user.target

Once you've put that file in /etc/systemd/system, you should run systemctl daemon-reload in order that Systemd acknowledges that file. You should also run that command each time you modify it. Use systemctl enable celery.service if you want the celery service to automatically start when (re)booting the system.

Optionally you can specify extra dependencies for the celery service: e.g. if you use RabbitMQ as a broker, you could specify rabbitmq-server.service in both After= and Requires= in the [Unit] systemd section.

To configure user, group, chdir change settings: User, Group, and WorkingDirectory defined in /etc/systemd/system/celery.service.

You can also use systemd-tmpfiles in order to create working directories (for logs and pid).

file:: /etc/tmpfiles.d/celery.conf

d /run/celery 0755 celery celery -
d /var/log/celery 0755 celery celery -

示例配置¶

Example configuration

中文

以下是一个针对 Python 项目的配置文件示例：

/etc/conf.d/celery:

# 要启动的节点名称
# 这里仅启动一个节点
CELERYD_NODES="w1"
# 或者启动三个节点：
#CELERYD_NODES="w1 w2 w3"

# 'celery' 命令的绝对路径或相对路径：
CELERY_BIN="/usr/local/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# 要使用的应用实例
# 如果未使用应用可注释此行
CELERY_APP="proj"
# 或使用完整限定名：
#CELERY_APP="proj.tasks:app"

# 调用 manage.py 的方式
CELERYD_MULTI="multi"

# 传递给 worker 的额外命令行参数
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# - %n 会被替换为节点名称的第一部分
# - %I 会被替换为当前子进程索引
#   当使用 prefork 池时，该参数有助于避免竞争条件
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"

# 如需使用 Celery Beat，可添加以下选项
CELERYBEAT_PID_FILE="/var/run/celery/beat.pid"
CELERYBEAT_LOG_FILE="/var/log/celery/beat.log"

英文

This is an example configuration for a Python project:

/etc/conf.d/celery:

# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/usr/local/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="proj"
# or fully qualified:
#CELERY_APP="proj.tasks:app"

# How to call manage.py
CELERYD_MULTI="multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# - %n will be replaced with the first part of the nodename.
# - %I will be replaced with the current child process index
#   and is important when using the prefork pool to avoid race conditions.
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"

# you may wish to add these options for Celery Beat
CELERYBEAT_PID_FILE="/var/run/celery/beat.pid"
CELERYBEAT_LOG_FILE="/var/log/celery/beat.log"

服务文件：celerybeat.service¶

Service file: celerybeat.service

中文

以下是一个 Celery Beat 的 systemd 服务配置示例：

/etc/systemd/system/celerybeat.service:

[Unit]
Description=Celery Beat Service
After=network.target

[Service]
Type=simple
User=celery
Group=celery
EnvironmentFile=/etc/conf.d/celery
WorkingDirectory=/opt/celery
ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} beat  \
    --pidfile=${CELERYBEAT_PID_FILE} \
    --logfile=${CELERYBEAT_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}'
Restart=always

[Install]
WantedBy=multi-user.target

将该文件放入 /etc/systemd/system 后，你应当运行 systemctl daemon-reload 来使 Systemd 识别该文件。每次修改该文件后也需要运行此命令。如果你希望在系统启动或重启时自动启动 celery beat 服务，可以运行 systemctl enable celerybeat.service。

英文

This is an example systemd file for Celery Beat:

/etc/systemd/system/celerybeat.service:

[Unit]
Description=Celery Beat Service
After=network.target

[Service]
Type=simple
User=celery
Group=celery
EnvironmentFile=/etc/conf.d/celery
WorkingDirectory=/opt/celery
ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} beat  \
    --pidfile=${CELERYBEAT_PID_FILE} \
    --logfile=${CELERYBEAT_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}'
Restart=always

[Install]
WantedBy=multi-user.target

Once you've put that file in /etc/systemd/system, you should run systemctl daemon-reload in order that Systemd acknowledges that file. You should also run that command each time you modify it. Use systemctl enable celerybeat.service if you want the celery beat service to automatically start when (re)booting the system.

使用超级用户权限 (root) 运行工作器¶

Running the worker with superuser privileges (root)

中文

以超级用户权限运行 worker 是非常危险的做法。应始终寻找替代方案，避免以 root 用户身份运行。因为 Celery 可能会执行消息中使用 pickle 序列化的任意代码 —— 这在以 root 身份运行时尤其危险。

默认情况下，Celery 不允许以 root 身份运行 worker。相关的错误信息可能不会出现在日志中，但如果使用了 C_FAKEFORK，则可能会显示出来。

若确实需要以 root 身份运行 Celery worker，可通过设置 C_FORCE_ROOT 强制允许。

如果以 root 身份运行但未设置 C_FORCE_ROOT，worker 表面上会以 "OK" 状态启动，但随后会立即退出，且没有明显错误提示。这种问题通常出现在项目在新的开发或生产环境中（无意间）以 root 用户身份运行时。

英文

Running the worker with superuser privileges is a very dangerous practice. There should always be a workaround to avoid running as root. Celery may run arbitrary code in messages serialized with pickle - this is dangerous, especially when run as root.

By default Celery won't run workers as root. The associated error message may not be visible in the logs but may be seen if C_FAKEFORK is used.

To force Celery to run workers as root use C_FORCE_ROOT.

When running as root without C_FORCE_ROOT the worker will appear to start with "OK" but exit immediately after with no apparent errors. This problem may appear when running the project in a new development or production environment (inadvertently) as root.

supervisor¶

https://pypi.org/project/supervisor/

extra/supervisord/

`launchd` (macOS)¶

extra/macOS

守护进程¶

通用初始化脚本¶

初始化脚本： `celeryd`¶

示例配置¶

使用登录 shell¶

示例 Django 配置¶

可用选项¶

初始化脚本：`celerybeat`¶

示例配置¶

示例 Django 配置¶

可用选项¶

故障排除¶

使用 `systemd`¶

服务文件：celery.service¶

示例配置¶

服务文件：celerybeat.service¶

使用超级用户权限 (root) 运行工作器¶

supervisor¶

`launchd` (macOS)¶

守护进程¶

通用初始化脚本¶

初始化脚本： celeryd¶

示例配置¶

使用登录 shell¶

示例 Django 配置¶

可用选项¶

初始化脚本：celerybeat¶

示例配置¶

示例 Django 配置¶

可用选项¶

故障排除¶

使用 systemd¶

服务文件：celery.service¶

示例配置¶

服务文件：celerybeat.service¶

使用超级用户权限 (root) 运行工作器¶

supervisor¶

launchd (macOS)¶

初始化脚本： `celeryd`¶

初始化脚本：`celerybeat`¶

使用 `systemd`¶

`launchd` (macOS)¶