3. Asyncio 在什么时候使用

3. When to Use Asyncio

中文英文

从广义上讲，Asyncio 是新的、流行的、讨论较多且令人兴奋的。

然而，对于何时应该在项目中采用它存在很多困惑。

我们什么时候应该在Python中使用asyncio？

Asyncio, broadly, is new, popular, much discussed, and exciting.

Nevertheless, there is a lot of confusion over when it should be adopted in a project.

When should we use asyncio in Python?

3.1 使用Python Asyncio的原因

3.1 Reasons to Use Asyncio in Python

中文英文

在 Python 项目中使用 asyncio 可能有 3 个最重要的原因。

他们是:

1. 使用 asyncio 以便在程序中采用协程。
2. 使用 asyncio 以便使用异步编程范例。
3. 使用 asyncio 以便使用非阻塞I/O。

There are perhaps 3 top-level reasons to use asyncio in a Python project.

They are:

Use asyncio in order to adopt coroutines in your program. Use asyncio in order to use the asynchronous programming paradigm. Use asyncio in order to use non-blocking I/O.

3.1.1 原因 1: 使用协程

3.1.1 Reason 1: To Use Coroutines

中文英文

我们可能会选择使用 asyncio，因为我们想使用协程。

我们可能想要使用协程，因为我们的程序中可以有比并发线程更多的并发协程。

协程是另一个并发单元，就像线程和进程一样。

基于线程的并发由线程模块提供，并由底层操作系统支持。 它适合阻塞 I/O 任务，例如从文件、套接字和设备读取和写入。

基于进程的并发性由多处理模块提供，并且也受到底层操作系统（如线程）的支持。 它适用于不需要太多进程间通信的 CPU 密集型任务，例如计算任务。

协程是由 Python 语言和运行时（标准解释器）提供的替代方案，并由 asyncio 模块进一步支持。 它们适用于带有子进程和套接字的非阻塞 I/O，但是，阻塞 I/O 和 CPU 密集型任务可以在幕后使用线程和进程以模拟非阻塞方式使用。

最后一点是微妙而关键的。 尽管我们可以选择使用协程来实现 Python 中引入的非阻塞功能，但实际上我们可以将它们用于任何任务。 如果我们愿意，任何使用线程或进程编写的程序都可以重写或使用协程编写。

线程和进程通过操作系统选择哪些线程和进程应该运行、何时运行以及运行多长时间来实现多任务处理。 操作在线程和进程之间快速切换，暂停那些未运行的线程和进程，并恢复那些授予时间运行的线程和进程。 这称为抢占式多任务处理。

Python 中的协程提供了另一种多任务处理类型，称为协作多任务处理(cooperating multitasking)。

协程是可以暂停和恢复的子例程（函数）。 它由 await 表达式挂起，并在 await 表达式解析后恢复。

这允许协程通过设计进行合作，选择如何以及何时暂停执行。

它是一种替代的、有趣的、强大的并发方法，不同于基于线程和基于进程的并发。

仅此一点就可以成为在项目中采用它的理由。

协程的另一个关键方面是它们是轻量级的。

它们比线程更轻量。 这意味着它们的启动速度更快并且使用的内存更少。 本质上，协程是一种特殊类型的函数，而线程则由 Python 对象表示，并与操作系统中必须与该对象交互的线程相关联。

因此，Python 程序中可能有数千个线程，但我们可以轻松地将数万或数十万个协程集中在一个线程中。

我们可能会因为协程的可扩展性而选择它们。

We may choose to use asyncio because we want to use coroutines.

We may want to use coroutines because we can have many more concurrent coroutines in our program than concurrent threads.

Coroutines are another unit of concurrency, like threads and processes.

Thread-based concurrency is provided by the threading module and is supported by the underlying operating system. It is suited to blocking I/O tasks such reading and writing from files, sockets, and devices.

Process-based concurrency is provided by the multiprocessing module and is also supported by the underlying operating system, like threads. It is suited to CPU-bound tasks that do not require much inter-process communication, such as compute tasks.

Coroutines are an alternative that is provided by the Python language and runtime (standard interpreter) and further supported by the asyncio module. They are suited to non-blocking I/O with subprocesses and sockets, however, blocking I/O and CPU-bound tasks can be used in a simulated non-blocking manner using threads and processes under the covers.

This last point is subtle and key. Although we can choose to use coroutines for the capability for which they were introduced into Python, non-blocking, we may in fact use them with any tasks. Any program written with threads or processes can be rewritten or instead written using coroutines if we so desire.

Threads and processes achieve multitasking via the operating system that chooses which threads and processes should run, when, and for how long. The operating switches between threads and processes rapidly, suspending those that are not running and resuming those granted time to run. This is called preemptive multitasking.

Coroutines in Python provide an alternative type of multitasking called cooperating multitasking.

A coroutine is a subroutine (function) that can be suspended and resumed. It is suspended by the await expression and resumed once the await expression is resolved.

This allows coroutines to cooperate by design, choosing how and when to suspend their execution.

It is an alternate, interesting, and powerful approach to concurrency, different from thread-based and process-based concurrency.

This alone may make it a reason to adopt it for a project.

Another key aspect of coroutines is that they are lightweight.

They are more lightweight than threads. This means they are faster to start and use less memory. Essentially a coroutine is a special type of function, whereas a thread is represented by a Python object and is associated with a thread in the operating system with which the object must interact.

As such, we may have thousands of threads in a Python program, but we could easily have tens or hundreds of thousands of coroutines all in one thread.

We may choose coroutines for their scalability.

3.1.2 原因 2: 使用异步编程

3.1.2 Reason 2: To Use Asynchronous Programming

中文英文

我们可能会选择使用asyncio，因为我们想在我们的程序中使用异步编程。

也就是说，我们要开发一个使用异步编程范式的Python程序。

异步意味着不同时，与同步或同时相对。

编程时，异步意味着请求操作，但未在请求时执行。 这是稍后执行的。

异步编程通常意味着围绕异步函数调用和任务的概念全力以赴并设计程序。

尽管还有其他方法可以实现异步编程的元素，但 Python 中的完全异步编程需要使用协程和 asyncio 模块。

它是一个 Python 库，允许我们使用异步编程模型运行代码。

— PAGE 3, PYTHON CONCURRENCY WITH ASYNCIO, 2022.

我们可能会选择使用asyncio，因为我们想在我们的程序中使用异步编程模块，这是一个有道理的理由。

明确地说，这个原因与使用非阻塞 I/O 无关。 异步编程可以独立于非阻塞 I/O 使用。

正如我们之前所看到的，协程可以异步执行非阻塞 I/O，但 asyncio 模块还提供了以异步方式执行阻塞 I/O 和 CPU 密集型任务的工具，通过线程和 流程。

We may choose to use asyncio because we want to use asynchronous programming in our program.

That is, we want to develop a Python program that uses the asynchronous programming paradigm.

Asynchronous means not at the same time, as opposed to synchronous or at the same time.

When programming, asynchronous means that the action is requested, although not performed at the time of the request. It is performed later.

Asynchronous programming often means going all in and designing the program around the concept of asynchronous function calls and tasks.

Although there are other ways to achieve elements of asynchronous programming, full asynchronous programming in Python requires the use of coroutines and the asyncio module.

It is a Python library that allows us to run code using an asynchronous programming model.

— PAGE 3, PYTHON CONCURRENCY WITH ASYNCIO, 2022.

We may choose to use asyncio because we want to use the asynchronous programming module in our program, and that is a defensible reason.

To be crystal clear, this reason is independent of using non-blocking I/O. Asynchronous programming can be used independently of non-blocking I/O.

As we saw previously, coroutines can execute non-blocking I/O asynchronously, but the asyncio module also provides the facility for executing blocking I/O and CPU-bound tasks in an asynchronous manner, simulating non-blocking under the covers via threads and processes.

3.1.3 原因 3: 使用非阻塞I/O

3.1.3 Reason 3: To Use Non-Blocking I/O

中文英文

我们可能会选择使用 asyncio，因为我们希望或需要在程序中使用非阻塞 I/O。

输入/输出或简称 I/O 表示从资源中读取或写入。

常见的例子包括：

• 硬盘驱动器：读取、写入、追加、重命名、删除等文件。
• 外设：鼠标、键盘、屏幕、打印机、串口、摄像头等。
• 互联网：下载和上传文件、获取网页、查询RSS等。
• 数据库：查询、更新、删除等SQL查询。
• 电子邮件：发送邮件、接收邮件、查询收件箱等。

与使用 CPU 进行计算相比，这些操作速度很慢。

在程序中实现这些操作的常见方式是发出读或写请求，然后等待数据发送或接收。

因此，这些操作通常称为阻塞 I/O 任务。

操作系统可以看到调用线程被阻塞，并将上下文切换到另一个将使用 CPU 的线程。

这意味着阻塞调用不会减慢整个系统的速度。 但它确实会停止或阻止进行阻塞调用的线程或程序。

这意味着阻塞调用不会减慢整个系统的速度。 但它确实会停止或阻止进行阻塞调用的线程或程序。

• Python 中的线程阻塞调用

非阻塞 I/O 是阻塞 I/O 的替代方案。

它需要底层操作系统的支持，就像阻塞 I/O 一样，所有现代操作系统都提供对某种形式的非阻塞 I/O 的支持。

非阻塞 I/O 允许将读取和写入调用作为异步请求进行。

操作系统将处理请求并在结果可用时通知调用程序。

• 非阻塞 I/O：通过异步请求和响应执行 I/O 操作，而不是等待操作完成。

因此，我们可以看到非阻塞 I/O 与异步编程的关系。 事实上，我们通过异步编程来使用非阻塞I/O，或者说非阻塞I/O是通过异步编程来实现的。

非阻塞 I/O 与异步编程的结合非常常见，因此通常简称为异步 I/O。

• 异步 I/O：一种简写，指的是异步编程与非阻塞 I/O 的结合。

Python 中的 asyncio 模块是专门为 Python 标准库添加对带有子进程（例如在操作系统上执行命令）和流（例如 TCP 套接字编程）的非阻塞 I/O 的支持而添加的。

我们可以使用线程以及Python线程池或线程池执行器提供的异步编程功能来模拟非阻塞I/O。

asyncio 模块通过协程、事件循环和对象来表示非阻塞子进程和流，为非阻塞 I/O 提供一流的异步编程。

我们可能会选择使用 asyncio，因为我们想在程序中使用异步 I/O，这是一个合理的理由。

We may choose to use asyncio because we want or require non-blocking I/O in our program.

Input/Output or I/O for short means reading or writing from a resource.

Common examples include:

• Hard disk drives: Reading, writing, appending, renaming, deleting, etc. files.
• Peripherals: mouse, keyboard, screen, printer, serial, camera, etc.
• Internet: Downloading and uploading files, getting a webpage, querying RSS, etc.
• Database: Select, update, delete, etc. SQL queries.
• Email: Send mail, receive mail, query inbox, etc.

These operations are slow, compared to calculating things with the CPU.

The common way these operations are implemented in programs is to make the read or write request and then wait for the data to be sent or received.

As such, these operations are commonly referred to as blocking I/O tasks.

The operating system can see that the calling thread is blocked and will context switch to another thread that will make use of the CPU.

This means that the blocking call does not slow down the entire system. But it does halt or block the thread or program making the blocking call.

You can learn more about blocking calls in the tutorial:

• Thread Blocking Call in Python

Non-blocking I/O is an alternative to blocking I/O.

It requires support in the underlying operating system, just like blocking I/O, and all modern operating systems provide support for some form of non-blocking I/O.

Non-blocking I/O allows read and write calls to be made as asynchronous requests.

The operating system will handle the request and notify the calling program when the results are available.

• Non-blocking I/O: Performing I/O operations via asynchronous requests and responses, rather than waiting for operations to complete.

As such, we can see how non-blocking I/O is related to asynchronous programming. In fact, we use non-blocking I/O via asynchronous programming, or non-blocking I/O is implemented via asynchronous programming.

The combination of non-blocking I/O with asynchronous programming is so common that it is commonly referred to by the shorthand of asynchronous I/O.

• Asynchronous I/O: A shorthand that refers to combining asynchronous programming with non-blocking I/O. T he asyncio module in Python was added specifically to add support for non-blocking I/O with subprocesses (e.g. executing commands on the operating system) and with streams (e.g. TCP socket programming) to the Python standard library.

We could simulate non-blocking I/O using threads and the asynchronous programming capability provided by Python thread pools or thread pool executors.

The asyncio module provides first-class asynchronous programming for non-blocking I/O via coroutines, event loops, and objects to represent non-blocking subprocesses and streams.

We may choose to use asyncio because we want to use asynchronous I/O in our program, and that is a defensible reason.

3.2 使用Asyncio的其他原因

3.2 Other Reasons to Use Asyncio

中文英文

理想情况下，我们会选择一个在项目要求的背景下得到辩护的理由。

有时我们可以控制函数和非函数需求，有时则不能。 在这种情况下，出于上述原因之一，我们可能会选择使用 asyncio。 在我们不这样做的情况下，我们可能会选择 asyncio 来交付解决特定问题的程序。

我们可能使用 asyncio 的其他一些原因包括：

1. 使用 asyncio 因为其他人已经为你做出了决定。
2. 使用 asyncio，因为您加入的项目已经在使用它。
3. 使用 asyncio 因为您想了解更多有关它的信息。

我们并不总是能够完全控制我们所从事的项目。

开始一项新工作、新角色或新项目并由直线经理或首席架构师告知具体设计和技术决策是很常见的。

使用 asyncio 可能是这些决定之一。

我们可能在项目上使用 asyncio，因为该项目已经在使用它。 你必须使用asyncio，而不是你选择使用asyncio。

一个相关的示例可能是使用您希望采用的 asyncio 解决问题的情况。

例如：

• 也许您需要使用第三方 API，并且代码示例使用 asyncio。
• 也许您需要集成使用 asyncio 的现有开源解决方案。
• 也许您偶然发现一些代码片段可以满足您的需要，但它们使用 asyncio。

由于缺乏替代解决方案，asyncio 可能会因您选择的解决方案而强加给您。

最后，我们可以为我们的Python项目选择asyncio来了解更多。

你可能会嘲笑，“那要求呢？”

您可能选择采用 asyncio 只是因为您想尝试一下，并且这是一个合理的理由。

在项目中使用 asyncio 将使您的工作更加具体。

Ideally, we would choose a reason that is defended in the context of the requirements of the project.

Sometimes we have control over the function and non-functional requirements and other times not. In the cases we do, we may choose to use asyncio for one of the reasons listed above. In the cases we don’t, we may be led to choose asyncio in order to deliver a program that solves a specific problem.

Some other reasons we may use asyncio include:

1. Use asyncio because someone else made the decision for you.
2. Use asyncio because the project you have joined is already using it.
3. Use asyncio because you want to learn more about it.

We don’t always have full control over the projects we work on.

It is common to start a new job, new role, or new project and be told by the line manager or lead architect of specific design and technology decisions.

Using asyncio may be one of these decisions.

We may use asyncio on a project because the project is already using it. You must use asyncio, rather than you choose to use asyncio.

A related example might be the case of a solution to a problem that uses asyncio that you wish to adopt.

For example:

• Perhaps you need to use a third-party API and the code examples use asyncio.
• Perhaps you need to integrate an existing open-source solution that uses asyncio.
• Perhaps you stumble across some code snippets that do what you need, yet they use asyncio.

For lack of alternate solutions, asyncio may be thrust upon you by your choice of solution.

Finally, we may choose asyncio for our Python project to learn more about.

You may scoff, “what about the requirements?”

You may choose to adopt asyncio just because you want to try it out and it can be a defensible reason.

Using asyncio in a project will make its workings concrete for you.

3.3 Asyncio在什么时候不适用

3.3 When to Not Use Asyncio

中文英文

我们花了很多时间来解释为什么应该使用 asyncio。

至少花一点时间来解释为什么我们不应该使用它可能是个好主意。

不使用 asyncio 的原因之一是您无法使用上述原因之一来捍卫它的使用。

这并不是万无一失的。 可能还有其他未在上面列出的原因使用它。

但是，如果您选择使用 asyncio 的原因，并且对于您的具体情况来说，这个原因感觉很薄弱或充满漏洞。 也许 asyncio 不是正确的解决方案。

我认为不使用 asyncio 的主要原因是它没有提供您认为的好处。

关于 Python 并发，尤其是 asyncio，存在许多误解。

例如:

• Asyncio 将解决全局解释器锁问题。
• 异步比线程更快。
• Asyncio 避免了对互斥锁和其他同步原语的需要。
• Asyncio 比线程更容易使用。

这些都是假的。

按照设计，一次只能运行一个协程，它们协作执行。 这就像 GIL 下的线程一样。 事实上，GIL 是一个正交问题，在大多数情况下使用 asyncio 时可能是不相关的。

任何可以使用 asyncio 编写的程序，都可以使用线程编写，并且速度可能会一样快，甚至更快。 它也可能会更简单、更容易被其他开发人员阅读和解释。

您可能会在线程中遇到任何并发故障模式，您可能会在协程中遇到这种情况。 您必须使协程免受死锁和竞争条件的影响，就像线程一样。

不使用 asyncio 的另一个原因是您不喜欢异步编程。

异步编程在许多不同的编程社区（尤其是 JavaScript 社区）中已经流行了一段时间。

它不同于过程式、面向对象和函数式编程，有些开发人员就是不喜欢它。

没问题。 如果您不喜欢它，请不要使用它。 这是一个合理的理由。

您可以通过多种方式实现相同的效果，特别是根据需要通过线程或进程执行器进行一些异步调用。

现在我们已经熟悉了何时使用 asyncio，让我们更详细地了解一下协程。

We have spent a lot of time on reasons why we should use asyncio.

It is probably a good idea to spend at least a moment on why we should not use it.

One reason to not use asyncio is that you cannot defend its use using one of the reasons above.

This is not foolproof. There may be other reasons to use it, not listed above.

But, if you pick a reason to use asyncio and the reason feels thin or full of holes for your specific case. Perhaps asyncio is not the right solution.

I think the major reason to not use asyncio is that it does not deliver the benefit that you think it does.

There are many misconceptions about Python concurrency, especially around asyncio.

For example:

• Asyncio will work around the global interpreter lock.
• Asyncio is faster than threads.
• Asyncio avoids the need for mutex locks and other synchronization primitives.
• Asyncio is easier to use than threads.

These are all false.

Only a single coroutine can run at a time by design, they cooperate to execute. This is just like threads under the GIL. In fact, the GIL is an orthogonal concern and probably irrelevant in most cases when using asyncio.

Any program you can write with asyncio, you can write with threads and it will probably be as fast or faster. It will also probably be simpler and easier to read and interpret by fellow developers.

Any concurrency failure mode you might expect with threads, you can encounter with coroutines. You must make coroutines safe from deadlocks and race conditions, just like threads.

Another reason to not use asyncio is that you don’t like asynchronous programming.

Asynchronous programming has been popular for some time now in a number of different programming communities, most notably the JavaScript community.

It is different from procedural, object-oriented, and functional programming, and some developers just don’t like it.

No problem. If you don’t like it, don’t use it. It’s a fair reason.

You can achieve the same effect in many ways, notably by sprinkling a few asynchronous calls in via thread or process executors as needed.

Now that we are familiar with when to use asyncio, let’s look at coroutines in more detail.

---

最后更新: 2024年9月4日
创建日期: 2024年9月4日