数据库

Databases

Tortoise 目前支持以下数据库:

  • SQLite(使用 aiosqlite )

  • PostgreSQL >= 9.4(使用 asyncpgpsycopg )

  • MySQL/MariaDB(使用 asyncmyaiomysql )

  • Microsoft SQL Server(使用 asyncodbc )

使用时,请确保已安装相应的 asyncio 驱动程序。

Tortoise currently supports the following databases:

  • SQLite (using aiosqlite )

  • PostgreSQL >= 9.4 (using asyncpg or psycopg )

  • MySQL/MariaDB (using asyncmy or aiomysql )

  • Microsoft SQL Server (using asyncodbc )

To use, please ensure that corresponding asyncio driver is installed.

DB_URL

Tortoise 支持以 URL 形式指定数据库配置。

格式为:

DB_TYPE://USERNAME:PASSWORD@HOST:PORT/DB_NAME?PARAM1=value&PARAM2=value

如果密码包含特殊字符,则需要进行 URL 编码:

>>> import urllib.parse
>>> urllib.parse.quote_plus("kx%jj5/g")
'kx%25jj5%2Fg'

支持的 DB_TYPE:

sqlite:

通常格式为 sqlite://DB_FILE 如果 DB_FILE 是 “/data/db.sqlite3”,则字符串为 sqlite:///data/db.sqlite3 (注意三个 /)

postgres:

使用 asyncpg: 通常格式为 postgres://postgres:pass@db.host:5432/somedb

或使用具体的 asyncpg/psycopg:

  • psycopg: psycopg://postgres:pass@db.host:5432/somedb

  • asyncpg: asyncpg://postgres:pass@db.host:5432/somedb

mysql:

通常格式为 mysql://myuser:mypass@db.host:3306/somedb

mssql:

通常格式为 mssql://myuser:mypass@db.host:1433/somedb?driver=the odbc driver

Tortoise supports specifying Database configuration in a URL form.

The form is:

DB_TYPE://USERNAME:PASSWORD@HOST:PORT/DB_NAME?PARAM1=value&PARAM2=value

If password contains special characters it need to be URL encoded:

>>> import urllib.parse
>>> urllib.parse.quote_plus("kx%jj5/g")
'kx%25jj5%2Fg'

The supported DB_TYPE:

sqlite:

Typically in the form of sqlite://DB_FILE So if the DB_FILE is “/data/db.sqlite3” then the string will be sqlite:///data/db.sqlite (note the three /’s)

postgres

Using asyncpg: Typically in the form of postgres://postgres:pass@db.host:5432/somedb

Or specifically asyncpg/psycopg using:

  • psycopg: psycopg://postgres:pass@db.host:5432/somedb

  • asyncpg: asyncpg://postgres:pass@db.host:5432/somedb

mysql:

Typically in the form of mysql://myuser:mypass@db.host:3306/somedb

mssql:

Typically in the form of mssql://myuser:mypass@db.host:1433/somedb?driver=the odbc driver

功能

Capabilities

由于每个数据库具有不同的功能集,我们在每个客户端上注册了 Capabilities。 主要目的是解决 SQL 之间的重大差异或常见问题。

Since each database has a different set of features we have a Capabilities that is registered on each client. Primarily this is to work around larger-than SQL differences, or common issues.

class tortoise.backends.base.client.Capabilities(dialect, *, daemon=True, requires_limit=False, inline_comment=False, supports_transactions=True, support_for_update=True, support_index_hint=False, support_update_limit_order_by=True)[source]

DB Client Capabilities indicates the supported feature-set, and is also used to note common workarounds to deficiencies.

Defaults are set with the following standard:

  • Deficiencies: assume it is working right.

  • Features: assume it doesn’t have it.

Parameters:
dialect

Dialect name of the DB Client driver.

daemon=True

Is the DB an external Daemon we connect to?

requires_limit=False

Indicates that this DB requires a LIMIT statement for an OFFSET statement to work.

inline_comment=False

Indicates that comments should be rendered in line with the DDL statement, and not as a separate statement.

supports_transactions=True

Indicates that this DB supports transactions.

support_for_update=True

Indicates that this DB supports SELECT … FOR UPDATE SQL statement.

support_index_hint=False

Support force index or use index.

support_update_limit_order_by=True

support update/delete with limit and order by.

SQLite

SQLite 是一个嵌入式数据库,可以运行在文件或内存中。适合用于本地开发或代码逻辑的测试,但不推荐用于生产环境。

Caution

SQLite 并不原生支持许多常见的数据类型,虽然我们在可能的情况下进行了仿真,但并不是所有的都完美。

例如,DecimalField 通过将值存储为字符串来保持精度,但在进行聚合/排序时,必须在浮点数之间进行转换。

同样,大小写不敏感的实现也仅部分完成。

数据库 URL 通常格式为 sqlite://DB_FILE,因此如果 DB_FILE 是 “/data/db.sqlite3”,则字符串为 sqlite:///data/db.sqlite3 (注意三个 /)。

SQLite is an embedded database, and can run on a file or in-memory. Good database for local development or testing of code logic, but not recommended for production use.

Caution

SQLite doesn’t support many of the common datatypes natively, although we do emulation where we can, not everything is perfect.

For example DecimalField has precision preserved by storing values as strings, except when doing aggregates/ordering on it. In those cases we have to cast to/from floating-point numbers.

Similarly case-insensitivity is only partially implemented.

DB URL is typically in the form of sqlite://DB_FILE So if the DB_FILE is “/data/db.sqlite3” then the string will be sqlite:///data/db.sqlite (note the three /’s)

必选参数

Required Parameters

path:

指向 SQLite3 文件的路径。:memory: 是一个特殊路径,表示使用内存数据库。

path:

Path to SQLite3 file. :memory: is a special path that indicates in-memory database.

可选参数

Optional parameters:

SQLite 可选参数基本上是任何文档中记录的 PRAGMA 语句 here.

journal_mode (默认为 WAL ):

指定 SQLite 日志模式。

journal_size_limit (默认为 16384 ):

日志大小。

foreign_keys (默认为 ON ):

设置为 OFF 以不强制执行引用完整性。

SQLite optional parameters is basically any of the PRAGMA statements documented here.

journal_mode (defaults to WAL ):

Specify SQLite journal mode.

journal_size_limit (defaults to 16384 ):

The journal size.

foreign_keys (defaults to ON )

Set to OFF to not enforce referential integrity.

PostgreSQL

数据库 URL 通常格式为 postgres://postgres:pass@db.host:5432/somedb ,或者,如果通过 Unix 域套接字连接,则为 postgres:///somedb

DB URL is typically in the form of postgres://postgres:pass@db.host:5432/somedb, or, if connecting via Unix domain socket postgres:///somedb.

必选参数

Required Parameters

user:

连接时使用的用户名。

password:

用户名的密码。

host:

数据库可用的网络主机。

port:

数据库可用的网络端口。(默认为 5432 )

database:

要使用的数据库。

user:

Username to connect with.

password:

Password for username.

host:

Network host that database is available at.

port:

Network port that database is available at. (defaults to 5432 )

database:

Database to use.

可选参数

Optional parameters:

PostgreSQL 可选参数是直接传递给驱动程序的参数,更多细节请参见 这里

minsize (默认为 1 ):

连接池的最小大小

maxsize (默认为 5 ):

连接池的最大大小

max_queries (默认为 50000 ):

在关闭并替换连接之前的最大查询次数。

max_inactive_connection_lifetime (默认为 300.0 ):

在假定连接已失效并强制重新连接之前的非活动连接持续时间。

schema (默认使用用户的默认模式):

默认使用的特定模式。

ssl (默认为 False ):

可以是 True 或用于自签名证书的自定义 SSL 上下文。有关更多信息,请参见 MSSQL/Oracle

如果缺少 userpasswordhostport 参数,我们将让 asyncpg/psycopg 从默认来源(标准 PostgreSQL 环境变量或默认值)中检索。

PostgreSQL optional parameters are pass-though parameters to the driver, see here for more details.

minsize (defaults to 1 ):

Minimum connection pool size

maxsize (defaults to 5 ):

Maximum connection pool size

max_queries (defaults to 50000 ):

Maximum no of queries before a connection is closed and replaced.

max_inactive_connection_lifetime (defaults to 300.0 ):

Duration of inactive connection before assuming that it has gone stale, and force a re-connect.

schema (uses user’s default schema by default):

A specific schema to use by default.

ssl (defaults to ‘’False`` ):

Either True or a custom SSL context for self-signed certificates. See MSSQL/Oracle for more info.

In case any of user, password, host, port parameters is missing, we are letting asyncpg/psycopg retrieve it from default sources (standard PostgreSQL environment variables or default values).

MySQL/MariaDB

DB URL 通常采用以下形式: mysql://myuser:mypass@db.host:3306/somedb

DB URL is typically in the form of mysql://myuser:mypass@db.host:3306/somedb

必选参数

Required Parameters

user:

连接所用的用户名。

password:

用户名的密码。

host:

数据库可用的网络主机。

port:

数据库可用的网络端口。(默认为 3306 )

database:

要使用的数据库。

user:

Username to connect with.

password:

Password for username.

host:

Network host that database is available at.

port:

Network port that database is available at. (defaults to 3306 )

database:

Database to use.

可选参数

Optional parameters:

MySQL 可选参数是传递给驱动程序的参数,更多细节请参见 here

minsize (默认为 1 ):

最小连接池大小

maxsize (默认为 5 ):

最大连接池大小

connect_timeout (默认为 None ):

在抛出错误之前等待连接的持续时间。

echo (默认为 False ):

设置为 True 以回显 SQL 查询(仅用于调试)

charset (默认为 utf8mb4 ):

设置使用的字符集

ssl (默认为 False ):

可以是 True 或自定义 SSL 上下文,用于自签名证书。有关更多信息,请参见 MSSQL/Oracle

MySQL optional parameters are pass-though parameters to the driver, see here for more details.

minsize (defaults to 1 ):

Minimum connection pool size

maxsize (defaults to 5 ):

Maximum connection pool size

connect_timeout (defaults to None ):

Duration to wait for connection before throwing error.

echo (defaults to False ):

Set to True` to echo SQL queries (debug only)

charset (defaults to utf8mb4 ):

Sets the character set in use

ssl (defaults to False ):

Either True or a custom SSL context for self-signed certificates. See MSSQL/Oracle for more info.

MSSQL/Oracle

数据库 URL 通常的形式为 mssql oracle://myuser:mypass@db.host:1433/somedb?driver=the odbc driver

DB URL is typically in the form of mssql or oracle://myuser:mypass@db.host:1433/somedb?driver=the odbc driver

必须参数

Required Parameters

user:

用于连接的用户名。

password:

用户名的密码。

host:

数据库可用的网络主机。

port:

数据库可用的网络端口。(默认为 1433 )

database:

要使用的数据库。

driver:

要使用的 ODBC 驱动程序。在您的 odbcinst.ini 文件中 ODBC 驱动程序的实际名称(您可以使用 odbcinst -j 命令找到其位置)。需要在系统中安装 unixodbc

user:

Username to connect with.

password:

Password for username.

host:

Network host that database is available at.

port:

Network port that database is available at. (defaults to 1433 )

database:

Database to use.

driver:

The ODBC driver to use. Actual name of the ODBC driver in your odbcinst.ini file (you can find it’s location using odbcinst -j command). It requires unixodbc to be installed in your system.

可选参数

Optional parameters:

MSSQL/Oracle 可选参数是传递给驱动程序的参数,详情见 here

minsize (默认为 1 ):

最小连接池大小

maxsize (默认为 10 ):

最大连接池大小

pool_recycle (默认为 -1 ):

连接池回收超时时间(秒)。

echo (默认为 False ):

设置为 True 以回显 SQL 查询(仅用于调试)。

MSSQL/Oracle optional parameters are pass-though parameters to the driver, see here for more details.

minsize (defaults to 1 ):

Minimum connection pool size

maxsize (defaults to 10 ):

Maximum connection pool size

pool_recycle (defaults to -1 ):

Pool recycle timeout in seconds.

echo (defaults to False ):

Set to True to echo SQL queries (debug only)

Oracle中的编码

Encoding in Oracle:

如果在 Varchar 字段中看到 ??? 值而不是实际文本(如俄文/中文等),请在客户端环境中设置 NLS_LANG 变量以支持 UTF8。例如,设置为 “American_America.UTF8”

If you get ??? values in Varchar fields instead of your actual text (russian/chinese/etc), then set NLS_LANG variable in your client environment to support UTF8. For example, “American_America.UTF8”.

传递自定义SSL证书

Passing in custom SSL Certificates

要传递自定义 SSL 证书,需要使用详细的初始化结构,因为 URL 解析器无法处理复杂对象。

# 这里我们创建一个自定义的 SSL 上下文
import ssl
ctx = ssl.create_default_context()
# 在这个示例中,我们禁用验证...
# 请不要这样做。请查看官方 Python ``ssl`` 模块文档
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

# 这里我们进行详细初始化
await Tortoise.init(
    config={
        "connections": {
            "default": {
                "engine": "tortoise.backends.asyncpg",
                "credentials": {
                    "database": None,
                    "host": "127.0.0.1",
                    "password": "moo",
                    "port": 54321,
                    "user": "postgres",
                    "ssl": ctx  # 在这里传递 SSL 上下文
                }
            }
        },
        "apps": {
            "models": {
                "models": ["some.models"],
                "default_connection": "default",
            }
        },
    }
)

To pass in a custom SSL Cert, one has to use the verbose init structure as the URL parser can’t handle complex objects.

# Here we create a custom SSL context
import ssl
ctx = ssl.create_default_context()
# And in this example we disable validation...
# Please don't do this. Look at the official Python ``ssl`` module documentation
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

# Here we do a verbose init
await Tortoise.init(
    config={
        "connections": {
            "default": {
                "engine": "tortoise.backends.asyncpg",
                "credentials": {
                    "database": None,
                    "host": "127.0.0.1",
                    "password": "moo",
                    "port": 54321,
                    "user": "postgres",
                    "ssl": ctx  # Here we pass in the SSL context
                }
            }
        },
        "apps": {
            "models": {
                "models": ["some.models"],
                "default_connection": "default",
            }
        },
    }
)

基本DB客户端

Base DB client

基础数据库客户端接口在此提供,但应仅在高级情况下直接使用。

The Base DB client interface is provided here, but should only be directly used as an advanced case.

class tortoise.backends.base.client.BaseDBAsyncClient(connection_name, fetch_inserted=True, **kwargs)[source]

Base class for containing a DB connection.

Parameters get passed as kwargs, and is mostly driver specific.

query_class Type[pypika.Query]

The PyPika Query dialect (low level dialect)

executor_class Type[BaseExecutor]

The executor dialect class (high level dialect)

schema_generator Type[BaseSchemaGenerator]

The DDL schema generator

capabilities Capabilities

Contains the connection capabilities

acquire_connection()[source]

Acquires a connection from the pool. Will return the current context connection if already in a transaction.

Return type:

Union[ConnectionWrapper[~T_conn], PoolConnectionWrapper[~T_conn]]

async close()[source]

Closes the DB connection.

Return type:

None

async create_connection(with_db)[source]

Establish a DB connection.

Parameters:
with_db

If True, then select the DB to use, else use default. Use case for this is to create/drop a database.

Return type:

None

async db_create()[source]

Created the database in the server. Typically only called by the test runner.

Need to have called create_connection()` with parameter with_db=False set to use the default connection instead of the configured one, else you would get errors indicating the database doesn’t exist.

Return type:

None

async db_delete()[source]

Delete the database from the Server. Typically only called by the test runner.

Need to have called create_connection()` with parameter with_db=False set to use the default connection instead of the configured one, else you would get errors indicating the database is in use.

Return type:

None

async execute_insert(query, values)[source]

Executes a RAW SQL insert statement, with provided parameters.

Parameters:
query

The SQL string, pre-parametrized for the target DB dialect.

values

A sequence of positional DB parameters.

Return type:

Any

Returns:

The primary key if it is generated by the DB. (Currently only integer autonumber PK’s)

async execute_many(query, values)[source]

Executes a RAW bulk insert statement, like execute_insert, but returns no data.

Parameters:
query

The SQL string, pre-parametrized for the target DB dialect.

values

A sequence of positional DB parameters.

Return type:

None

async execute_query(query, values=None)[source]

Executes a RAW SQL query statement, and returns the resultset.

Parameters:
query

The SQL string, pre-parametrized for the target DB dialect.

values=None

A sequence of positional DB parameters.

Return type:

Tuple[int, Sequence[dict]]

Returns:

A tuple of: (The number of rows affected, The resultset)

async execute_query_dict(query, values=None)[source]

Executes a RAW SQL query statement, and returns the resultset as a list of dicts.

Parameters:
query

The SQL string, pre-parametrized for the target DB dialect.

values=None

A sequence of positional DB parameters.

Return type:

List[dict]

async execute_script(query)[source]

Executes a RAW SQL script with multiple statements, and returns nothing.

Parameters:
query

The SQL string, which will be passed on verbatim. Semicolons is supported here.

Return type:

None