mirror of
https://gitee.com/fantix/kloop.git
synced 2024-11-22 02:11:01 +00:00
GitHub doesn't support multilingual
This commit is contained in:
Fantix King
parent
9cf03082c8
commit
85799a1624
3 changed files with 83 additions and 83 deletions
52
README.en.md
52
README.en.md
|
|
@ -1,52 +0,0 @@
|
||||||
# kLoop
|
|
||||||
|
|
||||||
kLoop is an implementation of the Python
|
|
||||||
[asyncio](https://docs.python.org/3/library/asyncio.html) event loop written
|
|
||||||
in [Cython](https://cython.org/), using
|
|
||||||
[io_uring](https://unixism.net/loti/what_is_io_uring.html) and
|
|
||||||
[kTLS](https://www.kernel.org/doc/html/latest/networking/tls-offload.html)
|
|
||||||
features of the Linux kernel, therefore called k(ernel)Loop.
|
|
||||||
|
|
||||||
kLoop is open-sourced and released under the
|
|
||||||
[MulanPSL - 2.0 license](http://license.coscl.org.cn/MulanPSL2).
|
|
||||||
|
|
||||||
**⚠️WARNING: THIS PROJECT IS IN PROOF-OF-CONCEPT STAGE!⚠️**
|
|
||||||
|
|
||||||
|
|
||||||
## Features
|
|
||||||
|
|
||||||
* **Minimum syscalls** - all I/O calls are done in the kernel thanks to
|
|
||||||
io_uring, and the only remaining syscall to `io_uring_enter()` is also
|
|
||||||
optimized to be only called when necessary using the `SQPOLL` feature. That
|
|
||||||
means most of the overhead of syscalls is gone;
|
|
||||||
* **No GIL in the main-loop** - the hot-path is written in Cython without GIL,
|
|
||||||
that means it's compiled into pure C code without Python structures, saving
|
|
||||||
some memory and execution time. On the other hand, GIL is only taken before
|
|
||||||
Python callbacks, so it's slightly more friendly to multithreading, which is
|
|
||||||
still not recommended though.
|
|
||||||
* **TLS offloading** - all symmetric-key encryption and decryption work is
|
|
||||||
offloaded to the NIC if supported, or to the thread pool of io_uring
|
|
||||||
otherwise. This allows us to free up CPU for more I/O, or leverage all the
|
|
||||||
CPU cores even if the application is running single-threaded.
|
|
||||||
* **Asynchronous DNS resolving** - we blended in the Rust
|
|
||||||
[trust-dns](https://github.com/bluejekyll/trust-dns/) library with a custom
|
|
||||||
I/O runtime bridging to io_uring in C (including reading the
|
|
||||||
`/etc/resolv.conf` and `/etc/hosts` files), providing flexible APIs to manage
|
|
||||||
the concurrency, cache and configuration in Python.
|
|
||||||
|
|
||||||
|
|
||||||
## Requirements
|
|
||||||
|
|
||||||
* Python >= 3.10
|
|
||||||
* Linux >= 5.11 (enable kTLS with `modprobe tls`)
|
|
||||||
* OpenSSL >= 3.0 (kTLS receive offloading requires the latest development
|
|
||||||
version)
|
|
||||||
|
|
||||||
Development and testing is done on Ubuntu 22.04.
|
|
||||||
|
|
||||||
|
|
||||||
## Architecture Diagram
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Looks like the Lucky Charms factory, says @aaronbrighton ...
|
|
||||||
66
README.md
66
README.md
|
|
@ -1,48 +1,52 @@
|
||||||
# kLoop
|
# kLoop
|
||||||
|
|
||||||
[English](README.en.md)
|
kLoop is an implementation of the Python
|
||||||
|
[asyncio](https://docs.python.org/3/library/asyncio.html) event loop written
|
||||||
kLoop 是一个 Python
|
in [Cython](https://cython.org/), using
|
||||||
[asyncio](https://docs.python.org/3/library/asyncio.html)
|
[io_uring](https://unixism.net/loti/what_is_io_uring.html) and
|
||||||
事件循环的实现，主要用 [Cython](https://cython.org/) 编写，重点使用了 Linux 内核的
|
|
||||||
[io_uring](https://unixism.net/loti/what_is_io_uring.html) 和
|
|
||||||
[kTLS](https://www.kernel.org/doc/html/latest/networking/tls-offload.html)
|
[kTLS](https://www.kernel.org/doc/html/latest/networking/tls-offload.html)
|
||||||
功能，故称作 k(ernel)Loop。
|
features of the Linux kernel, therefore called k(ernel)Loop.
|
||||||
|
|
||||||
您可在[木兰宽松许可证, 第2版](http://license.coscl.org.cn/MulanPSL2)允许的范围内使用
|
kLoop is open-sourced and released under the
|
||||||
kLoop 的源代码或发行版。
|
[MulanPSL - 2.0 license](http://license.coscl.org.cn/MulanPSL2).
|
||||||
|
|
||||||
**⚠️警告：项目仍在概念验证当中，满地都是坑！⚠️**
|
**⚠️WARNING: THIS PROJECT IS IN PROOF-OF-CONCEPT STAGE!⚠️**
|
||||||
|
|
||||||
|
|
||||||
## 主要特性
|
## Features
|
||||||
|
|
||||||
* **系统调用次数非常少**：得益于 io_uring，所有 I/O 调用都由内核完成；因为启用了
|
* **Minimum syscalls** - all I/O calls are done in the kernel thanks to
|
||||||
`SQPOLL`，唯一的系统调用 `io_uring_enter()`
|
io_uring, and the only remaining syscall to `io_uring_enter()` is also
|
||||||
也仅在必要时才会用到，因此节省下了几乎所有系统调用的额外开销；
|
optimized to be only called when necessary using the `SQPOLL` feature. That
|
||||||
* **主循环主体不占有 GIL**：虽然使用了 Cython 编写，但大部分最核心的代码都被编译为不需要
|
means most of the overhead of syscalls is gone;
|
||||||
Python 结构的普通 C 代码，一方面节省内存开销和处理时间，另一方面不持有 GIL，仅在有
|
* **No GIL in the main-loop** - the hot-path is written in Cython without GIL,
|
||||||
Python 回调时才上 GIL 锁，对多线程稍微友好一点，但还是推荐单线程跑；
|
that means it's compiled into pure C code without Python structures, saving
|
||||||
* **内核网卡代工 TLS**：如果硬件支持，TLS 通讯的全部对称加密解密皆由网卡完成，空出 CPU
|
some memory and execution time. On the other hand, GIL is only taken before
|
||||||
来做更多的 I/O；即使网卡不支持，io_uring
|
Python callbacks, so it's slightly more friendly to multithreading, which is
|
||||||
也可以在内核中开多线程来执行对称加解密，不受应用端单线程的限制，充分利用多核 CPU；
|
still not recommended though.
|
||||||
* **纯异步 DNS 解析**：混编进了 Rust 写的
|
* **TLS offloading** - all symmetric-key encryption and decryption work is
|
||||||
[trust-dns](https://github.com/bluejekyll/trust-dns/)，I/O 层直接走
|
offloaded to the NIC if supported, or to the thread pool of io_uring
|
||||||
io_uring（包括加载 `/etc/resolv.conf` 和 `/etc/hosts` 文件），在 C 和 Rust
|
otherwise. This allows us to free up CPU for more I/O, or leverage all the
|
||||||
之间互相调用完成 DNS 解析，并且提供了更加灵活的 Python 接口来控制并发、缓存和配置文件。
|
CPU cores even if the application is running single-threaded.
|
||||||
|
* **Asynchronous DNS resolving** - we blended in the Rust
|
||||||
|
[trust-dns](https://github.com/bluejekyll/trust-dns/) library with a custom
|
||||||
|
I/O runtime bridging to io_uring in C (including reading the
|
||||||
|
`/etc/resolv.conf` and `/etc/hosts` files), providing flexible APIs to manage
|
||||||
|
the concurrency, cache and configuration in Python.
|
||||||
|
|
||||||
|
|
||||||
## 环境需求
|
## Requirements
|
||||||
|
|
||||||
* Python >= 3.10
|
* Python >= 3.10
|
||||||
* Linux >= 5.11 (用 `modprobe tls` 命令来启用 kTLS 模块)
|
* Linux >= 5.11 (enable kTLS with `modprobe tls`)
|
||||||
* OpenSSL >= 3.0（支持 kTLS 收包代工需要最新的开发版本）
|
* OpenSSL >= 3.0 (kTLS receive offloading requires the latest development
|
||||||
|
version)
|
||||||
|
|
||||||
目前主要是在 Ubuntu 22.04 上开发测试的。
|
Development and testing is done on Ubuntu 22.04.
|
||||||
|
|
||||||
|
|
||||||
## 架构图
|
## Architecture Diagram
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@aaronbrighton 说像 Lucky Charms 卡通麦片工厂……
|
Looks like the Lucky Charms factory, says @aaronbrighton ...
|
||||||
|
|
|
||||||
48
README.zh.md
Normal file
48
README.zh.md
Normal file
|
|
@ -0,0 +1,48 @@
|
||||||
|
# kLoop
|
||||||
|
|
||||||
|
[English](README.en.md)
|
||||||
|
|
||||||
|
kLoop 是一个 Python
|
||||||
|
[asyncio](https://docs.python.org/3/library/asyncio.html)
|
||||||
|
事件循环的实现，主要用 [Cython](https://cython.org/) 编写，重点使用了 Linux 内核的
|
||||||
|
[io_uring](https://unixism.net/loti/what_is_io_uring.html) 和
|
||||||
|
[kTLS](https://www.kernel.org/doc/html/latest/networking/tls-offload.html)
|
||||||
|
功能，故称作 k(ernel)Loop。
|
||||||
|
|
||||||
|
您可在[木兰宽松许可证, 第2版](http://license.coscl.org.cn/MulanPSL2)允许的范围内使用
|
||||||
|
kLoop 的源代码或发行版。
|
||||||
|
|
||||||
|
**⚠️警告：项目仍在概念验证当中，满地都是坑！⚠️**
|
||||||
|
|
||||||
|
|
||||||
|
## 主要特性
|
||||||
|
|
||||||
|
* **系统调用次数非常少**：得益于 io_uring，所有 I/O 调用都由内核完成；因为启用了
|
||||||
|
`SQPOLL`，唯一的系统调用 `io_uring_enter()`
|
||||||
|
也仅在必要时才会用到，因此节省下了几乎所有系统调用的额外开销；
|
||||||
|
* **主循环主体不占有 GIL**：虽然使用了 Cython 编写，但大部分最核心的代码都被编译为不需要
|
||||||
|
Python 结构的普通 C 代码，一方面节省内存开销和处理时间，另一方面不持有 GIL，仅在有
|
||||||
|
Python 回调时才上 GIL 锁，对多线程稍微友好一点，但还是推荐单线程跑；
|
||||||
|
* **内核网卡代工 TLS**：如果硬件支持，TLS 通讯的全部对称加密解密皆由网卡完成，空出 CPU
|
||||||
|
来做更多的 I/O；即使网卡不支持，io_uring
|
||||||
|
也可以在内核中开多线程来执行对称加解密，不受应用端单线程的限制，充分利用多核 CPU；
|
||||||
|
* **纯异步 DNS 解析**：混编进了 Rust 写的
|
||||||
|
[trust-dns](https://github.com/bluejekyll/trust-dns/)，I/O 层直接走
|
||||||
|
io_uring（包括加载 `/etc/resolv.conf` 和 `/etc/hosts` 文件），在 C 和 Rust
|
||||||
|
之间互相调用完成 DNS 解析，并且提供了更加灵活的 Python 接口来控制并发、缓存和配置文件。
|
||||||
|
|
||||||
|
|
||||||
|
## 环境需求
|
||||||
|
|
||||||
|
* Python >= 3.10
|
||||||
|
* Linux >= 5.11 (用 `modprobe tls` 命令来启用 kTLS 模块)
|
||||||
|
* OpenSSL >= 3.0（支持 kTLS 收包代工需要最新的开发版本）
|
||||||
|
|
||||||
|
目前主要是在 Ubuntu 22.04 上开发测试的。
|
||||||
|
|
||||||
|
|
||||||
|
## 架构图
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
@aaronbrighton 说像 Lucky Charms 卡通麦片工厂……
|
||||||
Loading…
Reference in a new issue