mirror of
https://gitee.com/fantix/kloop.git
synced 2024-11-25 03:41:00 +00:00
Update README and bump to 0.2
This commit is contained in:
parent
64b105657a
commit
3b4b6c5c67
5 changed files with 69 additions and 11 deletions
|
@ -1,5 +1,5 @@
|
|||
include Makefile
|
||||
include architecture.png
|
||||
recursive-include src *.pyx *.pxd *.h
|
||||
graft tests
|
||||
global-exclude *.py[cod] *.c
|
||||
|
||||
|
|
36
README.en.md
36
README.en.md
|
@ -13,8 +13,40 @@ kLoop is open-sourced and released under the
|
|||
**⚠️WARNING: THIS PROJECT IS IN PROOF-OF-CONCEPT STAGE!⚠️**
|
||||
|
||||
|
||||
## Features
|
||||
|
||||
* **Minimum syscalls** - all I/O calls are done in the kernel thanks to
|
||||
io_uring, and the only remaining syscall to `io_uring_enter()` is also
|
||||
optimized to be only called when necessary using the `SQPOLL` feature. That
|
||||
means most of the overhead of syscalls is gone;
|
||||
* **No GIL in the main-loop** - the hot-path is written in Cython without GIL,
|
||||
that means it's compiled into pure C code without Python structures, saving
|
||||
some memory and execution time. On the other hand, GIL is only taken before
|
||||
Python callbacks, so it's slightly more friendly to multithreading, which is
|
||||
still not recommended though.
|
||||
* **TLS offloading** - all symmetric-key encryption and decryption work is
|
||||
offloaded to the NIC if supported, or to the thread pool of io_uring
|
||||
otherwise. This allows us to free up CPU for more I/O, or leverage all the
|
||||
CPU cores even if the application is running single-threaded.
|
||||
* **Asynchronous DNS resolving** - we blended in the Rust
|
||||
[trust-dns](https://github.com/bluejekyll/trust-dns/) library with a custom
|
||||
I/O runtime bridging to io_uring in C (including reading the
|
||||
`/etc/resolv.conf` and `/etc/hosts` files), providing flexible APIs to manage
|
||||
the concurrency, cache and configuration in Python.
|
||||
|
||||
|
||||
## Requirements
|
||||
|
||||
* Python >= 3.8
|
||||
* Python >= 3.10
|
||||
* Linux >= 5.11 (enable kTLS with `modprobe ktls`)
|
||||
* OpenSSL >= 3.0
|
||||
* OpenSSL >= 3.0 (kTLS receive offloading requires the latest development
|
||||
version)
|
||||
|
||||
Development and testing is done on Ubuntu 22.04.
|
||||
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
![architecture.png](architecture.png)
|
||||
|
||||
Looks like the Lucky Charms factory, says @aaronbrighton ...
|
||||
|
|
38
README.md
38
README.md
|
@ -3,20 +3,46 @@
|
|||
[English](README.en.md)
|
||||
|
||||
kLoop 是一个 Python
|
||||
[asyncio](https://docs.python.org/3/library/asyncio.html) event loop 的实现,
|
||||
主要用 [Cython](https://cython.org/) 编写,重点使用了 Linux 内核的
|
||||
[asyncio](https://docs.python.org/3/library/asyncio.html)
|
||||
事件循环的实现,主要用 [Cython](https://cython.org/) 编写,重点使用了 Linux 内核的
|
||||
[io_uring](https://unixism.net/loti/what_is_io_uring.html) 和
|
||||
[kTLS](https://www.kernel.org/doc/html/latest/networking/tls-offload.html)
|
||||
功能,故称作 k(ernel)Loop。
|
||||
|
||||
您可在[木兰宽松许可证, 第2版](http://license.coscl.org.cn/MulanPSL2)的范围内使用
|
||||
kLoop 的源代码。
|
||||
您可在[木兰宽松许可证, 第2版](http://license.coscl.org.cn/MulanPSL2)允许的范围内使用
|
||||
kLoop 的源代码或发行版。
|
||||
|
||||
**⚠️警告:项目仍在概念验证当中,满地都是坑!⚠️**
|
||||
|
||||
|
||||
## 主要特性
|
||||
|
||||
* **系统调用次数非常少**:得益于 io_uring,所有 I/O 调用都由内核完成;因为启用了
|
||||
`SQPOLL`,唯一的系统调用 `io_uring_enter()`
|
||||
也仅在必要时才会用到,因此节省下了几乎所有系统调用的额外开销;
|
||||
* **主循环主体不占有 GIL**:虽然使用了 Cython 编写,但大部分最核心的代码都被编译为不需要
|
||||
Python 结构的普通 C 代码,一方面节省内存开销和处理时间,另一方面不持有 GIL,仅在有
|
||||
Python 回调时才上 GIL 锁,对多线程稍微友好一点,但还是推荐单线程跑;
|
||||
* **内核网卡代工 TLS**:如果硬件支持,TLS 通讯的全部对称加密解密皆由网卡完成,空出 CPU
|
||||
来做更多的 I/O;即使网卡不支持,io_uring
|
||||
也可以在内核中开多线程来执行对称加解密,不受应用端单线程的限制,充分利用多核 CPU;
|
||||
* **纯异步 DNS 解析**:混编进了 Rust 写的
|
||||
[trust-dns](https://github.com/bluejekyll/trust-dns/),I/O 层直接走
|
||||
io_uring(包括加载 `/etc/resolv.conf` 和 `/etc/hosts` 文件),在 C 和 Rust
|
||||
之间互相调用完成 DNS 解析,并且提供了更加灵活的 Python 接口来控制并发、缓存和配置文件。
|
||||
|
||||
|
||||
## 环境需求
|
||||
|
||||
* Python >= 3.8
|
||||
* Python >= 3.10
|
||||
* Linux >= 5.11 (用 `modprobe ktls` 命令来启用 kTLS 模块)
|
||||
* OpenSSL >= 3.0
|
||||
* OpenSSL >= 3.0(支持 kTLS 收包代工需要最新的开发版本)
|
||||
|
||||
目前主要是在 Ubuntu 22.04 上开发测试的。
|
||||
|
||||
|
||||
## 架构图
|
||||
|
||||
![架构图.png](architecture.png)
|
||||
|
||||
@aaronbrighton 说像 Lucky Charms 卡通麦片工厂……
|
||||
|
|
0
architecture.png
Normal file
0
architecture.png
Normal file
|
@ -1,6 +1,6 @@
|
|||
[metadata]
|
||||
name = kLoop
|
||||
version = 0.0.1
|
||||
version = 0.2.0
|
||||
author = Fantix King
|
||||
author_email = fantix.king@gmail.com
|
||||
description = An asyncio event loop using Linux io_uring and kTLS.
|
||||
|
@ -18,7 +18,7 @@ classifiers =
|
|||
package_dir =
|
||||
= src
|
||||
packages = find:
|
||||
python_requires = >=3.8
|
||||
python_requires = >=3.10
|
||||
|
||||
[options.packages.find]
|
||||
where = src
|
||||
|
|
Loading…
Reference in a new issue