Add api keys db path as env var (#1)

* Add API Keys DB path to command & env vars

* Create API Keys DB directory if not exists before sqlite connection & Add docker-compose example
This commit is contained in:
Minosity-VR 2022-07-15 13:22:04 +02:00 committed by GitHub
parent 393eebe235
commit 752d2aae2c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 48 additions and 5 deletions

View file

@ -164,6 +164,8 @@ docker-compose up -d --build
> Feel free to change the [`docker-compose.yml`](https://github.com/LibreTranslate/LibreTranslate/blob/main/docker-compose.yml) file to adapt it to your deployment needs, or use an extra `docker-compose.prod.yml` file for your deployment configuration. > Feel free to change the [`docker-compose.yml`](https://github.com/LibreTranslate/LibreTranslate/blob/main/docker-compose.yml) file to adapt it to your deployment needs, or use an extra `docker-compose.prod.yml` file for your deployment configuration.
> The models are stored inside the container under `/root/.local/share` and `/root/.local/cache`. Feel free to use volumes if you do not want to redownload the models when the container is destroyed. Be aware that this will prevent the models from being updated!
### CUDA ### CUDA
You can use hardware acceleration to speed up translations on a GPU machine with CUDA 11.2 and [nvidia-docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) installed. You can use hardware acceleration to speed up translations on a GPU machine with CUDA 11.2 and [nvidia-docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) installed.
@ -190,6 +192,7 @@ docker-compose -f docker-compose.cuda.yml up -d --build
| --frontend-language-target | Set frontend default language - target | `es` | LT_FRONTEND_LANGUAGE_TARGET | | --frontend-language-target | Set frontend default language - target | `es` | LT_FRONTEND_LANGUAGE_TARGET |
| --frontend-timeout | Set frontend translation timeout | `500` | LT_FRONTEND_TIMEOUT | | --frontend-timeout | Set frontend translation timeout | `500` | LT_FRONTEND_TIMEOUT |
| --api-keys | Enable API keys database for per-user rate limits lookup | `Don't use API keys` | LT_API_KEYS | | --api-keys | Enable API keys database for per-user rate limits lookup | `Don't use API keys` | LT_API_KEYS |
| --api-keys-db-path | Use a specific path inside the container for the local database. Can be absolute or relative | `api_keys.db` | LT_API_KEYS_DB_PATH |
| --api-keys-remote | Use this remote endpoint to query for valid API keys instead of using the local database | `Use local API key database` | LT_API_KEYS_REMOTE | | --api-keys-remote | Use this remote endpoint to query for valid API keys instead of using the local database | `Use local API key database` | LT_API_KEYS_REMOTE |
| --get-api-key-link | Show a link in the UI where to direct users to get an API key | `Don't show a link` | LT_GET_API_KEY_LINK | | --get-api-key-link | Show a link in the UI where to direct users to get an API key | `Don't show a link` | LT_GET_API_KEY_LINK |
| --require-api-key-origin | Require use of an API key for programmatic access to the API, unless the request origin matches this domain | `No restrictions on domain origin` | LT_REQUIRE_API_KEY_ORIGIN | | --require-api-key-origin | Require use of an API key for programmatic access to the API, unless the request origin matches this domain | `No restrictions on domain origin` | LT_REQUIRE_API_KEY_ORIGIN |
@ -222,7 +225,7 @@ See ["LibreTranslate: your own translation service on Kubernetes" by JM Robles](
LibreTranslate supports per-user limit quotas, e.g. you can issue API keys to users so that they can enjoy higher requests limits per minute (if you also set `--req-limit`). By default all users are rate-limited based on `--req-limit`, but passing an optional `api_key` parameter to the REST endpoints allows a user to enjoy higher request limits. LibreTranslate supports per-user limit quotas, e.g. you can issue API keys to users so that they can enjoy higher requests limits per minute (if you also set `--req-limit`). By default all users are rate-limited based on `--req-limit`, but passing an optional `api_key` parameter to the REST endpoints allows a user to enjoy higher request limits.
To use API keys simply start LibreTranslate with the `--api-keys` option. To use API keys simply start LibreTranslate with the `--api-keys` option. If you modified the API keys database path with the option `--api-keys-db-path`, you must specify the path with the same argument flag when using the `ltmanage keys` command.
### Add New Keys ### Add New Keys
@ -232,6 +235,11 @@ To issue a new API key with 120 requests per minute limits:
ltmanage keys add 120 ltmanage keys add 120
``` ```
If you changed the API keys database path:
```bash
ltmanage keys --api-keys-db-path path/to/db/dbName.db add 120
```
### Remove Keys ### Remove Keys
```bash ```bash

View file

@ -1,13 +1,18 @@
import os
import sqlite3 import sqlite3
import uuid import uuid
import requests import requests
from expiringdict import ExpiringDict from expiringdict import ExpiringDict
from app.default_values import DEFAULT_ARGUMENTS as DEFARGS
DEFAULT_DB_PATH = "api_keys.db" DEFAULT_DB_PATH = DEFARGS['API_KEYS_DB_PATH']
class Database: class Database:
def __init__(self, db_path=DEFAULT_DB_PATH, max_cache_len=1000, max_cache_age=30): def __init__(self, db_path=DEFAULT_DB_PATH, max_cache_len=1000, max_cache_age=30):
db_dir = os.path.dirname(db_path)
if not db_dir == "" and not os.path.exists(db_dir):
os.makedirs(db_dir)
self.db_path = db_path self.db_path = db_path
self.cache = ExpiringDict(max_len=max_cache_len, max_age_seconds=max_cache_age) self.cache = ExpiringDict(max_len=max_cache_len, max_age_seconds=max_cache_age)

View file

@ -151,7 +151,7 @@ def create_app(args):
if args.api_keys_remote: if args.api_keys_remote:
api_keys_db = RemoteDatabase(args.api_keys_remote) api_keys_db = RemoteDatabase(args.api_keys_remote)
else: else:
api_keys_db = Database() api_keys_db = Database(args.api_keys_db_path)
from flask_limiter import Limiter from flask_limiter import Limiter

View file

@ -106,6 +106,11 @@ _default_options_objects = [
'default_value': False, 'default_value': False,
'value_type': 'bool' 'value_type': 'bool'
}, },
{
'name': 'API_KEYS_DB_PATH',
'default_value': 'api_keys.db',
'value_type': 'str'
},
{ {
'name': 'API_KEYS_REMOTE', 'name': 'API_KEYS_REMOTE',
'default_value': '', 'default_value': '',

View file

@ -89,6 +89,12 @@ def get_args():
action="store_true", action="store_true",
help="Enable API keys database for per-user rate limits lookup", help="Enable API keys database for per-user rate limits lookup",
) )
parser.add_argument(
"--api-keys-db-path",
default=DEFARGS['API_KEYS_DB_PATH'],
type=str,
help="Use a specific path inside the container for the local database. Can be absolute or relative (%(default)s)",
)
parser.add_argument( parser.add_argument(
"--api-keys-remote", "--api-keys-remote",
default=DEFARGS['API_KEYS_REMOTE'], default=DEFARGS['API_KEYS_REMOTE'],

View file

@ -1,6 +1,8 @@
import argparse import argparse
import os
from app.api_keys import Database from app.api_keys import Database
from app.default_values import DEFAULT_ARGUMENTS as DEFARGS
def manage(): def manage():
@ -10,6 +12,12 @@ def manage():
) )
keys_parser = subparsers.add_parser("keys", help="Manage API keys database") keys_parser = subparsers.add_parser("keys", help="Manage API keys database")
keys_parser.add_argument(
"--api-keys-db-path",
default=DEFARGS['API_KEYS_DB_PATH'],
type=str,
help="Use a specific path inside the container for the local database",
)
keys_subparser = keys_parser.add_subparsers( keys_subparser = keys_parser.add_subparsers(
help="", dest="sub_command", title="Command List" help="", dest="sub_command", title="Command List"
) )
@ -30,7 +38,10 @@ def manage():
args = parser.parse_args() args = parser.parse_args()
if args.command == "keys": if args.command == "keys":
db = Database() if not os.path.exists(args.api_keys_db_path):
print("No such database: %s" % args.api_keys_db_path)
exit(1)
db = Database(args.api_keys_db_path)
if args.sub_command is None: if args.sub_command is None:
# Print keys # Print keys
keys = db.all() keys = db.all()

View file

@ -8,4 +8,12 @@ services:
ports: ports:
- 5000:5000 - 5000:5000
## Uncomment above command and define your args if necessary ## Uncomment above command and define your args if necessary
# command: --ssl --ga-id MY-GA-ID --req-limit 100 --char-limit 500 # command: --ssl --ga-id MY-GA-ID --req-limit 100 --char-limit 500
## Uncomment this section and the `volumes` section if you want to backup your API keys
# environment:
# - LT_API_KEYS_DB_PATH=/app/db/api_keys.db # Same result as `db/api_keys.db` or `./db/api_keys.db`
# volumes:
# - libretranslate_api_keys:/app/db/api_keys.db
# volumes:
# libretranslate_api_keys: