Commit graph

41 commits

Author SHA1 Message Date
Rafael Caricio fd92b25190
Make queues configurable 2023-03-11 16:38:32 +01:00
Rafael Caricio aac0b44c7f
Make possible to provide app state to tasks 2023-03-10 23:41:34 +01:00
Rafael Caricio 61e566ed9f
Supports graceful shutdown 2023-03-09 16:59:45 +01:00
Rafael Caricio c1fcc87885
Revamp project using newtype and rework tasks table 2023-03-07 16:41:20 +01:00
Rafael Caricio 18303be796
Make workers go brrrr... 2023-03-04 20:46:09 +01:00
Rafael Caricio 0be173ef02
Support only async 2023-03-04 19:07:17 +01:00
Ayrat Badykov 9b92a4a34e
Start documenting the project (#96)
* Start documenting the project

* Add AsyncQueue implementation

* Documenting Async runnable

* By default value in uniq function

* Fix errors and warnings async runnable docs

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* format something

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* uniq documentation

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_queue.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* Update src/asynk/async_runnable.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

Co-authored-by: pxp9 <pepe.marquezromero@gmail.com>
Co-authored-by: Pmarquez <48651252+pxp9@users.noreply.github.com>
2022-10-18 08:30:17 +02:00
Ayrat Badykov 11a7dcc41e
update readme (#95) 2022-09-25 16:05:27 +03:00
Ayrat Badykov 7c0aa2fab9
Retries (#92)
* add retries and errors fields

* Exponential backoff

* async retries (#93)
2022-09-17 19:39:47 +03:00
Pmarquez d3fc479b68
Docs improvement (#80)
* I little bit improve docs

* README

* disable tests for docs
2022-09-03 14:05:58 +03:00
Pmarquez b975e893e2
From db errors (#77)
* Re-work errors

* delete aliases
2022-08-31 17:45:13 +00:00
Pmarquez 360140d064
Blocking refactor (#74)
* Adapting code to new Fang tables structure and refactoring to make it more generic (Not finished)

* Refactoring of the blocking module

* Finishing blocking module, starting to modify tests

* Worker tests done, Queue tests left

* Maybe all tests done ??

* Makefile clippy

* Examples fixed

* asynk feature

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>
2022-08-29 16:59:22 +00:00
Pmarquez dffee33b9c
Asynk change schema (#75)
* add uniq method for AsyncRunnable trait (#70)

* add uniq method for asyncrunnable

* add hash enum

* remove string

* return bool values

* Task struct modified (#71)

* Task struct modified

* asynk module try to adapt new scheme

* delete period in millis

* delete period in millis completed

* Cron support :D

* Cron and single Schedule support :D

* Current timestamp value

* fix bug and new test that confirms that it was a bug

* fix a call

* Update Cargo.toml

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

* comments suggestions

* fix clippy

* Better user api for schedule with cron

* Cron tested with example

* Comments adressed

* Comments adressed

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>
2022-08-27 15:58:38 +00:00
Pmarquez 1dc513c4a7
Sleep with Duration (#67)
* Sleep in millis

* SleepParams Duration :D

* Readme updated!

* Changing Schedule and SleepParams to std::time::Duration

* Millis period in Periodic tasks

* Readme!

* Error management

* period_in_millis i64

* fix clippy
2022-08-18 14:39:10 +03:00
Pmarquez cf2ce19c97
Handle imports and dependencies (#56)
* imports, exports and README

* serde_json not necessay

* deleting tokio_postgres

* authors

* blocking example

* Update fang_examples/simple_async_worker/src/lib.rs

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>
2022-08-03 08:37:53 +00:00
Ayrat Badykov 133d142761
Async worker pool draft (#49)
* Async worker pool draft

* ignore all targets

* simple worker example

* add debug logs

* fix

* fix tests

* remove log

* remove debug logs

* fix clippy
2022-07-31 13:32:37 +00:00
Pmarquez 2d724c3776
Async worker test (#46)
* preparing code to tests

* changing queue tests and starting worker tests with new struct AsyncQueueTest

* fix clippy and delete Debug trait

* fix warnings

* Get task by id in worker tests and AsyncQueueTest #[cfg(test)]

* Defaults and AsyncWorker Builder

* Testing :D

* Execute task specific type

* deleting Options and comment

* insert task is back git !!

* Test remove tasks and changing insert task auxiliar func
2022-07-27 17:05:05 +00:00
Pmarquez 8d0a23e2f9
Async Worker (#44)
* stating with async worker

* async sleep solve and update state test

* uncomenting line

* retention mode and sleep params in lib.rs

* fixing code style

* trying to solve

* dont like changes

* Add AsyncQueueable trait (#45)

* add trait

* update runnable trait

* fix test

Co-authored-by: Ayrat Badykov <ayratin555@gmail.com>
2022-07-23 17:24:22 +03:00
Ayrat Badykov d585bde870
split into async and blocking features (#36)
* split into async and blocking features

* update Cargo.toml file

* fix example
2022-07-20 16:08:02 +00:00
Pmarquez abe324fc57
Fetch task (#26)
* fetch task async and renaming every Job/job to Task/task

* fixing cargo clippy

* Task instead Job in readme

* change one task because if not test will fail, redefined all tasks structs

* derive feature

* deleting schema

* changing query

* task builder

* fix bug enum fang task state

* fetch_test based on metadata, good improve testing fetch_task

* deleting toSql FromSql derives that are useless for Task

* builders and fail_task change
2022-07-19 13:49:43 +00:00
Pmarquez a60eb08fa6
fang_tasks queries done (#24)
* fang_tasks queries done

* queries with transaction rollback

* deleting unused imports

* some tests

* deleting schema , deleting feature section , updated_at corrected

* rollback and commit methods and execute modified

* transaction empty message

* fix cargo clippy
2022-07-18 10:44:05 +00:00
Ayrat Badykov a707ce2f5c
start working on async queries 2022-07-17 10:13:45 +03:00
Ayrat Badykov f999e55fa4
fix clippy 2022-07-17 09:16:54 +03:00
Ayrat Badykov b1fbe8c5af
create sync mod 2022-07-17 09:10:32 +03:00
Ayrat Badykov 3bb45022d9
update typetag (#19)
* update `typetag`

typetage now requires rust 1.62

* stable rust

* disable extra-unused-lifetimes

the warning originates from diesel macro

* fix warning name
2022-07-01 09:03:27 +03:00
J 0473978460
Support graceful shutdown of worker (#14)
* Include shared state to allow graceful shutdown

Graceful shutdown of executors allows the current task to finish before
exiting. This prevents half completed tasks in the general case: when
workers are being scaled down.

To accomplish this a shared state (using an `Arc<RwLock<WorkerState>>`)
is created in a WorkerPool on instantiation. This shared state is then
passed to each thread (spawned with `WorkerThread::spawn_in_pool`), and
finally passed to the `Executor` instantiated by the `WorkerThread`.
This allows the infinit loop in the executor to receive signals from the
`WorkerPool`, and exit gracefully when requested.

* Add basic error handling

Add `FangError` enum derived from `thiserror::Error`. This should be the
default the error type for the Fang library, all errors returned by Fang
should be a value in this enum.

Use FangError for errors returned by start and shutdown, remove unwraps.

* Include instructions for running tests locally

* Track handles of panic'd worker threads

Allows the Drop trait impl of ThreadWorker to access the
`thread_join_handles` of WorkerPool so it can update the thread handle
when the previous thread unexpectedly exited and a new one is being
started. This is done in a way that prevents data leaks (by using a
Hashmap keyed off the name of the worker thread). It also ensures that
threads started from the Drop impl are properly joined on shutdown.

* Fix WorkerThread drop implementation

WorkerThread can not have `Clone` derived on it, as each cloned copy
will try to restart the thread when it's dropped, leading to an infinite
number of thread spawns till stack overflow. Oops

* Remove Option from SharedState type declaration

Instead of having an Option wrapping an enum, have the option codified
as a state in the enum.

* Bump version to 0.5.0

* Add integration test for shutdown

* Update simple_worker example to include signal monitoring and shutdown

* Update readme to mention using signal-hook to gracefully shutdown worker
2021-12-05 08:19:08 +02:00
Ayrat Badykov 4bf654c9b5
Do not use env logger, call log instead directly (#11)
* Do not use env logger, call log instead directly

* update example
2021-08-22 21:25:49 +03:00
Ayrat Badykov 93e7e57d9a
Add simple example (#9)
* Add simple example

* fix env_logger
2021-07-31 09:47:53 +03:00
Ayrat Badykov 824e32f17b
pass PgConnection into run function (#8)
* pass PgConnection into run function

Changes:

- rename Postgres into Queue
- pass PgConnection into run function

* remove all tasks of specific type

* add pgconnection

* pass connection to every queue function

* add connection pool

* reuse connection in test

* update readme
2021-07-25 15:20:16 +03:00
Ayrat Badykov 7d92fd2d6a
Update Readme (#7)
* Update Readme

* change migration path in the readme
2021-07-24 12:13:59 +03:00
Ayrat Badykov 45eb336b8a
Periodic tasks (#5)
* Periodic tasks

* schedule next execution

* add scheduler

* ignore test

* fix clippy

* make start public

* check if the period task already exists

* do not insert task if it's already in the queue

* fix tests
2021-07-18 22:09:30 +03:00
Ayrat Badykov 12f6944e59
add github actions (#4)
* add github actions

* fix clippy

* skip check

* get rid of collect

* fix tests

* run ignored tests
2021-07-11 13:17:02 +03:00
Ayrat Badykov 4f50385c96
simplify usage by re-exporting serde traits (#3)
* simplify usage by re-exporting serde traits

* improve readme formatting

* use typetag

* try different approach

* reexport through macro

* fix re-exporting

* add CHANGELOG entry

* use typetag::serde in README
2021-07-11 10:26:20 +03:00
Ayrat Badykov a2e4885bdb
re-export all structs 2021-06-24 07:48:33 +03:00
Ayrat Badykov d293e31ac8
run tasks in threads 2021-06-23 13:48:03 +03:00
Ayrat Badykov 3bc32c9c93
add job pool 2021-06-23 08:32:35 +03:00
Ayrat Badykov 3de7038e62
add scheduler 2021-06-20 11:58:03 +03:00
Ayrat Badykov e52c81ddde
execute a task 2021-06-20 11:10:02 +03:00
Ayrat Badykov 506fd1c4cb
insert tasks 2021-06-06 11:44:46 +03:00
Ayrat Badykov 8f1f1cc7fa
start working on storage level 2021-06-05 14:39:19 +03:00
Ayrat Badykov d31d6096fa
initial commit 2021-05-30 11:35:00 +03:00