This commit also decouples the "import" and "update" functions inside
ContentProxy. If a content array is available, it must be passed to the
new importEntry method.
Entry API can now have these new fields:
- content
- language
- preview_picture
- published_at
Re-use the ContentProxy to be able to do the same using the web UI (in the future).
htmLawed is used to clean stuff from content, I hope it’ll be enough to avoid security breach.
Lower content validation when we want to update an entry with content already defined. Before, language & content_type were required. If there weren’t provided, we re-fetched the content using graby. I think these fields aren’t required for an entry to be created. So I removed them.
Which means some import from the v1 export won’t be re-fetched since they provide content, url & title.
Also, remove liberation link from Readability import to avoid overlaping import (from wallabag v1, which had the same link)
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
add docs
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
use username as default
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
rename user to username
typo
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
Instead of just say “Failed to save entry” we’ll save the entry at all cost and try to fetch content. If fetching content failed, the entry will still be saved at least, but without content.
- using javibravo/simpleue
- internal config value are now `import_with_redis` & `import_with_rabbit` which are more clear
- if both option are enable rabbit will be choosen
- services imports related to async are now splitted into 2 files: `redis.yml` & `rabbit.yml`
-
At the moment only Readability & wallabag v2 import allow created_at import.
Pocket removed `time_added` field from their API v2 to v3...
And wallabag v1 doesn't export that value.
Instead of queing real Entry to process, we queue all the item to import from Pocket in a raw format.
Then, the worker retrieve that information, find / create the entry and save it.
graby will throw an Exception in some case (like a bad url, a restricted url or a secured pdf).
Import doesn't handle that case and break the whole import.
With that commit the import isn't stopped but the entry is just skipped.
Also, as a bonus, I've added extra test on WallabagImportV2 when the json is empty.
Instead of using a HTTP request we just login user like FOSUser does.
It allows us to mock service in container for functional tests.
Also, fix a bad config name in fos_user for firewall
And finally, add functional test to PocketImport
