Commit graph

99 commits

Author SHA1 Message Date
Jeremy Benoist 990adfb34c
Move prepareEntry to dedicated place
Yeah first try was ugly, now each part are in the dedicated place.
Also, the date is hardly truncated to 10 chars because Firefox date are 16 chars long and Chrome are 17 chars long. So instead of divised them by a huge number, I prefer to truncate them.
2016-09-26 07:30:02 +02:00
Jeremy Benoist 12d93e6896
Update Firefox file
With real data, the previous looks more than a Chrome converted file.
Also, fix date conversion (hope so).
2016-09-25 22:24:07 +02:00
Jeremy Benoist 27acc6ddb8
Fix bad date format in Browser import 2016-09-25 15:32:02 +02:00
Nicolas Lœuillet bd206a84d8
Fixed tests by removing clear() 2016-09-25 12:29:19 +02:00
Thomas Citharel 64b1229b2d
fix tests 2016-09-25 12:29:19 +02:00
Thomas Citharel 2c61db30b7
cs & fixes 2016-09-25 12:29:19 +02:00
Thomas Citharel 59201088b4
bring chrome and firefox as separate imports 2016-09-25 12:29:18 +02:00
Nicolas Lœuillet efe659ab84
Add Chrome path for Mac OS 2016-09-25 12:28:55 +02:00
Thomas Citharel ae669126e7
Import Firefox & Chrome bookmarks into wallabag 2016-09-25 12:28:54 +02:00
Jeremy Benoist 13a522dfbd
Display a message when async import won’t work
Mostly if Redis or RabbitMQ isn’t installed / launched.
I guess it’ll avoid some issues.
2016-09-24 19:56:15 +02:00
Jeremy Benoist 59b97fae99
Avoid losing entry when fetching fail
Instead of just say “Failed to save entry” we’ll save the entry at all cost and try to fetch content. If fetching content failed, the entry will still be saved at least, but without content.
2016-09-17 07:40:56 +02:00
Jeremy Benoist fbb319f064
Missing some migrations
and CS
2016-09-16 22:58:33 +02:00
Jeremy Benoist ebe0787e09
Moved Pocket token to user config 2016-09-16 22:22:25 +02:00
Jeremy Benoist 4fc998245c
Only display message in queue for admin
Instead of for EVERYONE
2016-09-16 20:08:07 +02:00
Jeremy Benoist ac87e0db2a
AMPQ -> AMQP 2016-09-14 10:18:02 +02:00
Jeremy Benoist c80cc01afa
Change flash message for queued articles 2016-09-13 21:09:05 +02:00
Jeremy Benoist e01a3c98d6
Display how many messages are queue
- update the docker-composer to add Redis
- add migrations
2016-09-13 20:32:52 +02:00
Jeremy Benoist ebf5e5087d
Add tests on ImportCommand 2016-09-11 22:15:31 +02:00
Jeremy Benoist 015c7a8359
Add more tests
And ability to define how many messages can be hanle by the redis worker before stopping (usefull for tests)
2016-09-11 22:15:31 +02:00
Jeremy Benoist 7d862f83b9
Re-facto EntryConsumer
Using an abstract method allow to share code but also can be used it we add a new broker in the future
2016-09-11 22:15:31 +02:00
Jeremy Benoist dc69e25f97
Display a message when async is enabled 2016-09-11 22:15:28 +02:00
Jeremy Benoist b3437d58ae
Enable Redis async import
- using javibravo/simpleue
- internal config value are now `import_with_redis` & `import_with_rabbit` which are more clear
- if both option are enable rabbit will be choosen
- services imports related to async are now splitted into 2 files: `redis.yml` & `rabbit.yml`
-
2016-09-11 21:58:56 +02:00
Jeremy Benoist 7f7531171f
Retrieve created date from Pocket 2016-09-11 21:58:56 +02:00
Jeremy Benoist 13470c3596
Add test for RabbitMQ
Also update Symfony deps
2016-09-11 21:58:56 +02:00
Jeremy Benoist 8664069e1a
Fix DateTime & clear() 2016-09-11 21:58:55 +02:00
Jeremy Benoist 6d65c0a8b0
Add ability to define created_at for all import
At the moment only Readability & wallabag v2 import allow created_at import.
Pocket removed `time_added` field from their API v2 to v3...
And wallabag v1 doesn't export that value.
2016-09-11 21:58:55 +02:00
Jeremy Benoist 3aca0a9f00
CS 2016-09-11 21:58:55 +02:00
Jeremy Benoist 3849a9f323
Some cleanup & refactor 2016-09-11 21:58:55 +02:00
Jeremy Benoist 02f6489572
Retrieve all items from Pocket
5000 by 5000.
Also, retrieve newest item first.
2016-09-11 21:58:31 +02:00
Jeremy Benoist c98db1b653
Convert other imports to Rabbit 2016-09-11 21:58:31 +02:00
Jeremy Benoist ef75e1220e
Send every imported item to the queue
Instead of queing real Entry to process, we queue all the item to import from Pocket in a raw format.
Then, the worker retrieve that information, find / create the entry and save it.
2016-09-11 21:57:46 +02:00
Nicolas Lœuillet 40d2a29443
Replace RabbitMQ injection with CraueConfiguration 2016-09-11 21:57:09 +02:00
Nicolas Lœuillet 56c778b415
1st draft for rabbitMQ 2016-09-11 21:57:09 +02:00
Jeremy Benoist 88e88016b9
Clearing entities in the loop fail on Postgres
It looks like when you clear entities on Postgres some references are lost and tags are not saved :-/
2016-09-11 16:30:01 +02:00
Jeremy Benoist 58fadbc9df
Fix error on EntityManager clear
Introduced in the recent 2.5.5 release.
Also updated deps.
2016-09-11 15:53:16 +02:00
Jeremy Benoist a1a1077059
Add tests on ReadabilityImport 2016-09-01 13:39:41 +02:00
Jeremy Benoist 03e3753f6b
Add Readability import
Based on the JSON export instead of the API (which will be shutting down by the September 30, 2016)
2016-09-01 08:00:30 +02:00
Jeremy Benoist 79efca1e6f
Merge remote-tracking branch 'origin/master' into 2.1 2016-08-22 23:03:16 +02:00
Jeremy Benoist 4e4e8e9899 Merge pull request #2221 from wallabag/replace-favorite-star
Replaced favorite word/icon with star one
2016-08-21 22:11:20 +02:00
Jeremy Benoist 19d9efab32
Avoid breaking import when fetching fail
graby will throw an Exception in some case (like a bad url, a restricted url or a secured pdf).

Import doesn't handle that case and break the whole import.
With that commit the import isn't stopped but the entry is just skipped.

Also, as a  bonus, I've added extra test on WallabagImportV2 when the json is empty.
2016-08-20 01:17:26 +02:00
Nicolas Lœuillet 519ba0b5e7 Replaced favorite word/icon with star one
Fix #2041
2016-08-18 14:27:17 +02:00
Jeremy Benoist f49d9ca383 Merge branch 'master' into 2.1 2016-06-23 11:47:46 +02:00
Jeremy Benoist 23634d5d84 Jump to Symfony 3.1 2016-06-22 17:59:35 +02:00
Nicolas Lœuillet 235026e2c7 Enhance option in command 2016-05-27 09:43:49 +02:00
Nicolas Lœuillet 3fad6c74fe Add CLI import for wallabag v2 files 2016-05-27 09:43:48 +02:00
Nicolas Lœuillet 23d24b1750
Add tests 2016-04-18 15:29:57 +02:00
Nicolas Lœuillet dc12084d93 Add doctrine clear after flush, thanks to @BitOne talk at Symfony Live 2016-04-18 14:48:18 +02:00
Jeremy Benoist 4094ea4771 Convert array + phpDoc
Thanks for https://github.com/thomasbachem/php-short-array-syntax-converter
2016-04-12 12:25:29 +02:00
Jeremy Benoist b787a7757e Refacto wallabag import
Use an abstract class to store all common action from wallabag vX import.
Move specificity in v1 & v2 import.
2016-03-28 16:43:33 +02:00
Jeremy Benoist 8f336fda64 Tags were not imported in wallabag v2 import
Also, simplify exportAs matching format
2016-03-27 23:58:37 +02:00