Commit graph

72 commits

Author SHA1 Message Date
Kevin Decherf
19802d8bd5 Improve performance of REST exists call
I've noticed that the endpoint `/api/entries/exists` used by the "Sweep
articles" feature on the Android app failed almost all the time on my
instance.

After checking the corresponding method I found that
`EntryRestController::getEntriesExistsAction()` could be improved.

Here is the former way the method worked:

```
for id in [list of ids]
  get full entry by id
  if null
    get full entry by given id

return array of ids or array of hashes
```

With this behavior on my instance I could expect up to 13k SQL requests
when sweeping articles from the Android app. Morever the repository
fetches all fields (content included) while the method only returns ids
or hashes.

The new behavior is described as follow:

```
get ids, hashes by [list of ids]
merge with provided [list of ids] // this part will complete the final
                                  // array with not found ids

return array of ids or array of hashes
```

In my case this change reduces the number of SQL requests to only 135
(_considering one request for 50 articles_)

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2021-08-05 23:19:08 +02:00
Nicolas Lœuillet
a07872238b
Fixed bug when we delete entry via API
Fixes #4291
2020-04-06 16:11:05 +02:00
Jeremy Benoist
8d4ed0df06
Update deps
Also CS (because cs-fixer got an update)

Package operations: 0 installs, 26 updates, 0 removals
  - Updating twig/twig (v2.12.1 => v2.12.2)
  - Updating symfony/symfony (v3.4.33 => v3.4.34)
  - Updating doctrine/event-manager (v1.0.0 => 1.1.0)
  - Updating doctrine/collections (v1.6.2 => 1.6.3)
  - Updating doctrine/cache (v1.8.1 => 1.9.0)
  - Updating doctrine/persistence (1.1.1 => 1.2.0)
  - Updating doctrine/inflector (v1.3.0 => 1.3.1)
  - Updating symfony/mime (v4.3.5 => v4.3.7)
  - Updating swiftmailer/swiftmailer (v6.2.1 => v6.2.3)
  - Updating symfony/swiftmailer-bundle (v3.3.0 => v3.3.1)
  - Updating doctrine/dbal (v2.9.2 => v2.9.3)
  - Updating doctrine/instantiator (1.2.0 => 1.3.0)
  - Updating j0k3r/graby-site-config (1.0.93 => 1.0.94)
  - Updating phpoption/phpoption (1.5.0 => 1.5.2)
  - Updating symfony/http-client-contracts (v1.1.7 => v1.1.8)
  - Updating symfony/http-client (v4.3.5 => v4.3.7)
  - Updating sensiolabs/security-checker (v6.0.2 => v6.0.3)
  - Updating paragonie/constant_time_encoding (v2.2.3 => v2.3.0)
  - Updating scheb/two-factor-bundle (v4.7.1 => v4.8.0)
  - Updating symfony/phpunit-bridge (v4.3.6 => v4.3.7)
  - Updating composer/xdebug-handler (1.3.3 => 1.4.0)
  - Updating friendsofphp/php-cs-fixer (v2.15.3 => v2.16.0)
  - Updating doctrine/data-fixtures (v1.3.2 => 1.3.3)
  - Updating nette/schema (v1.0.0 => v1.0.1)
  - Updating nikic/php-parser (v4.2.4 => v4.3.0)
  - Updating sentry/sentry (2.2.2 => 2.2.4)
2019-11-12 14:18:58 +01:00
Jeremy Benoist
52e8d93248
Fix some Scrutinizer issues 2019-05-29 12:50:44 +02:00
Jérémy Benoist
2cbee36a01
Merge pull request #3944 from shtrom/always-hash-exists-url
Always hash exists url
2019-05-28 14:18:33 +02:00
adev
5f08426201
Fix because of some breaking changes of Graby 2.0 2019-05-28 11:42:27 +02:00
Jeremy Benoist
629a3797bc
Remove useless methods
Also fix a phpdoc block
2019-05-24 15:46:28 +02:00
Olivier Mehani
4a5516376b
Add Wallabag\CoreBundle\Helper\UrlHasher
Signed-off-by: Olivier Mehani <shtrom@ssji.net>
2019-05-24 15:17:46 +02:00
Olivier Mehani
31e276fc16
EntryRestController::getEntriesExistsAction: always find by hashed url
Simplify the logic from #3158 by hashing all the urls from the request,
and only doing a search by hash. This allows to get performance benefits
from the new indexed hash column even when using older clients that do
not hash the URL in the request.

Fixes: #3158, #3919

Signed-off-by: Olivier Mehani <shtrom@ssji.net>
2019-05-24 15:17:45 +02:00
Kevin Decherf
2c290747cb api/entries: add parameter detail to exclude or include content in response
detail=metadata will nullify the content field of entries in order to
make smaller responses.

detail=full keeps the former behavior, it sends the content of entries.
It's the default, for backward compatibility.

Fixes #2817

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-05-18 18:11:08 +02:00
Jérémy Benoist
76bc05ebc0
Fix ApiDoc about md5/sha1 2019-04-02 22:59:50 +02:00
Jeremy Benoist
c579ce2306
Some cleanup
Also, do not run the hashed_url migration into a Doctrine migration
2019-04-01 14:34:20 +02:00
Jeremy Benoist
9c2b2aae70
Keep url in exists endpoint
- Add migration
- Use md5 instead of sha512 (we don't need security here, just a hash)
- Update tests
2019-04-01 13:24:40 +02:00
Thomas Citharel
bfe02a0b48
Hash the urls to check if they exist
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2019-04-01 13:24:40 +02:00
Jeremy Benoist
3620dae1e6
Merge remote-tracking branch 'origin/master' into 2.4 2019-04-01 13:16:15 +02:00
Kevin Decherf
508302042f EntryRestController: add support of expect parameter to delete action
The expect parameter enables an application to request the whole entry
or only the id when requesting its deletion.

`expects` defaults to `entry` to prevent any API breakage.

Fixes #3711

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-02-20 15:57:50 +01:00
Jeremy Benoist
8010425444
Fix third argument to Route 2019-01-22 20:42:25 +01:00
Jeremy Benoist
019e1acc49
Factorize sendResponse between Api controllers
And run newer cs fixer
2019-01-22 20:42:25 +01:00
Jeremy Benoist
5419a8368e
Merge remote-tracking branch 'origin/master' into 2.4 2019-01-15 09:41:18 +01:00
Jeremy Benoist
78e3fafa3f
Avoid error when a bad order parameter is given
Only allowed parameter are asc & desc
2019-01-14 17:01:21 +01:00
Sébastien Viande
0e70e81227
Entry: add sort parameter archived 2018-09-21 10:33:33 +02:00
Sébastien Viande
7975395d10
Entry: add archived_at property and updateArchived method 2018-09-21 10:33:33 +02:00
Kevin Decherf
2a1ceb67b4 php-cs-fixer
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-09-05 14:25:32 +02:00
Kevin Decherf
f5ea67e4cf api: copy entry object before sending, to keep id
Workaround for https://github.com/wallabag/android-app/issues/646

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-12-17 21:07:46 +01:00
Kevin Decherf
af29e1bf07 Fix empty title and domain_name when exception is thrown during fetch
Add a new helper to set a default title when it's empty:
1/ use basename part of entry's path, if any
2/ or use domain name

Fixes #2053

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-12-13 22:44:31 +01:00
Kevin Decherf
3b771f51a6 Improve origin_url field description and translations
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-11-19 15:02:11 +01:00
Kevin Decherf
e0ef1a1c8b Add originUrl property to Entry, handle that in EntryRestController, handle migration
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-11-19 15:02:11 +01:00
Sébastien Viande
0978bd849e Fix #3361 Entry Rest controller getEntries cast tags params to string 2017-10-13 10:28:05 +02:00
Jeremy Benoist
f40c88eb1f
Jump to Symfony 3.3 & update others deps
Also update tests urls
2017-10-09 16:45:12 +02:00
François D
a991c46eed Set a starred_at field when an entry is starred.
This date is used to sort starred entries.

Can not use Entry::timestamps method otherwise starred_at will be updated each time entry is updated.
Add an updateStar method into Entry class
A migration script has been added in order to set starred_at field.
2017-08-25 21:19:47 +02:00
Nicolas Hart
52b84c11a5 Fix some namespaces and phpdoc 2017-07-29 22:51:50 +02:00
Étienne Gilli
07320a2bd2 Use the article domain as author for export files
When exporting an entry, use the domain name as author name for epub,
mobi and pdf formats, instead of 'wallabag'.
Change the author from array to string, because for now, there is always
only one author.
2017-07-08 19:53:43 +02:00
Jeremy Benoist
c18a2476b6
CS 2017-07-03 13:56:39 +02:00
Jeremy Benoist
a05b61159e
Fix PATCH method
The PATCH method for the entry should only update what user sent to us and not the whole entry as it was before.
Also, sending tags when patching an entry will now remove all current tags & assocatied new ones.
2017-07-03 13:45:04 +02:00
Jeremy Benoist
f808b01692
Add a real configuration for CS-Fixer 2017-07-01 09:52:38 +02:00
Jérémy Benoist
331e5b0261 CS 2017-06-28 09:18:22 +02:00
Jeremy Benoist
39ffaba323
Return null instead of false 2017-06-28 08:15:06 +02:00
Jeremy Benoist
18696f77fd
Avoid BC on exists endpoint 2017-06-23 08:30:23 +02:00
Jeremy Benoist
d9da186fb8
CS 2017-06-10 16:08:24 +02:00
Jeremy Benoist
a9c6577f0c
Few fixes 2017-06-10 15:37:25 +02:00
Jeremy Benoist
1112e54772
Add public filter/field in the API
Listing entries can now be filtered by “public”.
Creating or patching an entry can now set is to public or remove the public.
Entry response now include “is_public” boolean field
2017-06-10 15:31:57 +02:00
Jeremy Benoist
db0c48af36
Refactorize the way to save an Entry in the API 2017-06-07 15:07:55 +02:00
Jeremy Benoist
645291e8fe
Add ability to patch an entry with more fields
Like when we create an entry, we can now patch an entry with new fields:
- content
- language
- preview_picture
- published_at
- authors
2017-06-02 20:52:49 +02:00
Jeremy Benoist
d5c2cc54b5
Fix tests 2017-06-01 09:49:15 +02:00
Jerome Charaoui
7aba665e48
Avoid returning objects passed by reference.
Objects are always passed by reference, so it doesn't make sense to
return an object which is passed by reference as it will always be the
same object. This change makes the code a bit more readable.
2017-06-01 09:43:01 +02:00
Jeremy Benoist
f0378b4d7c
Forced date can now be a timestamp too
Add adding more tests for forced content
2017-05-31 14:00:15 +02:00
Jeremy Benoist
9e349f08a6
Improve docs 2017-05-31 14:00:15 +02:00
Jeremy Benoist
74a75f7d43
Use graby ContentExtractor to clean html
It might be better to re-use some graby functionalities to clean html instead of building a new system.
2017-05-31 14:00:15 +02:00
Jeremy Benoist
fb436e8ca0
Add support for authors 2017-05-31 13:59:45 +02:00
Jeremy Benoist
e668a8124c
Allow other fields to be send using API
Entry API can now have these new fields:
- content
- language
- preview_picture
- published_at

Re-use the ContentProxy to be able to do the same using the web UI (in the future).
htmLawed is used to clean stuff from content, I hope it’ll be enough to avoid security breach.

Lower content validation when we want to update an entry with content already defined. Before, language & content_type were required. If there weren’t provided, we re-fetched the content using graby. I think these fields aren’t required for an entry to be created. So I removed them.
Which means some import from the v1 export won’t be re-fetched since they provide content, url & title.

Also, remove liberation link from Readability import to avoid overlaping import (from wallabag v1, which had the same link)
2017-05-31 13:59:45 +02:00