Commit graph

117 commits

Author SHA1 Message Date
Nicolas Lœuillet ca845b3204
Merge pull request #5379 from wallabag/fix-search-homepage
Fixed search on homepage
2021-08-20 11:16:33 +02:00
Kevin Decherf 19802d8bd5 Improve performance of REST exists call
I've noticed that the endpoint `/api/entries/exists` used by the "Sweep
articles" feature on the Android app failed almost all the time on my
instance.

After checking the corresponding method I found that
`EntryRestController::getEntriesExistsAction()` could be improved.

Here is the former way the method worked:

```
for id in [list of ids]
  get full entry by id
  if null
    get full entry by given id

return array of ids or array of hashes
```

With this behavior on my instance I could expect up to 13k SQL requests
when sweeping articles from the Android app. Morever the repository
fetches all fields (content included) while the method only returns ids
or hashes.

The new behavior is described as follow:

```
get ids, hashes by [list of ids]
merge with provided [list of ids] // this part will complete the final
                                  // array with not found ids

return array of ids or array of hashes
```

In my case this change reduces the number of SQL requests to only 135
(_considering one request for 50 articles_)

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2021-08-05 23:19:08 +02:00
Nicolas Lœuillet f664af752d Fixed search on homepage 2021-08-03 11:54:36 +02:00
Nicolas Lœuillet 5857031262 Fixed query to fetch URL to hash 2020-12-22 14:58:07 +01:00
Nicolas Lœuillet 68060f545a Added a query to parse only non-hashed URL
Fixed #4864
2020-12-20 09:23:59 +01:00
Jeremy Benoist c10b79e82f
Update PagerFanta deprecation 2020-07-29 06:36:43 +02:00
Jeremy Benoist 8d4ed0df06
Update deps
Also CS (because cs-fixer got an update)

Package operations: 0 installs, 26 updates, 0 removals
  - Updating twig/twig (v2.12.1 => v2.12.2)
  - Updating symfony/symfony (v3.4.33 => v3.4.34)
  - Updating doctrine/event-manager (v1.0.0 => 1.1.0)
  - Updating doctrine/collections (v1.6.2 => 1.6.3)
  - Updating doctrine/cache (v1.8.1 => 1.9.0)
  - Updating doctrine/persistence (1.1.1 => 1.2.0)
  - Updating doctrine/inflector (v1.3.0 => 1.3.1)
  - Updating symfony/mime (v4.3.5 => v4.3.7)
  - Updating swiftmailer/swiftmailer (v6.2.1 => v6.2.3)
  - Updating symfony/swiftmailer-bundle (v3.3.0 => v3.3.1)
  - Updating doctrine/dbal (v2.9.2 => v2.9.3)
  - Updating doctrine/instantiator (1.2.0 => 1.3.0)
  - Updating j0k3r/graby-site-config (1.0.93 => 1.0.94)
  - Updating phpoption/phpoption (1.5.0 => 1.5.2)
  - Updating symfony/http-client-contracts (v1.1.7 => v1.1.8)
  - Updating symfony/http-client (v4.3.5 => v4.3.7)
  - Updating sensiolabs/security-checker (v6.0.2 => v6.0.3)
  - Updating paragonie/constant_time_encoding (v2.2.3 => v2.3.0)
  - Updating scheb/two-factor-bundle (v4.7.1 => v4.8.0)
  - Updating symfony/phpunit-bridge (v4.3.6 => v4.3.7)
  - Updating composer/xdebug-handler (1.3.3 => 1.4.0)
  - Updating friendsofphp/php-cs-fixer (v2.15.3 => v2.16.0)
  - Updating doctrine/data-fixtures (v1.3.2 => 1.3.3)
  - Updating nette/schema (v1.0.0 => v1.0.1)
  - Updating nikic/php-parser (v4.2.4 => v4.3.0)
  - Updating sentry/sentry (2.2.2 => 2.2.4)
2019-11-12 14:18:58 +01:00
Jeremy Benoist 0f2d24feb4
Fix typo & CS 2019-06-05 17:09:05 +02:00
Kevin Decherf ad51743e8b
Show untagged entries count on tag list
Closes #3235

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-06-05 17:03:51 +02:00
Jérémy Benoist 16e1c07553
Merge pull request #3271 from wallabag/store-resolved-url
Add `given_url` in Entry table to check if a redirected url has already added
2019-06-05 11:38:00 +02:00
Jeremy Benoist 70df4c3359
Use two indexes instead of one for hashed urls
When using `OR` in a where clause, a composite index can't be used. We should use a `UNION` to take advantages of it.
Instead, create 2 indexes on each hashed urls and make 2 queries to find an url. It'll be faster than the previous solution.
2019-06-05 10:53:15 +02:00
Jeremy Benoist f3bfb875e9
Use hash given url to avoid duplicate
Using hashed url we can ensure an index on them to ensure it's fast.
2019-05-29 15:56:20 +02:00
Nicolas Lœuillet b7fa51ae7d
Added given_url in entry table
- Added index on entry table for given_url field
- Fix tests:

    The previous `bit.ly` url redirected to doc.wallabag but that url doesn't exist in the fixtures.
    I used our own internal "redirector" to create a redirect to an url which exist in the fixtures.

Also, updating current migration to use the new `WallabagMigration`.
2019-05-29 13:50:59 +02:00
Jeremy Benoist 52e8d93248
Fix some Scrutinizer issues 2019-05-29 12:50:44 +02:00
Jeremy Benoist 629a3797bc
Remove useless methods
Also fix a phpdoc block
2019-05-24 15:46:28 +02:00
Jeremy Benoist 0132ccd2a2
Change the way to define algorithm for hashing url 2019-05-24 15:17:46 +02:00
Olivier Mehani 4a5516376b
Add Wallabag\CoreBundle\Helper\UrlHasher
Signed-off-by: Olivier Mehani <shtrom@ssji.net>
2019-05-24 15:17:46 +02:00
Olivier Mehani d5744bf0df
Delegate findByUrlAndUserId to findByHashedUrlAndUserId
Signed-off-by: Olivier Mehani <shtrom@ssji.net>
2019-05-24 15:17:46 +02:00
Kevin Decherf 2c290747cb api/entries: add parameter detail to exclude or include content in response
detail=metadata will nullify the content field of entries in order to
make smaller responses.

detail=full keeps the former behavior, it sends the content of entries.
It's the default, for backward compatibility.

Fixes #2817

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-05-18 18:11:08 +02:00
Jeremy Benoist c579ce2306
Some cleanup
Also, do not run the hashed_url migration into a Doctrine migration
2019-04-01 14:34:20 +02:00
Jeremy Benoist 9c2b2aae70
Keep url in exists endpoint
- Add migration
- Use md5 instead of sha512 (we don't need security here, just a hash)
- Update tests
2019-04-01 13:24:40 +02:00
Jeremy Benoist 091bafeb4c
Handle no random result found 2019-01-19 22:30:50 +01:00
Jeremy Benoist 50f35f0db2
Move icon into the top menu bar
Change the way to select a random entry:
- select all ids from the given user (with filters)
- choose randomly one in php
- find that entry
2019-01-19 22:24:10 +01:00
Jeremy Benoist 9a57653aec
Redirect to the current view instead of homepage 2019-01-19 21:10:16 +01:00
Jeremy Benoist 062fad434a
Better random function 2019-01-19 21:09:33 +01:00
Jeremy Benoist f85d220c19
Fix tests 2019-01-19 21:09:33 +01:00
Nicolas Lœuillet 09ef25c3c3
Added random feature 2019-01-19 21:09:32 +01:00
Jeremy Benoist 5419a8368e
Merge remote-tracking branch 'origin/master' into 2.4 2019-01-15 09:41:18 +01:00
Jeremy Benoist 78e3fafa3f
Avoid error when a bad order parameter is given
Only allowed parameter are asc & desc
2019-01-14 17:01:21 +01:00
Jeremy Benoist 9007fe0062
Sort archive page by archived at 2018-09-21 11:18:29 +02:00
Sébastien Viande 7c0d682687
Code Style 2018-09-21 10:33:33 +02:00
Sébastien Viande 0e70e81227
Entry: add sort parameter archived 2018-09-21 10:33:33 +02:00
Jeremy Benoist 17476f4d8d
Add missing parameters
Parameters weren’t passed to the sub function.
2018-09-07 13:56:14 +02:00
Kevin Decherf b8115ff46b php-cs-fixer
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-09-05 18:44:08 +02:00
Kevin Decherf 0636697289 EntryRepository: refactor getBuilderForUntaggedByUser
Improve SQL performance by replacing size(e.tags) with a left join and a
null condition

Move the QueryBuilder logic into getRawBuilderForUntaggedByUser

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-09-05 18:44:08 +02:00
Kevin Decherf b7c5fda512 EntryRepository: refactor getBuilderByUser
We refactor getBuilderByUser to separate QueryBuilder getter and the
orderBy(). The previous code of getBuilderByUser() has been moved to
getSortedQueryBuilderByUser(). getBuildByUser() now returns a
QueryBuilder without the call to orderBy().

A new method named sortQueryBuilder() returns a given QueryBuilder with
an orderBy() call using given sort parameters.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-09-05 18:44:08 +02:00
Kevin Decherf 2a1ceb67b4 php-cs-fixer
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-09-05 14:25:32 +02:00
Sébastien Viande fafdf17117 Fix #3361 check type for tags in entry repository
Check is $tags is a string before explode
2017-10-12 07:38:27 +02:00
Jeremy Benoist 9c4a7388da
Remove unused function
Introduce after the rebase I guess
2017-09-06 22:58:35 +02:00
Jeremy Benoist 33264c2d02
Fix tests 2017-09-06 22:49:16 +02:00
Jeremy Benoist 7c04b7396a
Multiple tag search was broken from API
First, the setParameter() were done on the same parameter which in fact
just duplicated the condition in the SQL query (like `where t.label =
'test' and t.label = 'test'`.

Changed the parameter doesn't help because the query was then wrong.

Changing the way to match associated tags for an entry and it worked.
2017-09-06 22:49:15 +02:00
François D a991c46eed Set a starred_at field when an entry is starred.
This date is used to sort starred entries.

Can not use Entry::timestamps method otherwise starred_at will be updated each time entry is updated.
Add an updateStar method into Entry class
A migration script has been added in order to set starred_at field.
2017-08-25 21:19:47 +02:00
Nicolas Hart dbf1188c5b rename getAllEntriesIdAndUrl into findAllEntriesIdAndUrlByUserId 2017-08-22 10:42:54 +02:00
Nicolas Hart 215409a8b2 rename getAllEntriesId into findAllEntriesIdByUserId 2017-08-22 10:41:28 +02:00
Nicolas Hart 511f1ce1e8 Add reload entry command 2017-08-21 10:45:48 +02:00
Nicolas Hart 935e9fffb4 Reduce number of queries on tag list 2017-08-06 23:02:32 +02:00
Nicolas Hart 52b84c11a5 Fix some namespaces and phpdoc 2017-07-29 22:51:50 +02:00
Jeremy Benoist 38520658ad
Fix tests 2017-07-03 07:30:54 +02:00
Jeremy Benoist f808b01692
Add a real configuration for CS-Fixer 2017-07-01 09:52:38 +02:00
Jeremy Benoist 1112e54772
Add public filter/field in the API
Listing entries can now be filtered by “public”.
Creating or patching an entry can now set is to public or remove the public.
Entry response now include “is_public” boolean field
2017-06-10 15:31:57 +02:00