Improve performance of REST exists call

I've noticed that the endpoint `/api/entries/exists` used by the "Sweep
articles" feature on the Android app failed almost all the time on my
instance.

After checking the corresponding method I found that
`EntryRestController::getEntriesExistsAction()` could be improved.

Here is the former way the method worked:

```
for id in [list of ids]
  get full entry by id
  if null
    get full entry by given id

return array of ids or array of hashes
```

With this behavior on my instance I could expect up to 13k SQL requests
when sweeping articles from the Android app. Morever the repository
fetches all fields (content included) while the method only returns ids
or hashes.

The new behavior is described as follow:

```
get ids, hashes by [list of ids]
merge with provided [list of ids] // this part will complete the final
                                  // array with not found ids

return array of ids or array of hashes
```

In my case this change reduces the number of SQL requests to only 135
(_considering one request for 50 articles_)

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
This commit is contained in:
Kevin Decherf 2021-04-29 00:19:37 +02:00
parent dc6e1af684
commit 19802d8bd5
2 changed files with 33 additions and 21 deletions

View file

@ -67,11 +67,24 @@ class EntryRestController extends WallabagRestController
throw $this->createAccessDeniedException('URL is empty?, logged user id: ' . $this->getUser()->getId());
}
$results = [];
foreach ($hashedUrls as $hashedUrlToSearch) {
$res = $repo->findByHashedUrlAndUserId($hashedUrlToSearch, $this->getUser()->getId());
$results = array_fill_keys($hashedUrls, null);
$res = $repo->findByUserIdAndBatchHashedUrls($this->getUser()->getId(), $hashedUrls);
foreach ($res as $e) {
$_hashedUrl = array_keys($hashedUrls, 'blah', true);
if ([] !== array_keys($hashedUrls, $e['hashedUrl'], true)) {
$_hashedUrl = $e['hashedUrl'];
} elseif ([] !== array_keys($hashedUrls, $e['hashedGivenUrl'], true)) {
$_hashedUrl = $e['hashedGivenUrl'];
} else {
continue;
}
$results[$_hashedUrl] = $e['id'];
}
$results[$hashedUrlToSearch] = $this->returnExistInformation($res, $returnId);
if (false === $returnId) {
$results = array_map(function ($v) {
return null !== $v;
}, $results);
}
$results = $this->replaceUrlHashes($results, $urlHashMap);
@ -840,21 +853,4 @@ class EntryRestController extends WallabagRestController
'origin_url' => $request->request->get('origin_url', ''),
];
}
/**
* Return information about the entry if it exist and depending on the id or not.
*
* @param Entry|bool|null $entry
* @param bool $returnId
*
* @return bool|int
*/
private function returnExistInformation($entry, $returnId)
{
if ($returnId) {
return $entry instanceof Entry ? $entry->getId() : null;
}
return $entry instanceof Entry;
}
}

View file

@ -421,6 +421,22 @@ class EntryRepository extends EntityRepository
return false;
}
public function findByUserIdAndBatchHashedUrls($userId, $hashedUrls)
{
$qb = $this->createQueryBuilder('e')->select(['e.id', 'e.hashedUrl', 'e.hashedGivenUrl']);
$res = $qb->where('e.user = :user_id')->setParameter('user_id', $userId)
->andWhere(
$qb->expr()->orX(
$qb->expr()->in('e.hashedUrl', $hashedUrls),
$qb->expr()->in('e.hashedGivenUrl', $hashedUrls)
)
)
->getQuery()
->getResult();
return $res;
}
/**
* Count all entries for a user.
*