Commit graph

87 commits

Author SHA1 Message Date
JT Smith
6da76ffaae Typofixes 2022-10-03 18:31:43 -06:00
Kevin Decherf
1608bf5a4e Replace iconv() calls with Transliterator
See https://stackoverflow.com/a/35178027/954513

Closes #5377

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-03-21 22:12:11 +01:00
Jeremy Benoist
381f9681b5
Fix CS 2021-01-18 10:38:56 +01:00
Kevin Decherf
b22eb27623 ContentProxy: replace ignoreUrl with new RuleBasedIgnoreOriginProcessor
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2020-04-25 15:59:23 +02:00
Jeremy Benoist
8d4ed0df06
Update deps
Also CS (because cs-fixer got an update)

Package operations: 0 installs, 26 updates, 0 removals
  - Updating twig/twig (v2.12.1 => v2.12.2)
  - Updating symfony/symfony (v3.4.33 => v3.4.34)
  - Updating doctrine/event-manager (v1.0.0 => 1.1.0)
  - Updating doctrine/collections (v1.6.2 => 1.6.3)
  - Updating doctrine/cache (v1.8.1 => 1.9.0)
  - Updating doctrine/persistence (1.1.1 => 1.2.0)
  - Updating doctrine/inflector (v1.3.0 => 1.3.1)
  - Updating symfony/mime (v4.3.5 => v4.3.7)
  - Updating swiftmailer/swiftmailer (v6.2.1 => v6.2.3)
  - Updating symfony/swiftmailer-bundle (v3.3.0 => v3.3.1)
  - Updating doctrine/dbal (v2.9.2 => v2.9.3)
  - Updating doctrine/instantiator (1.2.0 => 1.3.0)
  - Updating j0k3r/graby-site-config (1.0.93 => 1.0.94)
  - Updating phpoption/phpoption (1.5.0 => 1.5.2)
  - Updating symfony/http-client-contracts (v1.1.7 => v1.1.8)
  - Updating symfony/http-client (v4.3.5 => v4.3.7)
  - Updating sensiolabs/security-checker (v6.0.2 => v6.0.3)
  - Updating paragonie/constant_time_encoding (v2.2.3 => v2.3.0)
  - Updating scheb/two-factor-bundle (v4.7.1 => v4.8.0)
  - Updating symfony/phpunit-bridge (v4.3.6 => v4.3.7)
  - Updating composer/xdebug-handler (1.3.3 => 1.4.0)
  - Updating friendsofphp/php-cs-fixer (v2.15.3 => v2.16.0)
  - Updating doctrine/data-fixtures (v1.3.2 => 1.3.3)
  - Updating nette/schema (v1.0.0 => v1.0.1)
  - Updating nikic/php-parser (v4.2.4 => v4.3.0)
  - Updating sentry/sentry (2.2.2 => 2.2.4)
2019-11-12 14:18:58 +01:00
Jeremy Benoist
f3bfb875e9
Use hash given url to avoid duplicate
Using hashed url we can ensure an index on them to ensure it's fast.
2019-05-29 15:56:20 +02:00
Nicolas Lœuillet
b7fa51ae7d
Added given_url in entry table
- Added index on entry table for given_url field
- Fix tests:

    The previous `bit.ly` url redirected to doc.wallabag but that url doesn't exist in the fixtures.
    I used our own internal "redirector" to create a redirect to an url which exist in the fixtures.

Also, updating current migration to use the new `WallabagMigration`.
2019-05-29 13:50:59 +02:00
Jeremy Benoist
6e68417f03
Fix tests after rebase 2019-05-28 12:02:17 +02:00
Jeremy Benoist
b6c1e1bacc
Fix some tests 2019-05-28 11:44:20 +02:00
adev
5f08426201
Fix because of some breaking changes of Graby 2.0 2019-05-28 11:42:27 +02:00
nicofrand
423efadefc Set first picture as preview picture 2019-05-21 20:38:22 +02:00
Kevin Decherf
2dbb5b2307 Enable no-referrer on img tags, enable strict-origin-when-cross-origin by default
Fixes #3889

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-05-10 23:07:26 +02:00
Jeremy Benoist
8ca858ee73
Fix PHP warning
Looks like sometimes (usually from import) the `html` key isn’t available.
2019-03-18 06:23:41 +01:00
Kevin Decherf
1b220426e2 phpcs
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-24 22:33:32 +02:00
Kevin Decherf
6059967951 updateOriginUrl: remove 'query string' case from ignore list
Two urls with a different query string may refer to two different pages
so keep them both.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-24 22:27:27 +02:00
Kevin Decherf
44e63667d9 updateOriginUrl: add comment blocks for the parse_url diff check
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-24 22:13:03 +02:00
Kevin Decherf
5ba5e22a09 updateOriginUrl: rewrite some if, resolving feedbacks from PR
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-24 21:54:09 +02:00
Kevin Decherf
b49c87acf1 ignoreOriginUrl: add initial support of ignore lists
Add the ability to specify hosts and patterns lists to ignore the given
entry url and replace it with the fetched content url without touching
to origin_url.

This initial support should be reworked in the following months to move
the hardcoded ignore lists in the database.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:42:09 +02:00
Kevin Decherf
fc040c749d updateOriginUrl: add behavior when diff is fragment and query
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:08:58 +02:00
Kevin Decherf
e07fadea76 Refactor updateOriginUrl to include new behaviors behaviors
- Leave origin_url unchanged if difference is an ending slash
- Leave origin_url unchanged if difference is scheme
- Ignore (noop) if difference is query string or fragment

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:01:16 +02:00
Kevin Decherf
781864b954 ContentProxy: swap entry url to origin_url and set new url according to graby content
Closes #3529

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-21 16:15:31 +02:00
Kevin Decherf
4a81360efc ContentProxy: fix a corner case when entry.url is empty in updateEntry
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-21 16:13:20 +02:00
Tobi823
83f1c3274f Run php-cs-fixer for fixing coding standard issues 2018-09-23 22:20:43 +02:00
Tobi823
7a65c2017b Override the value of the given parameter ($title) with the (hopefully)
correct (to UTF-8) converted PDF title
2018-09-21 13:23:39 +02:00
Tobi823
c01d953292 Add tests for logic
Try to translate the title of a PDF from UTF-8 (then UTF-16BE, then WINDOWS-1252) to UTF-8
2018-09-21 13:15:00 +02:00
Tobi823
f80f16dfc8 Try to detect the character encoding in PDFs and try to translate
the title from the PDF to UTF-8
2018-09-21 13:15:00 +02:00
Tobi823
8648f0c005 Remove type declaration for PHP 5 compatibility 2018-09-21 13:15:00 +02:00
Tobi823
d76a5a6d60 Bugfix: Sanitize the title of a saved webpage from invalid UTF-8 characters 2018-09-21 13:15:00 +02:00
Kevin Decherf
2a1ceb67b4 php-cs-fixer
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-09-05 14:25:32 +02:00
Kevin Decherf
af29e1bf07 Fix empty title and domain_name when exception is thrown during fetch
Add a new helper to set a default title when it's empty:
1/ use basename part of entry's path, if any
2/ or use domain name

Fixes #2053

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-12-13 22:44:31 +01:00
Jeremy Benoist
709e21a3f4
Define storeArticleHeaders false by default
Fix tests which must use `$storeArticleHeaders`.
Fix CS
2017-11-21 10:37:36 +01:00
Nicolas Lœuillet
8a21985474 Added internal setting to enable/disable headers storage 2017-11-20 18:47:48 +01:00
Jeremy Benoist
3ef055ced3
CS 2017-10-09 16:47:15 +02:00
Jeremy Benoist
ff9f89fd23
Add a test for updatePublishedAt
To avoid error when a content is re-submitted and it previously add a
published date.

Also, fix the `testPostSameEntry`
2017-07-24 17:07:47 +02:00
Simounet
b236d3f627
Fix updatePublishedAt on already parsed article's date 2017-07-24 16:39:07 +02:00
Jeremy Benoist
c18a2476b6
CS 2017-07-03 13:56:39 +02:00
Jeremy Benoist
d0ec2ddd23
Fix validateAndSetPreviewPicture
Which wasn't covered by a test!
2017-07-03 13:45:04 +02:00
Jeremy Benoist
a05b61159e
Fix PATCH method
The PATCH method for the entry should only update what user sent to us and not the whole entry as it was before.
Also, sending tags when patching an entry will now remove all current tags & assocatied new ones.
2017-07-03 13:45:04 +02:00
Jeremy Benoist
f808b01692
Add a real configuration for CS-Fixer 2017-07-01 09:52:38 +02:00
Thomas Citharel
41d45c6122 Fix empty language and preview pics 2017-06-12 16:46:33 +02:00
Jeremy Benoist
80e49ba7b0
Convert - to _ in language
Mostly to increase language supports
2017-06-09 11:42:09 +02:00
Jeremy Benoist
42f3bb2c63
Use Locale instead of Language 2017-06-09 11:28:04 +02:00
Jeremy Benoist
be54dfe4e6
CS 2017-06-08 21:56:20 +02:00
Jeremy Benoist
0d349ea670
Validate language & preview picture fields
Instead of saving the value of each field right into the content without any validation, it seems better to validate them.
This might sounds obvious now we say that.
2017-06-08 21:51:46 +02:00
Jeremy Benoist
645291e8fe
Add ability to patch an entry with more fields
Like when we create an entry, we can now patch an entry with new fields:
- content
- language
- preview_picture
- published_at
- authors
2017-06-02 20:52:49 +02:00
Jeremy Benoist
ec97072152
No need to catch that Exception 2017-06-01 11:45:02 +02:00
Jeremy Benoist
6acadf8e98
Rewrote code & fix tests 2017-06-01 11:31:45 +02:00
Jeremy Benoist
843182c7cf
CS 2017-06-01 09:52:09 +02:00
Jeremy Benoist
d5c2cc54b5
Fix tests 2017-06-01 09:49:15 +02:00
Jerome Charaoui
d0e9b3d640
Add disableContentUpdate import option
This commit also decouples the "import" and "update" functions inside
ContentProxy. If a content array is available, it must be passed to the
new importEntry method.
2017-06-01 09:48:14 +02:00