Commit graph

61 commits

Author SHA1 Message Date
Kevin Decherf
b49c87acf1 ignoreOriginUrl: add initial support of ignore lists
Add the ability to specify hosts and patterns lists to ignore the given
entry url and replace it with the fetched content url without touching
to origin_url.

This initial support should be reworked in the following months to move
the hardcoded ignore lists in the database.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:42:09 +02:00
Kevin Decherf
fc040c749d updateOriginUrl: add behavior when diff is fragment and query
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:08:58 +02:00
Kevin Decherf
e07fadea76 Refactor updateOriginUrl to include new behaviors behaviors
- Leave origin_url unchanged if difference is an ending slash
- Leave origin_url unchanged if difference is scheme
- Ignore (noop) if difference is query string or fragment

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:01:16 +02:00
Kevin Decherf
781864b954 ContentProxy: swap entry url to origin_url and set new url according to graby content
Closes #3529

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-21 16:15:31 +02:00
Tobi823
28cc645b93 Run php-cs-fixer for fixing coding standard issues (on ContentProxyTest) 2018-09-23 23:42:05 +02:00
Tobi823
d64139d812 Make helper methods strToHex and hexToStr in ContentProxyTest.php private
to prevent misusage (from outside this class)
2018-09-21 13:31:28 +02:00
Tobi823
c01d953292 Add tests for logic
Try to translate the title of a PDF from UTF-8 (then UTF-16BE, then WINDOWS-1252) to UTF-8
2018-09-21 13:15:00 +02:00
Simounet
e6f12c0734 More robust srcset image attribute handling
Linked to HTMLawed PR https://github.com/kesar/HTMLawed/pull/17
2018-07-12 14:29:30 +02:00
Simounet
3fbbe0d9f1 Fix image downloading on null image path 2018-07-05 11:40:51 +02:00
Jeremy Benoist
778543311f
Fix tests 2018-06-06 17:34:20 +02:00
Simounet
c15bb5ad72 Fix srcset attribute on images downloaded 2018-06-01 13:49:16 +02:00
Jeremy Benoist
5661e8d42a
Fix countable in tests 2017-12-18 13:29:34 +01:00
Jeremy Benoist
bd91bd5c32
Use namespaced PHPUnit classes 2017-12-18 13:29:33 +01:00
Jeremy Benoist
709e21a3f4
Define storeArticleHeaders false by default
Fix tests which must use `$storeArticleHeaders`.
Fix CS
2017-11-21 10:37:36 +01:00
Nicolas Lœuillet
8a21985474 Added internal setting to enable/disable headers storage 2017-11-20 18:47:48 +01:00
Jeremy Benoist
c18a2476b6
CS 2017-07-03 13:56:39 +02:00
Jeremy Benoist
d0ec2ddd23
Fix validateAndSetPreviewPicture
Which wasn't covered by a test!
2017-07-03 13:45:04 +02:00
Jeremy Benoist
a05b61159e
Fix PATCH method
The PATCH method for the entry should only update what user sent to us and not the whole entry as it was before.
Also, sending tags when patching an entry will now remove all current tags & assocatied new ones.
2017-07-03 13:45:04 +02:00
Jeremy Benoist
38520658ad
Fix tests 2017-07-03 07:30:54 +02:00
Jeremy Benoist
f808b01692
Add a real configuration for CS-Fixer 2017-07-01 09:52:38 +02:00
Jérémy Benoist
80784b782b Merge pull request #2683 from wallabag/credentials-in-db
Store credentials in DB
2017-06-20 16:40:48 +02:00
Jeremy Benoist
906424c1b6
Crypt site credential password 2017-06-20 16:03:35 +02:00
Thomas Citharel
41d45c6122 Fix empty language and preview pics 2017-06-12 16:46:33 +02:00
Jeremy Benoist
0d349ea670
Validate language & preview picture fields
Instead of saving the value of each field right into the content without any validation, it seems better to validate them.
This might sounds obvious now we say that.
2017-06-08 21:51:46 +02:00
Jeremy Benoist
577c0b6dd8
Use an alternative way to detect image
When parsing content to retrieve images to save locally, we only check for the content-type of the image response.
In some case, that value is empty.
Now we’re also checking for the first few bytes of the content as an alternative to detect if it’s an image wallabag can handle.
We might get higher image supports using that alternative method.
2017-06-05 22:54:02 +02:00
Jérémy Benoist
a687c8d915 Merge pull request #2708 from jcharaoui/import-disablecontentupdate
Import disableContentUpdate
2017-06-02 11:26:37 +02:00
Jeremy Benoist
fcad69a427
Replace images with &
Images with `&` in the path weren’t well replaced because they might be with `&amp;` in the html instead.

Replacing `&` with `&amp;` fix the problem.
2017-06-01 22:50:33 +02:00
Jeremy Benoist
6acadf8e98
Rewrote code & fix tests 2017-06-01 11:31:45 +02:00
Jeremy Benoist
d5c2cc54b5
Fix tests 2017-06-01 09:49:15 +02:00
Jerome Charaoui
d0e9b3d640
Add disableContentUpdate import option
This commit also decouples the "import" and "update" functions inside
ContentProxy. If a content array is available, it must be passed to the
new importEntry method.
2017-06-01 09:48:14 +02:00
Jerome Charaoui
7aba665e48
Avoid returning objects passed by reference.
Objects are always passed by reference, so it doesn't make sense to
return an object which is passed by reference as it will always be the
same object. This change makes the code a bit more readable.
2017-06-01 09:43:01 +02:00
Jeremy Benoist
695af58857
Fix tests following rebase 2017-05-31 14:00:15 +02:00
Jeremy Benoist
38a04dee34
CS 2017-05-31 14:00:15 +02:00
Jeremy Benoist
f0378b4d7c
Forced date can now be a timestamp too
Add adding more tests for forced content
2017-05-31 14:00:15 +02:00
Jeremy Benoist
0d6cfb884c
Remove htmlawed and use graby instead
Instead of using htmlawed (which is already used in graby) use graby directly (which require some refacto on graby side).
Still needs some tests.
2017-05-31 14:00:15 +02:00
Jeremy Benoist
74a75f7d43
Use graby ContentExtractor to clean html
It might be better to re-use some graby functionalities to clean html instead of building a new system.
2017-05-31 14:00:15 +02:00
Nicolas Lœuillet
4423b88c5b Merge pull request #3168 from wallabag/instapaper-tags-import
Add support for tag in Instapaper import
2017-05-31 12:32:09 +02:00
Jeremy Benoist
bad7df8c00
CS & improve tags assertions 2017-05-31 10:38:15 +02:00
Kevin Decherf
5dbf3f2326 TagController: ignore ActionMarkAsRead when removing tag from entry
Fixes #2835

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-05-31 00:36:46 +02:00
Jeremy Benoist
d181bd7285
Merge remote-tracking branch 'origin/master' into 2.3 2017-05-30 09:59:06 +02:00
Thomas Citharel
de8d2a9005 CS
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2017-05-28 12:59:48 +02:00
Thomas Citharel
6bc6fb1f60 Move Tags assigner to a separate file
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2017-05-27 22:08:14 +02:00
Damien Cassou
ac1509a66b
Fix documentation URL for "How can I help to fix that?" 2017-05-24 12:57:46 +02:00
Nicolas Lœuillet
a1146b6551
Added tests on logs records 2017-05-12 15:01:18 +02:00
Nicolas Lœuillet
3554364bed
Fixed tests 2017-05-12 13:47:53 +02:00
Nicolas Lœuillet
d09fe4d233
Added test for deduplication 2017-05-05 14:33:36 +02:00
Jeremy Benoist
3d71d40349
Avoid false preview image
If the website doesn't provide an og_image, the value will be false and so it'll be saved like that in the database.
We prefer to leave it as null instead of false.
2017-01-10 17:42:36 +01:00
Jerome Charaoui
a2c1b94e82 Revert switch to KernelTestCase for ContentProxyTest
Define the error string manually inside the test class instead of
fetching it from app config.
2016-12-03 09:50:36 -05:00
Jerome Charaoui
fc2b7bda53 Fix ContentProxy tests 2016-12-03 00:14:06 -05:00
Nicolas Lœuillet
10b3509757
Added http_status in Entry entity 2016-11-18 15:09:21 +01:00