Commit graph

76 commits

Author SHA1 Message Date
Kevin Decherf
2dbb5b2307 Enable no-referrer on img tags, enable strict-origin-when-cross-origin by default
Fixes #3889

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-05-10 23:07:26 +02:00
Jeremy Benoist
8ca858ee73
Fix PHP warning
Looks like sometimes (usually from import) the `html` key isn’t available.
2019-03-18 06:23:41 +01:00
Kevin Decherf
1b220426e2 phpcs
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-24 22:33:32 +02:00
Kevin Decherf
6059967951 updateOriginUrl: remove 'query string' case from ignore list
Two urls with a different query string may refer to two different pages
so keep them both.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-24 22:27:27 +02:00
Kevin Decherf
44e63667d9 updateOriginUrl: add comment blocks for the parse_url diff check
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-24 22:13:03 +02:00
Kevin Decherf
5ba5e22a09 updateOriginUrl: rewrite some if, resolving feedbacks from PR
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-24 21:54:09 +02:00
Kevin Decherf
b49c87acf1 ignoreOriginUrl: add initial support of ignore lists
Add the ability to specify hosts and patterns lists to ignore the given
entry url and replace it with the fetched content url without touching
to origin_url.

This initial support should be reworked in the following months to move
the hardcoded ignore lists in the database.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:42:09 +02:00
Kevin Decherf
fc040c749d updateOriginUrl: add behavior when diff is fragment and query
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:08:58 +02:00
Kevin Decherf
e07fadea76 Refactor updateOriginUrl to include new behaviors behaviors
- Leave origin_url unchanged if difference is an ending slash
- Leave origin_url unchanged if difference is scheme
- Ignore (noop) if difference is query string or fragment

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:01:16 +02:00
Kevin Decherf
781864b954 ContentProxy: swap entry url to origin_url and set new url according to graby content
Closes #3529

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-21 16:15:31 +02:00
Kevin Decherf
4a81360efc ContentProxy: fix a corner case when entry.url is empty in updateEntry
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-21 16:13:20 +02:00
Tobi823
83f1c3274f Run php-cs-fixer for fixing coding standard issues 2018-09-23 22:20:43 +02:00
Tobi823
7a65c2017b Override the value of the given parameter ($title) with the (hopefully)
correct (to UTF-8) converted PDF title
2018-09-21 13:23:39 +02:00
Tobi823
c01d953292 Add tests for logic
Try to translate the title of a PDF from UTF-8 (then UTF-16BE, then WINDOWS-1252) to UTF-8
2018-09-21 13:15:00 +02:00
Tobi823
f80f16dfc8 Try to detect the character encoding in PDFs and try to translate
the title from the PDF to UTF-8
2018-09-21 13:15:00 +02:00
Tobi823
8648f0c005 Remove type declaration for PHP 5 compatibility 2018-09-21 13:15:00 +02:00
Tobi823
d76a5a6d60 Bugfix: Sanitize the title of a saved webpage from invalid UTF-8 characters 2018-09-21 13:15:00 +02:00
Kevin Decherf
2a1ceb67b4 php-cs-fixer
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-09-05 14:25:32 +02:00
Kevin Decherf
af29e1bf07 Fix empty title and domain_name when exception is thrown during fetch
Add a new helper to set a default title when it's empty:
1/ use basename part of entry's path, if any
2/ or use domain name

Fixes #2053

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-12-13 22:44:31 +01:00
Jeremy Benoist
709e21a3f4
Define storeArticleHeaders false by default
Fix tests which must use `$storeArticleHeaders`.
Fix CS
2017-11-21 10:37:36 +01:00
Nicolas Lœuillet
8a21985474 Added internal setting to enable/disable headers storage 2017-11-20 18:47:48 +01:00
Jeremy Benoist
3ef055ced3
CS 2017-10-09 16:47:15 +02:00
Jeremy Benoist
ff9f89fd23
Add a test for updatePublishedAt
To avoid error when a content is re-submitted and it previously add a
published date.

Also, fix the `testPostSameEntry`
2017-07-24 17:07:47 +02:00
Simounet
b236d3f627
Fix updatePublishedAt on already parsed article's date 2017-07-24 16:39:07 +02:00
Jeremy Benoist
c18a2476b6
CS 2017-07-03 13:56:39 +02:00
Jeremy Benoist
d0ec2ddd23
Fix validateAndSetPreviewPicture
Which wasn't covered by a test!
2017-07-03 13:45:04 +02:00
Jeremy Benoist
a05b61159e
Fix PATCH method
The PATCH method for the entry should only update what user sent to us and not the whole entry as it was before.
Also, sending tags when patching an entry will now remove all current tags & assocatied new ones.
2017-07-03 13:45:04 +02:00
Jeremy Benoist
f808b01692
Add a real configuration for CS-Fixer 2017-07-01 09:52:38 +02:00
Thomas Citharel
41d45c6122 Fix empty language and preview pics 2017-06-12 16:46:33 +02:00
Jeremy Benoist
80e49ba7b0
Convert - to _ in language
Mostly to increase language supports
2017-06-09 11:42:09 +02:00
Jeremy Benoist
42f3bb2c63
Use Locale instead of Language 2017-06-09 11:28:04 +02:00
Jeremy Benoist
be54dfe4e6
CS 2017-06-08 21:56:20 +02:00
Jeremy Benoist
0d349ea670
Validate language & preview picture fields
Instead of saving the value of each field right into the content without any validation, it seems better to validate them.
This might sounds obvious now we say that.
2017-06-08 21:51:46 +02:00
Jeremy Benoist
645291e8fe
Add ability to patch an entry with more fields
Like when we create an entry, we can now patch an entry with new fields:
- content
- language
- preview_picture
- published_at
- authors
2017-06-02 20:52:49 +02:00
Jeremy Benoist
ec97072152
No need to catch that Exception 2017-06-01 11:45:02 +02:00
Jeremy Benoist
6acadf8e98
Rewrote code & fix tests 2017-06-01 11:31:45 +02:00
Jeremy Benoist
843182c7cf
CS 2017-06-01 09:52:09 +02:00
Jeremy Benoist
d5c2cc54b5
Fix tests 2017-06-01 09:49:15 +02:00
Jerome Charaoui
d0e9b3d640
Add disableContentUpdate import option
This commit also decouples the "import" and "update" functions inside
ContentProxy. If a content array is available, it must be passed to the
new importEntry method.
2017-06-01 09:48:14 +02:00
Jerome Charaoui
7aba665e48
Avoid returning objects passed by reference.
Objects are always passed by reference, so it doesn't make sense to
return an object which is passed by reference as it will always be the
same object. This change makes the code a bit more readable.
2017-06-01 09:43:01 +02:00
Jeremy Benoist
f0378b4d7c
Forced date can now be a timestamp too
Add adding more tests for forced content
2017-05-31 14:00:15 +02:00
Jeremy Benoist
9e349f08a6
Improve docs 2017-05-31 14:00:15 +02:00
Jeremy Benoist
0d6cfb884c
Remove htmlawed and use graby instead
Instead of using htmlawed (which is already used in graby) use graby directly (which require some refacto on graby side).
Still needs some tests.
2017-05-31 14:00:15 +02:00
Jeremy Benoist
74a75f7d43
Use graby ContentExtractor to clean html
It might be better to re-use some graby functionalities to clean html instead of building a new system.
2017-05-31 14:00:15 +02:00
Jeremy Benoist
e668a8124c
Allow other fields to be send using API
Entry API can now have these new fields:
- content
- language
- preview_picture
- published_at

Re-use the ContentProxy to be able to do the same using the web UI (in the future).
htmLawed is used to clean stuff from content, I hope it’ll be enough to avoid security breach.

Lower content validation when we want to update an entry with content already defined. Before, language & content_type were required. If there weren’t provided, we re-fetched the content using graby. I think these fields aren’t required for an entry to be created. So I removed them.
Which means some import from the v1 export won’t be re-fetched since they provide content, url & title.

Also, remove liberation link from Readability import to avoid overlaping import (from wallabag v1, which had the same link)
2017-05-31 13:59:45 +02:00
Thomas Citharel
5d3deafd3e CS
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2017-05-28 01:16:01 +02:00
Thomas Citharel
6bc6fb1f60 Move Tags assigner to a separate file
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2017-05-27 22:08:14 +02:00
Nicolas Lœuillet
dda6a6addc
Added headers field in Entry 2017-05-11 14:18:21 +02:00
Nicolas Lœuillet
7b0b3622ab Added author of article 2017-04-09 15:24:51 +02:00
Nicolas Lœuillet
5e9009ce86 Added publication date 2017-04-05 22:22:52 +02:00