Merge pull request #2255 from wallabag/readability

Add Readability import
This commit is contained in:
Jeremy Benoist 2016-09-01 14:18:48 +02:00 committed by GitHub
commit bd8c13e8b0
24 changed files with 716 additions and 17 deletions

View file

@ -30,14 +30,25 @@ You need to authorize wallabag to interact with your Pocket account.
Your data will be imported. Data import can be a demanding process for your server
(we need to work on this import to improve it).
From Instapaper
---------------
*Feature not yet implemented in wallabag v2.*
From Readability
----------------
Export your Readability data
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On the tools (`https://www.readability.com/tools/<https://www.readability.com/tools/>`_) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).
Import your data into wallabag 2.x
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Click on ``Import`` link in the menu, on ``Import contents`` in Readability section
and then select your json file and upload it.
Your data will be imported. Data import can be a demanding process for your server (we need to work on this import to improve it).
From Instapaper
---------------
*Feature not yet implemented in wallabag v2.*
From HTML or JSON file

View file

@ -30,15 +30,27 @@ Vous devez autoriser wallabag à se connecter à votre compte Pocket.
Vos données vont être importées. L'import de données est une action qui peut être couteuse
pour votre serveur (nous devons encore travailler pour améliorer cet import).
Depuis Readability
------------------
Exportez vos données de Readability
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sur la page des outils (`https://www.readability.com/tools/<https://www.readability.com/tools/>`_), cliquez sur "Export your data" dans la section "Data Export". Vous allez recevoir un email avec un lien pour télécharger le json.
Importez vos données dans wallabag 2.x
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Cliquez sur le lien ``Importer`` dans le menu, sur ``Importer les contenus`` dans
la section Readability et ensuite sélectionnez votre fichier json pour l'uploader.
Vos données vont être importées. L'import de données est une action qui peut être couteuse pour votre serveur (nous devons encore travailler pour améliorer cet import).
Depuis Instapaper
-----------------
*Fonctionnalité pas encore implémentée dans wallabag v2.*
Depuis Readability
------------------
*Fonctionnalité pas encore implémentée dans wallabag v2.*
Depuis un fichier HTML ou JSON
------------------------------

View file

@ -292,6 +292,7 @@ quickstart:
# pocket: 'Migrate from Pocket'
# wallabag_v1: 'Migrate from wallabag v1'
# wallabag_v2: 'Migrate from wallabag v2'
# readability: 'Migrate from Readability'
# developer:
# title: 'Developers'
# create_application: 'Create your third application'
@ -341,6 +342,10 @@ import:
# wallabag_v2:
# page_title: 'Import > Wallabag v2'
# description: 'This importer will import all your wallabag v2 articles. Go to All articles, then, on the export sidebar, click on "JSON". You will have a "All articles.json" file.'
# readability:
# page_title: 'Import > Readability'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
# page_title: 'Developer'

View file

@ -292,6 +292,7 @@ quickstart:
pocket: 'von Pocket migrieren'
wallabag_v1: 'von wallabag v1 migrieren'
wallabag_v2: 'von wallabag v2 migrieren'
readability: 'von Readability migrieren'
developer:
title: 'Entwickler'
create_application: 'Erstelle eine Anwendung und nutze die wallabag API'
@ -341,6 +342,10 @@ import:
wallabag_v2:
page_title: 'Aus wallabag v2 importieren'
description: 'Dieser Import wird all deine Artikel aus wallabag v2 importieren. Gehe auf "Alle Artikel" und dann, in der Exportieren-Seitenleiste auf "JSON". Dabei erhältst du eine "All articles.json"-Datei.'
readability:
page_title: 'Aus Readability importieren'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
page_title: 'Entwickler'

View file

@ -292,6 +292,7 @@ quickstart:
pocket: 'Migrate from Pocket'
wallabag_v1: 'Migrate from wallabag v1'
wallabag_v2: 'Migrate from wallabag v2'
readability: 'Migrate from Readability'
developer:
title: 'Developers'
create_application: 'Create your third application'
@ -341,6 +342,10 @@ import:
wallabag_v2:
page_title: 'Import > Wallabag v2'
description: 'This importer will import all your wallabag v2 articles. Go to All articles, then, on the export sidebar, click on "JSON". You will have a "All articles.json" file.'
readability:
page_title: 'Import > Readability'
description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
page_title: 'Developer'

View file

@ -292,6 +292,7 @@ quickstart:
pocket: 'Migrar desde Pocket'
wallabag_v1: 'Migrar desde wallabag v1'
wallabag_v2: 'Migrar desde wallabag v2'
readability: 'Migrar desde Readability'
developer:
title: 'Promotores'
create_application: 'Cree su tercera aplicación'
@ -341,6 +342,10 @@ import:
wallabag_v2:
page_title: 'Importar > Wallabag v2'
description: 'Va a importar sus artículos de otra instancia de wallabag v2. Vaya a Todos los artículos, entonces, en la barra lateral, oprima en "JSON". Usted tendrá un fichero "All articles.json"'
readability:
page_title: 'Importar > Readability'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
page_title: 'Promotor'

View file

@ -292,6 +292,7 @@ quickstart:
pocket: 'مهاجرت از Pocket'
wallabag_v1: 'مهاجرت از نسخهٔ یکم wallabag'
wallabag_v2: 'مهاجرت از نسخهٔ دوم wallabag'
readability: 'مهاجرت از نسخهٔ دوم Readability'
developer:
title: 'برنامه‌نویسان'
create_application: 'برنامهٔ wallabag خود را بسازید'
@ -316,16 +317,16 @@ tag:
# see_untagged_entries: 'See untagged entries'
import:
page_title: 'درون‌ریزی'
page_description: 'به درون‌ریز wallabag خوش آمدید. لطفاً سرویس قبلی خود را که می‌خواهید از آن مهاجرت کنید انتخاب کنید.'
action:
page_title: 'درون‌ریزی'
page_description: 'به درون‌ریز wallabag خوش آمدید. لطفاً سرویس قبلی خود را که می‌خواهید از آن مهاجرت کنید انتخاب کنید.'
action:
import_contents: 'درون‌ریزی مقاله‌ها'
form:
form:
mark_as_read_title: 'علامت‌زدن همه به عنوان خوانده‌شده؟'
mark_as_read_label: 'همهٔ مقاله‌های درون‌ریزی شده را به عنوان خوانده‌شده علامت بزن'
file_label: 'پرونده'
save_label: 'بارگذاری پرونده'
pocket:
pocket:
page_title: 'درون‌ریزی > Pocket'
description: "این برنامه همهٔ داده‌های Pocket شما را درون‌ریزی می‌کند. سرویس Pocket اجازه نمی‌دهد که متن مقاله‌ها را درون‌ریزی کنیم، بنابراین wallabag متن مقاله‌ها را دوباره از اینترنت دریافت می‌کند."
config_missing:
@ -334,13 +335,17 @@ import:
user_message: 'مدیر سرور شما باید یک API Key برای Pocket تعریف کند.'
authorize_message: 'شما می‌توانید داده‌هایتان را از حساب Pocket خود درون‌ریزی کنید. روی دکمهٔ زیر کلیک کنید و به برنامه اجازه دهید تا به getpocket.com وصل شود.'
connect_to_pocket: 'به Pocket وصل شو و داده‌ها را دریافت کن'
wallabag_v1:
wallabag_v1:
page_title: 'درون‌ریزی > Wallabag v1'
description: 'این برنامه همهٔ داده‌های شما را در نسخهٔ ۱ wallabag درون‌ریزی می‌کند. در صفحهٔ تنظیمات، روی "JSON export" در بخش "Export your wallabag data" کلیک کنید. با این کار شما پرونده‌ای به شکل "wallabag-export-1-xxxx-xx-xx.json" دریافت خواهید کرد.'
how_to: 'لطفاً پرونده را انتخاب کنید و روی دکمهٔ زیر کلیک کنید تا بارگذاری و درون‌ریزی شود.'
wallabag_v2:
wallabag_v2:
page_title: 'درون‌ریزی > Wallabag v2'
description: 'این برنامه همهٔ داده‌های شما را در نسخهٔ ۲ wallabag درون‌ریزی می‌کند. به بخش «همهٔ مقاله‌ها» بروید و در بخش «برون‌ریزی» روی "JSON" کلیک کنید. با این کار شما پرونده‌ای به شکل "All articles.json" دریافت خواهید کرد.'
readability:
page_title: 'درون‌ریزی > Readability'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
# page_title: 'Developer'

View file

@ -292,6 +292,7 @@ quickstart:
pocket: 'Migrer depuis Pocket'
wallabag_v1: 'Migrer depuis wallabag v1'
wallabag_v2: 'Migrer depuis wallabag v2'
readability: 'Migrer depuis Readability'
developer:
title: 'Pour les développeurs'
create_application: 'Créer votre application tierce'
@ -341,6 +342,10 @@ import:
wallabag_v2:
page_title: 'Importer > Wallabag v2'
description: "Cet outil va importer tous vos articles d'une autre instance de wallabag v2. Allez dans tous vos articles, puis, sur la barre latérale, cliquez sur \"JSON\". Vous allez récupérer un fichier \"All articles.json\""
readability:
page_title: 'Importer > Readability'
description: 'Cet outil va importer toutes vos données de Readability. Sur la page des outils (https://www.readability.com/tools/), cliquez sur "Export your data" dans la section "Data Export". Vous allez recevoir un email avec un lien pour télécharger le json.'
how_to: "Choisissez le fichier de votre export Readability et cliquez sur le bouton ci-dessous pour l'importer."
developer:
page_title: 'Développeur'

View file

@ -291,6 +291,7 @@ quickstart:
pocket: 'Trasferisci da Pocket'
wallabag_v1: 'Trasferisci da wallabag v1'
wallabag_v2: 'Trasferisci da wallabag v2'
readability: 'Trasferisci da Readability'
developer:
title: 'Sviluppatori'
create_application: 'Crea la tua applicazione'
@ -340,6 +341,10 @@ import:
wallabag_v2:
page_title: 'Importa da > Wallabag v2'
description: 'Questo importatore copierà tutti i tuoi dati da un wallabag v2. Vai in "Tutti i contenuti", e, nella sidebar di esportazione, clicca su "JSON". Otterrai un file "Tutti i contenuti.json".'
readability:
page_title: 'Importa da > Readability'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
page_title: 'Sviluppatori'

View file

@ -292,6 +292,7 @@ quickstart:
pocket: 'Migrar dempuèi Pocket'
wallabag_v1: 'Migrar dempuèi wallabag v1'
wallabag_v2: 'Migrar dempuèi wallabag v2'
readability: 'Migrar dempuèi Readability'
developer:
title: 'Pels desvolopadors'
create_application: 'Crear vòstra aplicacion tèrça'
@ -341,6 +342,10 @@ import:
wallabag_v2:
page_title: 'Importer > Wallabag v2'
description: "Aquesta aisina importarà totas vòstras donadas d'una instància mai de wallabag v2. Anatz dins totes vòstres articles, puèi, sus la barra laterala, clicatz sus \"JSON\". Traparatz un fichièr \"All articles.json\""
readability:
page_title: 'Importer > Readability'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
page_title: 'Desvolopador'

View file

@ -292,6 +292,7 @@ quickstart:
pocket: 'Migruj z Pocket'
wallabag_v1: 'Migruj z wallabag v1'
wallabag_v2: 'Migruj z wallabag v2'
readability: 'Migruj z Readability'
developer:
title: 'Deweloperzy'
create_application: 'Stwórz swoją aplikację'
@ -341,6 +342,10 @@ import:
wallabag_v2:
page_title: 'Import > Wallabag v2'
description: 'Ten importer, zaimportuje wszystkie twoje artykułu z wallabag v2. Idź do wszystkich artykułów, a następnie na panelu exportu kliknij na "JSON". Otrzymasz plik "All articles.json".'
readability:
page_title: 'Import > Readability'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
page_title: 'Deweloper'

View file

@ -292,6 +292,7 @@ quickstart:
# pocket: 'Migrate from Pocket'
# wallabag_v1: 'Migrate from wallabag v1'
# wallabag_v2: 'Migrate from wallabag v2'
# readability: 'Migrate from Readability'
# developer:
# title: 'Developers'
# create_application: 'Create your third application'
@ -341,6 +342,10 @@ import:
# wallabag_v2:
# page_title: 'Import > Wallabag v2'
# description: 'This importer will import all your wallabag v2 articles. Go to All articles, then, on the export sidebar, click on "JSON". You will have a "All articles.json" file.'
# readability:
# page_title: 'Import > Readability'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
# page_title: 'Developer'

View file

@ -292,6 +292,7 @@ quickstart:
pocket: "Pocket üzerindeki verilerinizi wallabag'e aktarın"
wallabag_v1: "wallabag v1 üzerindeki verilerinizi wallabag'in yeni sürümüne aktarın"
wallabag_v2: "wallabag v2 üzerindeki verilerinizi wallabag'in yeni sürümüne aktarın"
readability: "Readability üzerindeki verilerinizi wallabag'e aktarın'"
developer:
# title: 'Developers'
# create_application: 'Create your third application'
@ -341,6 +342,10 @@ import:
wallabag_v2:
page_title: 'İçe Aktar > Wallabag v2'
# description: 'This importer will import all your wallabag v2 articles. Go to All articles, then, on the export sidebar, click on "JSON". You will have a "All articles.json" file.'
readability:
page_title: 'İçe Aktar > Readability'
# description: 'This importer will import all your Readability articles. On the tools (https://www.readability.com/tools/) page, click on "Export your data" in the "Data Export" section. You will received an email to download a json (which does not end with .json in fact).'
# how_to: 'Please select your Readability export and click on the below button to upload and import it.'
developer:
# page_title: 'Developer'

View file

@ -39,6 +39,7 @@
<li><a href="{{ path('import_pocket') }}">{{ 'quickstart.migrate.pocket'|trans }}</a></li>
<li><a href="{{ path('import_wallabag_v1') }}">{{ 'quickstart.migrate.wallabag_v1'|trans }}</a></li>
<li><a href="{{ path('import_wallabag_v2') }}">{{ 'quickstart.migrate.wallabag_v2'|trans }}</a></li>
<li><a href="{{ path('import_readability') }}">{{ 'quickstart.migrate.readability'|trans }}</a></li>
</ul>
<h4>{{ 'quickstart.developer.title'|trans }}</h4>

View file

@ -44,6 +44,7 @@
<li><a href="{{ path('import_pocket') }}">{{ 'quickstart.migrate.pocket'|trans }}</a></li>
<li><a href="{{ path('import_wallabag_v1') }}">{{ 'quickstart.migrate.wallabag_v1'|trans }}</a></li>
<li><a href="{{ path('import_wallabag_v2') }}">{{ 'quickstart.migrate.wallabag_v2'|trans }}</a></li>
<li><a href="{{ path('import_readability') }}">{{ 'quickstart.migrate.readability'|trans }}</a></li>
</ul>
<h4>{{ 'quickstart.developer.title'|trans }}</h4>

View file

@ -0,0 +1,65 @@
<?php
namespace Wallabag\ImportBundle\Controller;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;
use Symfony\Component\HttpFoundation\Request;
use Wallabag\ImportBundle\Form\Type\UploadImportType;
class ReadabilityController extends Controller
{
/**
* @Route("/readability", name="import_readability")
*/
public function indexAction(Request $request)
{
$form = $this->createForm(UploadImportType::class);
$form->handleRequest($request);
$readability = $this->get('wallabag_import.readability.import');
if ($form->isValid()) {
$file = $form->get('file')->getData();
$markAsRead = $form->get('mark_as_read')->getData();
$name = 'readability_'.$this->getUser()->getId().'.json';
if (in_array($file->getClientMimeType(), $this->getParameter('wallabag_import.allow_mimetypes')) && $file->move($this->getParameter('wallabag_import.resource_dir'), $name)) {
$res = $readability
->setUser($this->getUser())
->setFilepath($this->getParameter('wallabag_import.resource_dir').'/'.$name)
->setMarkAsRead($markAsRead)
->import();
$message = 'flashes.import.notice.failed';
if (true === $res) {
$summary = $readability->getSummary();
$message = $this->get('translator')->trans('flashes.import.notice.summary', [
'%imported%' => $summary['imported'],
'%skipped%' => $summary['skipped'],
]);
unlink($this->getParameter('wallabag_import.resource_dir').'/'.$name);
}
$this->get('session')->getFlashBag()->add(
'notice',
$message
);
return $this->redirect($this->generateUrl('homepage'));
} else {
$this->get('session')->getFlashBag()->add(
'notice',
'flashes.import.notice.failed_on_file'
);
}
}
return $this->render('WallabagImportBundle:Readability:index.html.twig', [
'form' => $form->createView(),
'import' => $readability,
]);
}
}

View file

@ -0,0 +1,179 @@
<?php
namespace Wallabag\ImportBundle\Import;
use Wallabag\CoreBundle\Entity\Entry;
use Wallabag\UserBundle\Entity\User;
class ReadabilityImport extends AbstractImport
{
private $user;
private $skippedEntries = 0;
private $importedEntries = 0;
private $filepath;
private $markAsRead;
/**
* We define the user in a custom call because on the import command there is no logged in user.
* So we can't retrieve user from the `security.token_storage` service.
*
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
return $this;
}
/**
* {@inheritdoc}
*/
public function getName()
{
return 'Readability';
}
/**
* {@inheritdoc}
*/
public function getUrl()
{
return 'import_readability';
}
/**
* {@inheritdoc}
*/
public function getDescription()
{
return 'import.readability.description';
}
/**
* Set file path to the json file.
*
* @param string $filepath
*/
public function setFilepath($filepath)
{
$this->filepath = $filepath;
return $this;
}
/**
* Set whether articles must be all marked as read.
*
* @param bool $markAsRead
*/
public function setMarkAsRead($markAsRead)
{
$this->markAsRead = $markAsRead;
return $this;
}
/**
* Get whether articles must be all marked as read.
*/
public function getMarkAsRead()
{
return $this->markAsRead;
}
/**
* {@inheritdoc}
*/
public function getSummary()
{
return [
'skipped' => $this->skippedEntries,
'imported' => $this->importedEntries,
];
}
/**
* {@inheritdoc}
*/
public function import()
{
if (!$this->user) {
$this->logger->error('ReadabilityImport: user is not defined');
return false;
}
if (!file_exists($this->filepath) || !is_readable($this->filepath)) {
$this->logger->error('ReadabilityImport: unable to read file', ['filepath' => $this->filepath]);
return false;
}
$data = json_decode(file_get_contents($this->filepath), true);
if (empty($data) || empty($data['bookmarks'])) {
return false;
}
$this->parseEntries($data['bookmarks']);
return true;
}
/**
* Parse and insert all given entries.
*
* @param $entries
*/
protected function parseEntries($entries)
{
$i = 1;
foreach ($entries as $importedEntry) {
$existingEntry = $this->em
->getRepository('WallabagCoreBundle:Entry')
->findByUrlAndUserId($importedEntry['article__url'], $this->user->getId());
if (false !== $existingEntry) {
++$this->skippedEntries;
continue;
}
$data = [
'title' => $importedEntry['article__title'],
'url' => $importedEntry['article__url'],
'content_type' => '',
'language' => '',
'is_archived' => $importedEntry['archive'] || $this->markAsRead,
'is_starred' => $importedEntry['favorite'],
];
$entry = $this->fetchContent(
new Entry($this->user),
$data['url'],
$data
);
// jump to next entry in case of problem while getting content
if (false === $entry) {
++$this->skippedEntries;
continue;
}
$entry->setArchived($data['is_archived']);
$entry->setStarred($data['is_starred']);
$this->em->persist($entry);
++$this->importedEntries;
// flush every 20 entries
if (($i % 20) === 0) {
$this->em->flush();
$this->em->clear($entry);
}
++$i;
}
$this->em->flush();
}
}

View file

@ -43,3 +43,13 @@ services:
- [ setLogger, [ "@logger" ]]
tags:
- { name: wallabag_import.import, alias: wallabag_v2 }
wallabag_import.readability.import:
class: Wallabag\ImportBundle\Import\ReadabilityImport
arguments:
- "@doctrine.orm.entity_manager"
- "@wallabag_core.content_proxy"
calls:
- [ setLogger, [ "@logger" ]]
tags:
- { name: wallabag_import.import, alias: readability }

View file

@ -0,0 +1,43 @@
{% extends "WallabagCoreBundle::layout.html.twig" %}
{% block title %}{{ 'import.readability.page_title'|trans }}{% endblock %}
{% block content %}
<div class="row">
<div class="col s12">
<div class="card-panel settings">
<div class="row">
<blockquote>{{ import.description|trans }}</blockquote>
<p>{{ 'import.readability.how_to'|trans }}</p>
<div class="col s12">
{{ form_start(form, {'method': 'POST'}) }}
{{ form_errors(form) }}
<div class="row">
<div class="file-field input-field col s12">
{{ form_errors(form.file) }}
<div class="btn">
<span>{{ form.file.vars.label|trans }}</span>
{{ form_widget(form.file) }}
</div>
<div class="file-path-wrapper">
<input class="file-path validate" type="text">
</div>
</div>
<div class="input-field col s6 with-checkbox">
<h6>{{ 'import.form.mark_as_read_title'|trans }}</h6>
{{ form_widget(form.mark_as_read) }}
{{ form_label(form.mark_as_read) }}
</div>
</div>
{{ form_widget(form.save, { 'attr': {'class': 'btn waves-effect waves-light'} }) }}
{{ form_rest(form) }}
</form>
</div>
</div>
</div>
</div>
</div>
{% endblock %}

View file

@ -24,6 +24,6 @@ class ImportControllerTest extends WallabagCoreTestCase
$crawler = $client->request('GET', '/import/');
$this->assertEquals(200, $client->getResponse()->getStatusCode());
$this->assertEquals(3, $crawler->filter('blockquote')->count());
$this->assertEquals(4, $crawler->filter('blockquote')->count());
}
}

View file

@ -0,0 +1,122 @@
<?php
namespace Tests\Wallabag\ImportBundle\Controller;
use Tests\Wallabag\CoreBundle\WallabagCoreTestCase;
use Symfony\Component\HttpFoundation\File\UploadedFile;
class ReadabilityControllerTest extends WallabagCoreTestCase
{
public function testImportReadability()
{
$this->logInAs('admin');
$client = $this->getClient();
$crawler = $client->request('GET', '/import/readability');
$this->assertEquals(200, $client->getResponse()->getStatusCode());
$this->assertEquals(1, $crawler->filter('form[name=upload_import_file] > button[type=submit]')->count());
$this->assertEquals(1, $crawler->filter('input[type=file]')->count());
}
public function testImportReadabilityWithFile()
{
$this->logInAs('admin');
$client = $this->getClient();
$crawler = $client->request('GET', '/import/readability');
$form = $crawler->filter('form[name=upload_import_file] > button[type=submit]')->form();
$file = new UploadedFile(__DIR__.'/../fixtures/readability.json', 'readability.json');
$data = [
'upload_import_file[file]' => $file,
];
$client->submit($form, $data);
$this->assertEquals(302, $client->getResponse()->getStatusCode());
$crawler = $client->followRedirect();
$content = $client->getContainer()
->get('doctrine.orm.entity_manager')
->getRepository('WallabagCoreBundle:Entry')
->findByUrlAndUserId(
'https://venngage.com/blog/hashtags-are-worthless/',
$this->getLoggedInUserId()
);
$this->assertGreaterThan(1, $body = $crawler->filter('body')->extract(['_text']));
$this->assertContains('flashes.import.notice.summary', $body[0]);
}
public function testImportReadabilityWithFileAndMarkAllAsRead()
{
$this->logInAs('admin');
$client = $this->getClient();
$crawler = $client->request('GET', '/import/readability');
$form = $crawler->filter('form[name=upload_import_file] > button[type=submit]')->form();
$file = new UploadedFile(__DIR__.'/../fixtures/readability-read.json', 'readability-read.json');
$data = [
'upload_import_file[file]' => $file,
'upload_import_file[mark_as_read]' => 1,
];
$client->submit($form, $data);
$this->assertEquals(302, $client->getResponse()->getStatusCode());
$crawler = $client->followRedirect();
$content1 = $client->getContainer()
->get('doctrine.orm.entity_manager')
->getRepository('WallabagCoreBundle:Entry')
->findByUrlAndUserId(
'https://blog.travis-ci.com/2016-07-28-what-we-learned-from-analyzing-2-million-travis-builds/',
$this->getLoggedInUserId()
);
$this->assertTrue($content1->isArchived());
$content2 = $client->getContainer()
->get('doctrine.orm.entity_manager')
->getRepository('WallabagCoreBundle:Entry')
->findByUrlAndUserId(
'https://facebook.github.io/graphql/',
$this->getLoggedInUserId()
);
$this->assertTrue($content2->isArchived());
$this->assertGreaterThan(1, $body = $crawler->filter('body')->extract(['_text']));
$this->assertContains('flashes.import.notice.summary', $body[0]);
}
public function testImportReadabilityWithEmptyFile()
{
$this->logInAs('admin');
$client = $this->getClient();
$crawler = $client->request('GET', '/import/readability');
$form = $crawler->filter('form[name=upload_import_file] > button[type=submit]')->form();
$file = new UploadedFile(__DIR__.'/../fixtures/test.txt', 'test.txt');
$data = [
'upload_import_file[file]' => $file,
];
$client->submit($form, $data);
$this->assertEquals(302, $client->getResponse()->getStatusCode());
$crawler = $client->followRedirect();
$this->assertGreaterThan(1, $body = $crawler->filter('body')->extract(['_text']));
$this->assertContains('flashes.import.notice.failed', $body[0]);
}
}

View file

@ -0,0 +1,150 @@
<?php
namespace Tests\Wallabag\ImportBundle\Import;
use Wallabag\ImportBundle\Import\ReadabilityImport;
use Wallabag\UserBundle\Entity\User;
use Wallabag\CoreBundle\Entity\Entry;
use Monolog\Logger;
use Monolog\Handler\TestHandler;
class ReadabilityImportTest extends \PHPUnit_Framework_TestCase
{
protected $user;
protected $em;
protected $logHandler;
protected $contentProxy;
private function getReadabilityImport($unsetUser = false)
{
$this->user = new User();
$this->em = $this->getMockBuilder('Doctrine\ORM\EntityManager')
->disableOriginalConstructor()
->getMock();
$this->contentProxy = $this->getMockBuilder('Wallabag\CoreBundle\Helper\ContentProxy')
->disableOriginalConstructor()
->getMock();
$wallabag = new ReadabilityImport($this->em, $this->contentProxy);
$this->logHandler = new TestHandler();
$logger = new Logger('test', [$this->logHandler]);
$wallabag->setLogger($logger);
if (false === $unsetUser) {
$wallabag->setUser($this->user);
}
return $wallabag;
}
public function testInit()
{
$readabilityImport = $this->getReadabilityImport();
$this->assertEquals('Readability', $readabilityImport->getName());
$this->assertNotEmpty($readabilityImport->getUrl());
$this->assertEquals('import.readability.description', $readabilityImport->getDescription());
}
public function testImport()
{
$readabilityImport = $this->getReadabilityImport();
$readabilityImport->setFilepath(__DIR__.'/../fixtures/readability.json');
$entryRepo = $this->getMockBuilder('Wallabag\CoreBundle\Repository\EntryRepository')
->disableOriginalConstructor()
->getMock();
$entryRepo->expects($this->exactly(2))
->method('findByUrlAndUserId')
->will($this->onConsecutiveCalls(false, true));
$this->em
->expects($this->any())
->method('getRepository')
->willReturn($entryRepo);
$entry = $this->getMockBuilder('Wallabag\CoreBundle\Entity\Entry')
->disableOriginalConstructor()
->getMock();
$this->contentProxy
->expects($this->exactly(1))
->method('updateEntry')
->willReturn($entry);
$res = $readabilityImport->import();
$this->assertTrue($res);
$this->assertEquals(['skipped' => 1, 'imported' => 1], $readabilityImport->getSummary());
}
public function testImportAndMarkAllAsRead()
{
$readabilityImport = $this->getReadabilityImport();
$readabilityImport->setFilepath(__DIR__.'/../fixtures/readability-read.json');
$entryRepo = $this->getMockBuilder('Wallabag\CoreBundle\Repository\EntryRepository')
->disableOriginalConstructor()
->getMock();
$entryRepo->expects($this->exactly(2))
->method('findByUrlAndUserId')
->will($this->onConsecutiveCalls(false, false));
$this->em
->expects($this->any())
->method('getRepository')
->willReturn($entryRepo);
$this->contentProxy
->expects($this->exactly(2))
->method('updateEntry')
->willReturn(new Entry($this->user));
// check that every entry persisted are archived
$this->em
->expects($this->any())
->method('persist')
->with($this->callback(function ($persistedEntry) {
return $persistedEntry->isArchived();
}));
$res = $readabilityImport->setMarkAsRead(true)->import();
$this->assertTrue($res);
$this->assertEquals(['skipped' => 0, 'imported' => 2], $readabilityImport->getSummary());
}
public function testImportBadFile()
{
$readabilityImport = $this->getReadabilityImport();
$readabilityImport->setFilepath(__DIR__.'/../fixtures/wallabag-v1.jsonx');
$res = $readabilityImport->import();
$this->assertFalse($res);
$records = $this->logHandler->getRecords();
$this->assertContains('ReadabilityImport: unable to read file', $records[0]['message']);
$this->assertEquals('ERROR', $records[0]['level_name']);
}
public function testImportUserNotDefined()
{
$readabilityImport = $this->getReadabilityImport(true);
$readabilityImport->setFilepath(__DIR__.'/../fixtures/readability.json');
$res = $readabilityImport->import();
$this->assertFalse($res);
$records = $this->logHandler->getRecords();
$this->assertContains('ReadabilityImport: user is not defined', $records[0]['message']);
$this->assertEquals('ERROR', $records[0]['level_name']);
}
}

View file

@ -0,0 +1,25 @@
{
"bookmarks": [
{
"article__excerpt": "This is a guest post from Moritz Beller from the Delft University of Technology in The Netherlands. His team produced amazing research on several million Travis CI builds, creating invaluable&hellip;",
"favorite": false,
"date_archived": "2016-08-02T06:49:30",
"article__url": "https://blog.travis-ci.com/2016-07-28-what-we-learned-from-analyzing-2-million-travis-builds/",
"date_added": "2016-08-01T05:24:16",
"date_favorited": null,
"article__title": "Travis",
"archive": true
},
{
"article__excerpt": "The GraphQL Type system describes the capabilities of a GraphQL server and is used to determine if a query is valid. The type system also describes the input types of query variables to determine if&hellip;",
"favorite": false,
"date_archived": "2016-07-19T06:48:31",
"article__url": "https://facebook.github.io/graphql/",
"date_added": "2016-06-24T17:50:16",
"date_favorited": null,
"article__title": "GraphQL",
"archive": true
}
],
"recommendations": []
}

View file

@ -0,0 +1,25 @@
{
"bookmarks": [
{
"article__excerpt": "When Twitter started it had so much promise to change the way we communicate. But now it has been ruined by the amount of garbage and hate we have to wade through. It&#x2019;s like that polluted&hellip;",
"favorite": false,
"date_archived": null,
"article__url": "https://venngage.com/blog/hashtags-are-worthless/",
"date_added": "2016-08-25T12:05:00",
"date_favorited": null,
"article__title": "We Looked At 167,943 Tweets & Found Out Hashtags Are Worthless",
"archive": false
},
{
"article__excerpt": "TL;DR: Re-use your DOM elements and remove the ones that are far away from the viewport. Use placeholders to account for delayed data. Here&#x2019;s a demo and the code for the infinite&hellip;",
"favorite": false,
"date_archived": "2016-08-26T12:21:54",
"article__url": "https://developers.google.com/web/updates/2016/07/infinite-scroller?imm_mid=0e6839&cmp=em-webops-na-na-newsltr_20160805",
"date_added": "2016-08-06T05:35:26",
"date_favorited": null,
"article__title": "Complexities of an infinite scroller | Web Updates",
"archive": true
}
],
"recommendations": []
}