diff --git a/README.md b/README.md index c8b6e92bf..2564d7af0 100644 --- a/README.md +++ b/README.md @@ -20,22 +20,25 @@ You can request an invite to https://bookwyrm.social by [email](mailto:mousereev ## Contributing -There are many ways you can contribute to this project, regardless of your level of technical expertise. +There are many ways you can contribute to this project, regardless of your level of technical expertise. ### Feedback and feature requests Please feel encouraged and welcome to point out bugs, suggestions, feature requests, and ideas for how things ought to work using [GitHub issues](https://github.com/mouse-reeve/bookwyrm/issues). ### Code contributions -Code contributons are gladly welcomed! If you're not sure where to start, take a look at the ["Good first issue"](https://github.com/mouse-reeve/bookwyrm/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) tag. Because BookWyrm is a small project, there isn't a lot of formal structure, but there is a huge capacity for one-on-one support, which can look like asking questions as you go, pair programming, video chats, et cetera, so please feel free to reach out. +Code contributions are gladly welcomed! If you're not sure where to start, take a look at the ["Good first issue"](https://github.com/mouse-reeve/bookwyrm/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) tag. Because BookWyrm is a small project, there isn't a lot of formal structure, but there is a huge capacity for one-on-one support, which can look like asking questions as you go, pair programming, video chats, et cetera, so please feel free to reach out. -If you have questions about the project or contributing, you can seet up a video call during BookWyrm ["office hours"](https://calendly.com/mouse-reeve/30min). +If you have questions about the project or contributing, you can set up a video call during BookWyrm ["office hours"](https://calendly.com/mouse-reeve/30min). + +### Translation +Do you speak a language besides English? BookWyrm needs localization! If you're comfortable using git and want to get into the code, there are [instructions](#workin-with-translations-and-locale-files) on how to create and edit localization files. If you feel more comfortable working in a regular text editor and would prefer not to run the application, get in touch directly and we can figure out a system, like emailing a text file, that works best. ### Financial Support BookWyrm is an ad-free passion project with no intentions of seeking out venture funding or corporate financial relationships. If you want to help keep the project going, you can donate to the [Patreon](https://www.patreon.com/bookwyrm), or make a one time gift via [PayPal](https://paypal.me/oulipo). ## About BookWyrm ### What it is and isn't -BookWyrm is a platform for social reading! You can use it to track what you're reading, review books, and follow your friends. It isn't primarily meant for cataloguing or as a datasource for books, but it does do both of those things to some degree. +BookWyrm is a platform for social reading! You can use it to track what you're reading, review books, and follow your friends. It isn't primarily meant for cataloguing or as a data-source for books, but it does do both of those things to some degree. ### The role of federation BookWyrm is built on [ActivityPub](http://activitypub.rocks/). With ActivityPub, it inter-operates with different instances of BookWyrm, and other ActivityPub compliant services, like Mastodon. This means you can run an instance for your book club, and still follow your friend who posts on a server devoted to 20th century Russian speculative fiction. It also means that your friend on mastodon can read and comment on a book review that you post on your BookWyrm instance. @@ -68,7 +71,7 @@ Since the project is still in its early stages, the features are growing every d - Private, followers-only, and public privacy levels for posting, shelves, and lists - Option for users to manually approve followers - Allow blocking and flagging for moderation - + ### The Tech Stack Web backend - [Django](https://www.djangoproject.com/) web server @@ -76,12 +79,12 @@ Web backend - [ActivityPub](http://activitypub.rocks/) federation - [Celery](http://celeryproject.org/) task queuing - [Redis](https://redis.io/) task backend - + Front end - Django templates - [Bulma.io](https://bulma.io/) css framework - Vanilla JavaScript, in moderation - + Deployment - [Docker](https://www.docker.com/) and docker-compose - [Gunicorn](https://gunicorn.org/) web runner @@ -109,19 +112,46 @@ docker-compose up Once the build is complete, you can access the instance at `localhost:1333` +### Editing static files +If you edit the CSS or JavaScript, you will need to run Django's `collectstatic` command in order for your changes to have effect. You can do this by running: +``` bash +./bw-dev collectstatic +``` + +### Workin with translations and locale files +Text in the html files are wrapped in translation tags (`{% trans %}` and `{% blocktrans %}`), and Django generates locale files for all the strings in which you can add translations for the text. You can find existing translations in the `locale/` directory. + +The application's language is set by a request header sent by your browser to the application, so to change the language of the application, you can change the default language requested by your browser. + +#### Adding a locale +To start translation into a language which is currently supported, run the django-admin `makemessages` command with the language code for the language you want to add (like `de` for German, or `en-gb` for British English): +``` bash +./bw-dev makemessages -l +``` + +#### Editing a locale +When you have a locale file, open the `django.po` in the directory for the language (for example, if you were adding German, `locale/de/LC_MESSAGES/django.po`. All the the text in the application will be shown in paired strings, with `msgid` as the original text, and `msgstr` as the translation (by default, this is set to an empty string, and will display the original text). + +Add you translations to the `msgstr` strings, and when you're ready, compile the locale by running: +``` bash +./bw-dev compilemessages +``` + +You can add the `-l ` to only compile one language. When you refresh the application, you should see your translations at work. + ## Installing in Production -This project is still young and isn't, at the momoment, very stable, so please procede with caution when running in production. +This project is still young and isn't, at the moment, very stable, so please proceed with caution when running in production. ### Server setup - Get a domain name and set up DNS for your server - - Set your server up with appropriate firewalls for running a web application (this instruction set is tested again Ubuntu 20.04) + - Set your server up with appropriate firewalls for running a web application (this instruction set is tested against Ubuntu 20.04) - Set up an email service (such as mailgun) and the appropriate SMTP/DNS settings - Install Docker and docker-compose ### Install and configure BookWyrm -The `production` branch of BookWyrm contains a number of tools not on the `main` branch that are suited for running in production, such as `docker-compose` changes to update the default commands or configuration of containers, and indivudal changes to container config to enable things like SSL or regular backups. +The `production` branch of BookWyrm contains a number of tools not on the `main` branch that are suited for running in production, such as `docker-compose` changes to update the default commands or configuration of containers, and individual changes to container config to enable things like SSL or regular backups. Instructions for running BookWyrm in production: @@ -171,3 +201,4 @@ There are three concepts in the book data model: - `Edition`, a concrete, actually published version of a book Whenever a user interacts with a book, they are interacting with a specific edition. Every work has a default edition, but the user can select other editions. Reviews aggregated for all editions of a work when you view an edition's page. + diff --git a/bookwyrm/goodreads_import.py b/bookwyrm/goodreads_import.py index 1b2b971c7..f5b84e179 100644 --- a/bookwyrm/goodreads_import.py +++ b/bookwyrm/goodreads_import.py @@ -1,121 +1,13 @@ ''' handle reading a csv from goodreads ''' -import csv -import logging +from bookwyrm.importer import Importer -from bookwyrm import models -from bookwyrm.models import ImportJob, ImportItem -from bookwyrm.tasks import app +# GoodReads is the default importer, thus Importer follows its structure. For a more complete example of overriding see librarything_import.py -logger = logging.getLogger(__name__) +class GoodreadsImporter(Importer): + service = 'GoodReads' - -def create_job(user, csv_file, include_reviews, privacy): - ''' check over a csv and creates a database entry for the job''' - job = ImportJob.objects.create( - user=user, - include_reviews=include_reviews, - privacy=privacy - ) - for index, entry in enumerate(list(csv.DictReader(csv_file))): - if not all(x in entry for x in ('ISBN13', 'Title', 'Author')): - raise ValueError('Author, title, and isbn must be in data.') - ImportItem(job=job, index=index, data=entry).save() - return job - - -def create_retry_job(user, original_job, items): - ''' retry items that didn't import ''' - job = ImportJob.objects.create( - user=user, - include_reviews=original_job.include_reviews, - privacy=original_job.privacy, - retry=True - ) - for item in items: - ImportItem(job=job, index=item.index, data=item.data).save() - return job - - -def start_import(job): - ''' initalizes a csv import job ''' - result = import_data.delay(job.id) - job.task_id = result.id - job.save() - - -@app.task -def import_data(job_id): - ''' does the actual lookup work in a celery task ''' - job = ImportJob.objects.get(id=job_id) - try: - for item in job.items.all(): - try: - item.resolve() - except Exception as e:# pylint: disable=broad-except - logger.exception(e) - item.fail_reason = 'Error loading book' - item.save() - continue - - if item.book: - item.save() - - # shelves book and handles reviews - handle_imported_book( - job.user, item, job.include_reviews, job.privacy) - else: - item.fail_reason = 'Could not find a match for book' - item.save() - finally: - job.complete = True - job.save() - - -def handle_imported_book(user, item, include_reviews, privacy): - ''' process a goodreads csv and then post about it ''' - if isinstance(item.book, models.Work): - item.book = item.book.default_edition - if not item.book: - return - - existing_shelf = models.ShelfBook.objects.filter( - book=item.book, user=user).exists() - - # shelve the book if it hasn't been shelved already - if item.shelf and not existing_shelf: - desired_shelf = models.Shelf.objects.get( - identifier=item.shelf, - user=user - ) - models.ShelfBook.objects.create( - book=item.book, shelf=desired_shelf, user=user) - - for read in item.reads: - # check for an existing readthrough with the same dates - if models.ReadThrough.objects.filter( - user=user, book=item.book, - start_date=read.start_date, - finish_date=read.finish_date - ).exists(): - continue - read.book = item.book - read.user = user - read.save() - - if include_reviews and (item.rating or item.review): - review_title = 'Review of {!r} on Goodreads'.format( - item.book.title, - ) if item.review else '' - - # we don't know the publication date of the review, - # but "now" is a bad guess - published_date_guess = item.date_read or item.date_added - models.Review.objects.create( - user=user, - book=item.book, - name=review_title, - content=item.review, - rating=item.rating, - published_date=published_date_guess, - privacy=privacy, - ) + def parse_fields(self, data): + data.update({'import_source': self.service }) + # add missing 'Date Started' field + data.update({'Date Started': None }) + return data diff --git a/bookwyrm/importer.py b/bookwyrm/importer.py new file mode 100644 index 000000000..a12884007 --- /dev/null +++ b/bookwyrm/importer.py @@ -0,0 +1,135 @@ +''' handle reading a csv from an external service, defaults are from GoodReads ''' +import csv +import logging + +from bookwyrm import models +from bookwyrm.models import ImportJob, ImportItem +from bookwyrm.tasks import app + +logger = logging.getLogger(__name__) + +class Importer: + service = 'Unknown' + delimiter = ',' + encoding = 'UTF-8' + mandatory_fields = ['Title', 'Author'] + + def create_job(self, user, csv_file, include_reviews, privacy): + ''' check over a csv and creates a database entry for the job''' + job = ImportJob.objects.create( + user=user, + include_reviews=include_reviews, + privacy=privacy + ) + for index, entry in enumerate(list(csv.DictReader(csv_file, delimiter=self.delimiter ))): + if not all(x in entry for x in self.mandatory_fields): + raise ValueError('Author and title must be in data.') + entry = self.parse_fields(entry) + self.save_item(job, index, entry) + return job + + + def save_item(self, job, index, data): + ImportItem(job=job, index=index, data=data).save() + + def parse_fields(self, entry): + entry.update({'import_source': self.service }) + return entry + + def create_retry_job(self, user, original_job, items): + ''' retry items that didn't import ''' + job = ImportJob.objects.create( + user=user, + include_reviews=original_job.include_reviews, + privacy=original_job.privacy, + retry=True + ) + for item in items: + self.save_item(job, item.index, item.data) + return job + + + def start_import(self, job): + ''' initalizes a csv import job ''' + result = import_data.delay(self.service, job.id) + job.task_id = result.id + job.save() + + +@app.task +def import_data(source, job_id): + ''' does the actual lookup work in a celery task ''' + job = ImportJob.objects.get(id=job_id) + try: + for item in job.items.all(): + try: + item.resolve() + except Exception as e:# pylint: disable=broad-except + logger.exception(e) + item.fail_reason = 'Error loading book' + item.save() + continue + + if item.book: + item.save() + + # shelves book and handles reviews + handle_imported_book(source, + job.user, item, job.include_reviews, job.privacy) + else: + item.fail_reason = 'Could not find a match for book' + item.save() + finally: + job.complete = True + job.save() + + +def handle_imported_book(source, user, item, include_reviews, privacy): + ''' process a csv and then post about it ''' + if isinstance(item.book, models.Work): + item.book = item.book.default_edition + if not item.book: + return + + existing_shelf = models.ShelfBook.objects.filter( + book=item.book, user=user).exists() + + # shelve the book if it hasn't been shelved already + if item.shelf and not existing_shelf: + desired_shelf = models.Shelf.objects.get( + identifier=item.shelf, + user=user + ) + models.ShelfBook.objects.create( + book=item.book, shelf=desired_shelf, user=user) + + for read in item.reads: + # check for an existing readthrough with the same dates + if models.ReadThrough.objects.filter( + user=user, book=item.book, + start_date=read.start_date, + finish_date=read.finish_date + ).exists(): + continue + read.book = item.book + read.user = user + read.save() + + if include_reviews and (item.rating or item.review): + review_title = 'Review of {!r} on {!r}'.format( + item.book.title, + source, + ) if item.review else '' + + # we don't know the publication date of the review, + # but "now" is a bad guess + published_date_guess = item.date_read or item.date_added + models.Review.objects.create( + user=user, + book=item.book, + name=review_title, + content=item.review, + rating=item.rating, + published_date=published_date_guess, + privacy=privacy, + ) diff --git a/bookwyrm/librarything_import.py b/bookwyrm/librarything_import.py new file mode 100644 index 000000000..0584daad9 --- /dev/null +++ b/bookwyrm/librarything_import.py @@ -0,0 +1,42 @@ +''' handle reading a csv from librarything ''' +import csv +import re +import math + +from bookwyrm import models +from bookwyrm.models import ImportItem +from bookwyrm.importer import Importer + + +class LibrarythingImporter(Importer): + service = 'LibraryThing' + delimiter = '\t' + encoding = 'ISO-8859-1' + # mandatory_fields : fields matching the book title and author + mandatory_fields = ['Title', 'Primary Author'] + + def parse_fields(self, initial): + data = {} + data['import_source'] = self.service + data['Book Id'] = initial['Book Id'] + data['Title'] = initial['Title'] + data['Author'] = initial['Primary Author'] + data['ISBN13'] = initial['ISBN'] + data['My Review'] = initial['Review'] + if initial['Rating']: + data['My Rating'] = math.ceil(float(initial['Rating'])) + else: + data['My Rating'] = '' + data['Date Added'] = re.sub('\[|\]', '', initial['Entry Date']) + data['Date Started'] = re.sub('\[|\]', '', initial['Date Started']) + data['Date Read'] = re.sub('\[|\]', '', initial['Date Read']) + + data['Exclusive Shelf'] = None + if data['Date Read']: + data['Exclusive Shelf'] = "read" + elif data['Date Started']: + data['Exclusive Shelf'] = "reading" + else: + data['Exclusive Shelf'] = "to-read" + + return data diff --git a/bookwyrm/migrations/0046_sitesettings_privacy_policy.py b/bookwyrm/migrations/0046_sitesettings_privacy_policy.py new file mode 100644 index 000000000..0c49d607c --- /dev/null +++ b/bookwyrm/migrations/0046_sitesettings_privacy_policy.py @@ -0,0 +1,18 @@ +# Generated by Django 3.0.7 on 2021-02-27 19:12 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('bookwyrm', '0045_auto_20210210_2114'), + ] + + operations = [ + migrations.AddField( + model_name='sitesettings', + name='privacy_policy', + field=models.TextField(default='Add a privacy policy here.'), + ), + ] diff --git a/bookwyrm/models/import_job.py b/bookwyrm/models/import_job.py index 407d820bb..ca05ddb08 100644 --- a/bookwyrm/models/import_job.py +++ b/bookwyrm/models/import_job.py @@ -97,8 +97,8 @@ class ImportItem(models.Model): def get_book_from_title_author(self): ''' search by title and author ''' search_term = construct_search_term( - self.data['Title'], - self.data['Author'] + self.title, + self.author ) search_result = connector_manager.first_search_result( search_term, min_confidence=0.999 @@ -149,6 +149,14 @@ class ImportItem(models.Model): dateutil.parser.parse(self.data['Date Added'])) return None + @property + def date_started(self): + ''' when the book was started ''' + if "Date Started" in self.data and self.data['Date Started']: + return timezone.make_aware( + dateutil.parser.parse(self.data['Date Started'])) + return None + @property def date_read(self): ''' the date a book was completed ''' @@ -160,18 +168,24 @@ class ImportItem(models.Model): @property def reads(self): ''' formats a read through dataset for the book in this line ''' - if (self.shelf == 'reading' - and self.date_added and not self.date_read): - return [ReadThrough(start_date=self.date_added)] + start_date = self.date_started + + # Goodreads special case (no 'date started' field) + if ((self.shelf == 'reading' or (self.shelf == 'read' and self.date_read)) + and self.date_added and not start_date): + start_date = self.date_added + + if (start_date and start_date is not None and not self.date_read): + return [ReadThrough(start_date=start_date)] if self.date_read: return [ReadThrough( - start_date=self.date_added, + start_date=start_date, finish_date=self.date_read, )] return [] def __repr__(self): - return "".format(self.data['Title']) + return "<{!r}Item {!r}>".format(self.data['import_source'], self.data['Title']) def __str__(self): return "{} by {}".format(self.data['Title'], self.data['Author']) diff --git a/bookwyrm/models/site.py b/bookwyrm/models/site.py index 4670bd948..d39718b30 100644 --- a/bookwyrm/models/site.py +++ b/bookwyrm/models/site.py @@ -20,6 +20,8 @@ class SiteSettings(models.Model): default='Contact an administrator to get an invite') code_of_conduct = models.TextField( default='Add a code of conduct here.') + privacy_policy = models.TextField( + default='Add a privacy policy here.') allow_registration = models.BooleanField(default=True) logo = models.ImageField( upload_to='logos/', null=True, blank=True diff --git a/bookwyrm/settings.py b/bookwyrm/settings.py index 46c38b5af..9446b09c0 100644 --- a/bookwyrm/settings.py +++ b/bookwyrm/settings.py @@ -1,9 +1,10 @@ ''' bookwyrm settings and configuration ''' import os - from environs import Env import requests +from django.utils.translation import gettext_lazy as _ + env = Env() DOMAIN = env('DOMAIN') @@ -27,6 +28,7 @@ EMAIL_USE_TLS = env('EMAIL_USE_TLS', True) # Build paths inside the project like this: os.path.join(BASE_DIR, ...) BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +LOCALE_PATHS = [os.path.join(BASE_DIR, 'locale'),] # Quick-start development settings - unsuitable for production # See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/ @@ -58,6 +60,7 @@ INSTALLED_APPS = [ MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.middleware.locale.LocaleMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', @@ -135,6 +138,10 @@ AUTH_PASSWORD_VALIDATORS = [ # https://docs.djangoproject.com/en/2.0/topics/i18n/ LANGUAGE_CODE = 'en-us' +LANGUAGES = [ + ('en-us', _('English')), +] + TIME_ZONE = 'UTC' diff --git a/bookwyrm/templates/author.html b/bookwyrm/templates/author.html index 9f2054d0a..9dd831894 100644 --- a/bookwyrm/templates/author.html +++ b/bookwyrm/templates/author.html @@ -1,4 +1,5 @@ {% extends 'layout.html' %} +{% load i18n %} {% load bookwyrm_tags %} {% block content %}
@@ -9,8 +10,8 @@ {% if request.user.is_authenticated and perms.bookwyrm.edit_book %}
- - Edit Author + + {% trans "Edit Author" %}
@@ -25,12 +26,12 @@

{% endif %} {% if author.wikipedia_link %} -

Wikipedia

+

{% trans "Wikipedia" %}

{% endif %}
-

Books by {{ author.name }}

+

{% blocktrans with name=author.name %}Books by {{ name }}{% endblocktrans %}

{% include 'snippets/book_tiles.html' with books=books %}
{% endblock %} diff --git a/bookwyrm/templates/book.html b/bookwyrm/templates/book.html index dc7d4ce83..35ddba373 100644 --- a/bookwyrm/templates/book.html +++ b/bookwyrm/templates/book.html @@ -1,4 +1,5 @@ {% extends 'layout.html' %} +{% load i18n %} {% load bookwyrm_tags %} {% load humanize %} {% block content %} @@ -23,8 +24,8 @@ {% if request.user.is_authenticated and perms.bookwyrm.edit_book %}
- - Edit Book + + {% trans "Edit Book" %}
@@ -39,13 +40,13 @@ {% if request.user.is_authenticated and not book.cover %}
-

Add cover

+

{% trans "Add cover" %}

{% csrf_token %} - +
{% endif %} @@ -54,21 +55,21 @@
{% if book.isbn_13 %}
-
ISBN:
+
{% trans "ISBN:" %}
{{ book.isbn_13 }}
{% endif %} {% if book.oclc_number %}
-
OCLC Number:
+
{% trans "OCLC Number:" %}
{{ book.oclc_number }}
{% endif %} {% if book.asin %}
-
ASIN:
+
{% trans "ASIN:" %}
{{ book.asin }}
{% endif %} @@ -80,7 +81,7 @@

{% if book.openlibrary_key %} -

View on OpenLibrary

+

{% trans "View on OpenLibrary" %}

{% endif %} @@ -98,11 +99,11 @@
{% csrf_token %}

- +

- + {% include 'snippets/toggle/close_button.html' with text="Cancel" controls_text="add-description" controls_uid=book.id hide_inactive=True %}
@@ -134,20 +135,20 @@ {% if request.user.is_authenticated %}
-

Your reading activity

+

{% trans "Your reading activity" %}

{% include 'snippets/toggle/open_button.html' with text="Add read dates" icon="plus" class="is-small" controls_text="add-readthrough" %}
{% if not readthroughs.exists %} -

You don't have any reading activity for this book.

+

{% trans "You don't have any reading activity for this book." %}

{% endif %}