[mod] replace Markdown parser mistletoe by markdown-it-py

There are several reasons why we should prefer markdown-it-py over mistletoe:

- Get identical rendering results in SearXNG's `/info` pages and the SearXNG's
  project documentation which is build by Sphinx-doc.

  In the Sphinx-doc we use the MyST parser to render Markdown and the MyST
  parser itself is built on top of the markdown-it-py package.

- markdown-it-py has a typographer that supports *replacements*
  and *smartquotes* (e.g. em-dash, copyright, ellipsis, ...) [1]

- markdown-it-py is much more flexible compared to mistletoe [2]

- markdown-it-py is the fastest CommonMark compliant parser in python [3]

[1] https://markdown-it-py.readthedocs.io/en/latest/using.html#typographic-components
[2] https://markdown-it-py.readthedocs.io/en/latest/plugins.html
[3] https://markdown-it-py.readthedocs.io/en/latest/other.html#performance

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

docs/conf.py

@@ -126,6 +126,10 @@ extensions = [
     'notfound.extension',  # https://github.com/readthedocs/sphinx-notfound-page
 ]
 
+myst_enable_extensions = [
+  "replacements", "smartquotes"
+]
+
 suppress_warnings = ['myst.domains']
 
 intersphinx_mapping = {

requirements.txt

@@ -14,5 +14,5 @@ httpx-socks[asyncio]==0.7.2
 langdetect==1.0.9
 setproctitle==1.2.2
 redis==4.2.0
-mistletoe==0.8.2
+markdown-it-py==2.0.1
 typing_extensions==4.1.1

searx/infopage/__init__.py

@@ -29,7 +29,7 @@ import typing
 import urllib.parse
 import jinja2
 from flask.helpers import url_for
-import mistletoe
+from markdown_it import MarkdownIt
 
 from .. import get_setting
 from ..compat import cached_property
@@ -71,13 +71,17 @@ class InfoPage:
 
     @cached_property
     def html(self):
-        """Render Markdown (CommonMark_) to HTML by using mistletoe_.
+        """Render Markdown (CommonMark_) to HTML by using markdown-it-py_.
 
         .. _CommonMark: https://commonmark.org/
-        .. _mistletoe: https://github.com/miyuchina/mistletoe
+        .. _markdown-it-py: https://github.com/executablebooks/markdown-it-py
 
         """
-        return mistletoe.markdown(self.content)
+        return MarkdownIt(
+            "commonmark", {"typographer": True}
+        ).enable(
+            ["replacements", "smartquotes"]
+        ).render(self.content)
 
     def get_ctx(self):  # pylint: disable=no-self-use
         """Jinja context to render :py:obj:`InfoPage.content`"""