Favicon config

Implement alembic migrations and better DB models module, including some async rework

.codacy.yaml

@@ -0,0 +1,4 @@
+---
+exclude_paths:
+  - "example_config/**"
+  - "docs/source/**"

.envrc.example

@@ -0,0 +1 @@
+layout uv

.gitignore

@@ -77,10 +77,15 @@ celerybeat-schedule
 
 # dotenv
 .env
+*.env
+
+# direnv
+.envrc
 
 # virtualenv
 venv/
 ENV/
+.venv
 
 # Spyder project settings
 .spyderproject
@@ -88,8 +93,15 @@ ENV/
 # Rope project settings
 .ropeproject
 
+# JetBrains PyCharm/Idea
+.idea
+
 # vim
 *.swp
+*.swo
+
+# Zed editor
+.zed
 
 # digimarks
 static/favicons

CHANGELOG.md

@@ -7,13 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## TODO
 
-- Sorting of bookmarks
-  - Sort by title
-  - Sort by date
-- Logging of actions
 - Add new way of authentication and editing bookmark collections:
   https://github.com/aquatix/digimarks/issues/8 and https://github.com/aquatix/digimarks/issues/9
-- Change adding tags to use the MaterializeCSS tags: https://materializecss.com/chips.html
+- Change adding tags to use ~~the MaterializeCSS tags: https://materializecss.com/chips.html~~ a nice tags lib/styling
 - Do calls to the API endpoint of an existing bookmark when editing properties
   (for example to update tags, title and such, also to already suggest title)
 - Look into compatibility with del.icio.us, so we can make use of existing browser integration
@@ -22,6 +18,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- Settings now work through Pydantic Settings
+- New UI theme(s) through digui
+- Caching of the bookmarks, tags and more in the browser, for fast filtering and lookups
+- Sorting of bookmarks
+  - Sort by title
+  - Sort by date
+- Logging of actions
+- Recognise when a url already is in the list of known bookmarks and fill in the form with already-known data
+
+### Changed
+- Moved from Flask to FastAPI
+- Moved from Peewee ORM to SQLAlchemy
+
+### Removed
+- Jinja2 templates
+
+### Fixed
+
+
+## [1.2.0] - Flask is Fine (2023-07-30)
+
 ### Added
 - 'lightblue' theme
 - 'black amoled' theme

README.md

@@ -0,0 +1,130 @@
+# 🔖 digimarks
+
+[![PyPI version](https://img.shields.io/pypi/v/digimarks.svg)](https://pypi.python.org/pypi/digimarks/)
+[![PyPI license](https://img.shields.io/github/license/aquatix/digimarks.svg)](https://pypi.python.org/pypi/digimarks/)
+[![Codacy](https://api.codacy.com/project/badge/Grade/9a34319d917b43219a29e59e9ac75e3b)](https://app.codacy.com/app/aquatix/digimarks?utm_source=github.com&utm_medium=referral&utm_content=aquatix/digimarks&utm_campaign=badger)
+
+
+## 📚 Overview
+
+Simple bookmarking service, using a SQLite database to store bookmarks, supporting tags and automatic title fetching. Notes can be added, the items are cached locally in the browser, and the API is documented so everything can be accessed through that too.
+
+[digimarks source](https://codeberg.org/diginaut/digimarks)
+
+
+## 📥 Installation
+
+There are a few ways to install digimarks to your computer or server.
+
+### From PyPI
+
+Assuming you already are inside a virtualenv:
+
+```bash
+# Using the wonderfully fast uv
+uv pip install digimarks
+
+# Alternatively, use Python pip
+pip install digimarks
+```
+
+
+### From Git
+
+Create a new virtualenv (if you are not already in one) and install the
+necessary packages:
+
+```bash
+git clone https://codeberg.org/diginaut/digimarks.git
+cd digimarks
+# direnv will now create or activate a virtualenv
+# See https://codeberg.org/diginaut/dotfiles/src/branch/master/.config/direnv/direnvrc for direnv uv config
+# If you just want to run it, no need for development dependencies
+uv sync --active --no-dev
+# Otherwise, install everything in the active virtualenv
+uv sync --active
+```
+
+
+## ⚙️ Migrating from version 1
+
+To be able to use the new database schema's, you will need to migrate your existing `bookmarks.db` to one under the control of the `alembic` migrations tool.
+
+To do so, start with making a backup of this `bookmarks.db` file to a safe place.
+
+Then, stamp the initial migration into the database, and migrate to the latest version:
+
+```bash
+# Initiate migrations with the first one (only needs to be done once!)
+alembic stamp 115bcd2e1a38
+
+# Apply all migrations to get up-to-date
+alembic upgrade head
+```
+
+
+## 🛠️ Usage / example configuration
+
+⚠️ OUT OF DATE! ⚠️
+
+Copy `settings.py` from example_config to the parent directory and
+configure to your needs (*at the least* change the value of `SYSTEMKEY`).
+
+Do not forget to fill in the `MASHAPE_API_KEY` value, which you `can request on the RapidAPI website <https://rapidapi.com/realfavicongenerator/api/realfavicongenerator>`_.
+
+Run digimarks as a service under nginx or apache and call the appropriate url's when wanted.
+
+Url's are of the form `https://marks.example.com/<userkey>/<action>`
+
+digimarks can also be run from the command line: `uvicorn digimarks:app --reload`
+
+Be sure to export/set the `SECRETKEY` environment variable before running, it's needed for some management URI's.
+
+Run `gunicorn -k uvicorn.workers.UvicornWorker` for production. For an example of how to set up a server `see this article <https://www.slingacademy.com/article/deploying-fastapi-on-ubuntu-with-nginx-and-lets-encrypt/>`_ with configuration for nginx, uvicorn, systemd, security and such.
+
+The RQ background worker can be run from the command line: `rq worker --with-scheduler`
+
+Url's are of the form `https://hook.example.com/app/<appkey>/<triggerkey>`
+
+API documentation is auto-generated, and can be browsed at https://hook.example.com/docs
+
+
+## 🧩 Bookmarklet
+
+To easily save a link from your browser, open its bookmark manager and create a new bookmark with as url:
+
+```javascript
+javascript:location.href='http://marks.example.com/1234567890abcdef/add?url='+encodeURIComponent(location.href);
+```
+
+
+## Creating a new user
+
+After having set up the `settings.py` as under Usage, you can add a new user, by going to this path on your digimarks server:
+
+```
+/<secretkey>/adduser
+```
+
+where `secretkey` is the value set in settings.SYSTEMKEY
+
+digimarks will then redirect to the bookmarks overview page of the new user. Please remember the user key (the hash in the url), as it will not be visible otherwise in the interface.
+
+If you for whatever reason would lose this user key, just either look on the console (or webserver logs) where the list of available user keys is printed on digimarks startup, or open bookmarks.db with a SQLite editor.
+
+
+## 🔧 Server configuration
+
+- [systemd for digimarks API](https://codeberg.org/diginaut/digimarks/src/branch/fastapi/example_config/systemd/digimarks.service) which uses the [gunicorn config](https://codeberg.org/diginaut/digimarks/src/branch/fastapi/example_config/gunicorn_digimarks_conf.py)
+- [nginx for digimarks API](https://codeberg.org/diginaut/digimarks/src/branch/fastapi/example_config/nginx_digimarks.conf)
+- [more config](https://codeberg.org/diginaut/digimarks/src/branch/fastapi/example_config)
+
+
+## ✨ What's new?
+
+See the [Changelog](https://codeberg.org/diginaut/digimarks/src/branch/fastapi/CHANGELOG.md)
+
+
+## 🙏 Attributions
+
+'M' favicon by [Freepik](http://www.flaticon.com/free-icon/letter-m_2041)

README.rst

@@ -1,13 +1,17 @@
-digimarks
+🔖 digimarks
-=========
+===========
 
 |PyPI version| |PyPI license| |Code health| |Codacy|
 
-Simple bookmarking service, using a SQLite database to store bookmarks, supporting tags and automatic title fetching.
+
+📚 Overview
+----------
+
+Simple bookmarking service, using a SQLite database to store bookmarks, supporting tags and automatic title fetching. Notes can be added, the items are cached locally in the browser, and the API is documented so everything can be accessed through that too.
 
 
-Installation
+📥 Installation
-------------
+--------------
 
 From PyPI
 ~~~~~~~~~
@@ -16,6 +20,10 @@ Assuming you already are inside a virtualenv:
 
 .. code-block:: bash
 
+    # Using the wonderfully fast uv
+    uv pip install digimarks
+
+    # Alternatively, use Python pip
     pip install digimarks
 
 
@@ -27,24 +35,59 @@ necessary packages:
 
 .. code-block:: bash
 
-    git clone https://github.com/aquatix/digimarks.git
+    git clone https://codeberg.org/diginaut/digimarks.git
     cd digimarks
-    mkvirtualenv digimarks # or whatever project you are working on
+    # direnv will now create or activate a virtualenv
-    pip install -r requirements.txt
+    # See https://codeberg.org/diginaut/dotfiles/src/branch/master/.config/direnv/direnvrc for direnv uv config
+    # If you just want to run it, no need for development dependencies
+    uv sync --active --no-dev
+    # Otherwise, install everything in the active virtualenv
+    uv sync --active
 
 
-Usage / example configuration
+⚙️ Migrating from version 1
------------------------------
+--------------------------
+
+To be able to use the new database schema's, you will need to migrate your existing ``bookmarks.db`` to one under the control of the ``alembic`` migrations tool.
+
+To do so, start with making a backup of this ``bookmarks.db`` file to a safe place.
+
+Then, stamp the initial migration into the database, and migrate to the latest version:
+
+.. code-block:: bash
+
+   # Initiate migrations with the first one (only needs to be done once!)
+   alembic stamp 115bcd2e1a38
+
+   # Apply all migrations to get up-to-date
+   alembic upgrade head
+
+
+🛠️ Usage / example configuration
+-------------------------------
+
+⚠️ OUT OF DATE! ⚠️
 
 Copy ``settings.py`` from example_config to the parent directory and
 configure to your needs (*at the least* change the value of `SYSTEMKEY`).
 
-Do not forget to fill in the `MASHAPE_API_KEY` value, which you [can request on the RapidAPI website](https://rapidapi.com/realfavicongenerator/api/realfavicongenerator).
+Do not forget to fill in the ``MASHAPE_API_KEY`` value, which you `can request on the RapidAPI website <https://rapidapi.com/realfavicongenerator/api/realfavicongenerator>`_.
 
-Run digimarks as a service under nginx or apache and call the appropriate
+Run digimarks as a service under nginx or apache and call the appropriate url's when wanted.
-url's when wanted.
 
-Url's are of the form https://marks.example.com/<userkey>/<action>
+Url's are of the form ``https://marks.example.com/<userkey>/<action>``
+
+digimarks can also be run from the command line: ``uvicorn digimarks:app --reload``
+
+Be sure to export/set the ``SECRETKEY`` environment variable before running, it's needed for some management URI's.
+
+Run ``gunicorn -k uvicorn.workers.UvicornWorker`` for production. For an example of how to set up a server `see this article <https://www.slingacademy.com/article/deploying-fastapi-on-ubuntu-with-nginx-and-lets-encrypt/>`_ with configuration for nginx, uvicorn, systemd, security and such.
+
+The RQ background worker can be run from the command line: ``rq worker --with-scheduler``
+
+Url's are of the form https://hook.example.com/app/<appkey>/<triggerkey>
+
+API documentation is auto-generated, and can be browsed at https://hook.example.com/docs
 
 
 Bookmarklet
@@ -60,7 +103,7 @@ To easily save a link from your browser, open its bookmark manager and create a
 Creating a new user
 -------------------
 
-After having set up the ```settings.py``` as under Usage, you can add a new user, by going to this path on your digimarks server:
+After having set up the ``settings.py`` as under Usage, you can add a new user, by going to this path on your digimarks server:
 
 /<secretkey>/adduser
 
@@ -71,27 +114,27 @@ digimarks will then redirect to the bookmarks overview page of the new user. Ple
 If you for whatever reason would lose this user key, just either look on the console (or webserver logs) where the list of available user keys is printed on digimarks startup, or open bookmarks.db with a SQLite editor.
 
 
-Server configuration
+🔧 Server configuration
-~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~
 
-* `vhost for Apache2.4`_
+* `systemd for digimarks API`_ which uses the `gunicorn config`_
-* `uwsgi.ini`_
+* `nginx for digimarks API`_
+* `more config`_
 
 
-What's new?
+✨ What's new?
------------
+-------------
 
 See the `Changelog`_.
 
 
-Attributions
+🙏 Attributions
-------------
+--------------
 
 'M' favicon by `Freepik`_.
 
 
-.. _digimarks: https://github.com/aquatix/digimarks
+.. _digimarks: https://codeberg.org/diginaut/digimarks
-.. _webhook: https://en.wikipedia.org/wiki/Webhook
 .. |PyPI version| image:: https://img.shields.io/pypi/v/digimarks.svg
    :target: https://pypi.python.org/pypi/digimarks/
 .. |PyPI license| image:: https://img.shields.io/github/license/aquatix/digimarks.svg
@@ -102,8 +145,11 @@ Attributions
 .. |Codacy| image:: https://api.codacy.com/project/badge/Grade/9a34319d917b43219a29e59e9ac75e3b
    :alt: Codacy Badge
    :target: https://app.codacy.com/app/aquatix/digimarks?utm_source=github.com&utm_medium=referral&utm_content=aquatix/digimarks&utm_campaign=badger
-.. _hook settings: https://github.com/aquatix/digimarks/blob/master/example_config/examples.yaml
+.. _hook settings: https://codeberg.org/diginaut/digimarks/blob/master/example_config/examples.yaml
-.. _vhost for Apache2.4: https://github.com/aquatix/digimarks/blob/master/example_config/apache_vhost.conf
+.. _vhost for Apache2.4: https://codeberg.org/diginaut/digimarks/blob/master/example_config/apache_vhost.conf
-.. _uwsgi.ini: https://github.com/aquatix/digimarks/blob/master/example_config/uwsgi.ini
+.. _uwsgi.ini: https://codeberg.org/diginaut/digimarks/blob/master/example_config/uwsgi.ini
-.. _Changelog: https://github.com/aquatix/digimarks/blob/master/CHANGELOG.md
+.. _Changelog: https://codeberg.org/diginaut/digimarks/blob/master/CHANGELOG.md
 .. _Freepik: http://www.flaticon.com/free-icon/letter-m_2041
+.. _systemd for digimarks API: https://codeberg.org/diginaut/digimarks/blob/master/example_config/systemd/digimarks.service
+.. _gunicorn config: https://codeberg.org/diginaut/digimarks/src/branch/master/example_config/uwsgi.ini
+.. _more config: https://codeberg.org/diginaut/digimarks/src/branch/master/example_config

__init__.py

@@ -0,0 +1 @@
+"""Digimarks project."""

alembic.ini

@@ -0,0 +1,147 @@
+# A generic, single database configuration.
+
+[alembic]
+# path to migration scripts.
+# this is typically a path given in POSIX (e.g. forward slashes)
+# format, relative to the token %(here)s which refers to the location of this
+# ini file
+script_location = %(here)s/migrations
+
+# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
+# Uncomment the line below if you want the files to be prepended with date and time
+# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
+# for all available tokens
+# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
+
+# sys.path path, will be prepended to sys.path if present.
+# defaults to the current working directory.  for multiple paths, the path separator
+# is defined by "path_separator" below.
+prepend_sys_path = .
+
+# timezone to use when rendering the date within the migration file
+# as well as the filename.
+# If specified, requires the python>=3.9 or backports.zoneinfo library and tzdata library.
+# Any required deps can installed by adding `alembic[tz]` to the pip requirements
+# string value is passed to ZoneInfo()
+# leave blank for localtime
+# timezone =
+
+# max length of characters to apply to the "slug" field
+# truncate_slug_length = 40
+
+# set to 'true' to run the environment during
+# the 'revision' command, regardless of autogenerate
+# revision_environment = false
+
+# set to 'true' to allow .pyc and .pyo files without
+# a source .py file to be detected as revisions in the
+# versions/ directory
+# sourceless = false
+
+# version location specification; This defaults
+# to <script_location>/versions.  When using multiple version
+# directories, initial revisions must be specified with --version-path.
+# The path separator used here should be the separator specified by "path_separator"
+# below.
+# version_locations = %(here)s/bar:%(here)s/bat:%(here)s/alembic/versions
+
+# path_separator; This indicates what character is used to split lists of file
+# paths, including version_locations and prepend_sys_path within configparser
+# files such as alembic.ini.
+# The default rendered in new alembic.ini files is "os", which uses os.pathsep
+# to provide os-dependent path splitting.
+#
+# Note that in order to support legacy alembic.ini files, this default does NOT
+# take place if path_separator is not present in alembic.ini.  If this
+# option is omitted entirely, fallback logic is as follows:
+#
+# 1. Parsing of the version_locations option falls back to using the legacy
+#    "version_path_separator" key, which if absent then falls back to the legacy
+#    behavior of splitting on spaces and/or commas.
+# 2. Parsing of the prepend_sys_path option falls back to the legacy
+#    behavior of splitting on spaces, commas, or colons.
+#
+# Valid values for path_separator are:
+#
+# path_separator = :
+# path_separator = ;
+# path_separator = space
+# path_separator = newline
+#
+# Use os.pathsep. Default configuration used for new projects.
+path_separator = os
+
+
+# set to 'true' to search source files recursively
+# in each "version_locations" directory
+# new in Alembic version 1.10
+# recursive_version_locations = false
+
+# the output encoding used when revision files
+# are written from script.py.mako
+# output_encoding = utf-8
+
+# database URL.  This is consumed by the user-maintained env.py script only.
+# other means of configuring database URLs may be customized within the env.py
+# file.
+sqlalchemy.url = sqlite+aiosqlite:///bookmarks.db
+
+
+[post_write_hooks]
+# post_write_hooks defines scripts or Python functions that are run
+# on newly generated revision scripts.  See the documentation for further
+# detail and examples
+
+# format using "black" - use the console_scripts runner, against the "black" entrypoint
+# hooks = black
+# black.type = console_scripts
+# black.entrypoint = black
+# black.options = -l 79 REVISION_SCRIPT_FILENAME
+
+# lint with attempts to fix using "ruff" - use the module runner, against the "ruff" module
+# hooks = ruff
+# ruff.type = module
+# ruff.module = ruff
+# ruff.options = check --fix REVISION_SCRIPT_FILENAME
+
+# Alternatively, use the exec runner to execute a binary found on your PATH
+# hooks = ruff
+# ruff.type = exec
+# ruff.executable = ruff
+# ruff.options = check --fix REVISION_SCRIPT_FILENAME
+
+# Logging configuration.  This is also consumed by the user-maintained
+# env.py script only.
+[loggers]
+keys = root,sqlalchemy,alembic
+
+[handlers]
+keys = console
+
+[formatters]
+keys = generic
+
+[logger_root]
+level = WARNING
+handlers = console
+qualname =
+
+[logger_sqlalchemy]
+level = WARNING
+handlers =
+qualname = sqlalchemy.engine
+
+[logger_alembic]
+level = INFO
+handlers =
+qualname = alembic
+
+[handler_console]
+class = StreamHandler
+args = (sys.stderr,)
+level = NOTSET
+formatter = generic
+
+[formatter_generic]
+format = %(levelname)-5.5s [%(name)s] %(message)s
+datefmt = %H:%M:%S

example_config/apache_vhost.conf

@@ -1,29 +0,0 @@
-<VirtualHost *:80>
-    ServerAdmin webmaster@example.com
-    ServerName marks.example.com
-
-    WSGIDaemonProcess digimarks user=youruser group=youruser threads=5 python-path=/srv/marks.example.com/digimarks/
-    WSGIScriptAlias / /srv/marks.example.com/digimarks/wsgi.py
-
-    <Directory /srv/marks.example.com/digimarks>
-        WSGIProcessGroup digimarks
-        WSGIApplicationGroup %{GLOBAL}
-        Require all granted
-    </Directory>
-
-    <Directory /srv/marks.example.com/digimarks>
-    <Files wsgi.py>
-        Require all granted
-    </Files>
-    </Directory>
-
-    ErrorLog /var/log/apache2/error_marks.example.com.log
-
-    # Possible values include: debug, info, notice, warn, error, crit,
-    # alert, emerg.
-    LogLevel warn
-
-    CustomLog /var/log/apache2/access_marks.example.com.log combined
-    ServerSignature On
-
-</VirtualHost>

example_config/gunicorn_digimarks_conf.py

@@ -0,0 +1,14 @@
+# gunicorn_conf.py
+from multiprocessing import cpu_count
+
+bind = "127.0.0.1:8890"
+
+# Worker Options
+#workers = cpu_count() + 1
+workers = 1
+worker_class = 'uvicorn.workers.UvicornWorker'
+
+# Logging Options
+loglevel = 'debug'
+accesslog = '/var/log/digimarks/access_log'
+errorlog = '/var/log/digimarks/error_log'

example_config/nginx_digimarks.conf

@@ -0,0 +1,73 @@
+server {
+    server_name marks.example.org;
+    listen [::]:443 ssl; # managed by Certbot
+    listen 443 ssl; # managed by Certbot
+
+    real_ip_header      X-Forwarded-For;
+
+    access_log  /var/log/nginx/access_marks.example.org.log;
+    error_log  /var/log/nginx/error_marks.example.org.log  warn;
+
+    # Media: images, icons, video, audio, HTC
+    #location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|mp3|ogg|ogv|webm|htc|woff2|woff)$ {
+    #    expires 1M;
+    #    access_log off;
+    #    # max-age must be in seconds
+    #    add_header Cache-Control "max-age=2629746, public";
+    #}
+
+    # CSS and Javascript
+    #location ~* \.(?:css|js)$ {
+    #    expires 1M;
+    #    access_log off;
+    #    add_header Cache-Control "max-age=31556952, public";
+    #}
+
+    location / {
+        proxy_pass             http://127.0.0.1:8890;
+        proxy_read_timeout     60;
+        proxy_connect_timeout  60;
+        proxy_redirect         off;
+
+        # Allow the use of websockets
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection 'upgrade';
+        proxy_set_header Host $host;
+        proxy_cache_bypass $http_upgrade;
+    }
+
+    location /content/favicons/ {
+        alias /srv/www/marks.example.org/favicons/;
+        # This can certainly be cached, so do so for a month
+        expires 1M;
+        add_header Cache-Control "public";
+    }
+
+    location /static/ {
+        alias /srv/www/marks.example.org/digimarks/src/digimarks/static/;
+        # This can certainly be cached, so do so for a month
+        #expires 1M;
+        #add_header Cache-Control "public";
+    }
+
+    location /favicon.ico {
+        # Favicon for the webapp, shown in the browser
+        alias /srv/www/marks.example.org/digimarks/src/digimarks/static/favicon.ico;
+    }
+
+    ssl_certificate /etc/letsencrypt/live/marks.example.org/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/marks.example.org/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+}
+server {
+    if ($host = marks.example.org) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+    listen [::]:80 ;
+    listen 80;
+    server_name marks.example.org;
+    return 404; # managed by Certbot
+}

example_config/settings.py

@@ -1,20 +0,0 @@
-# Virtualenv to use with the wsgi file (optional)
-VENV = '/srv/marks.example.com/venv/bin/activate_this.py'
-
-PORT = 8086
-
-DEBUG = False
-
-# Password/url key to do admin stuff with, like adding a user
-# NB: change this to something else! For example, in bash:
-# echo -n "yourstring" | sha1sum
-SYSTEMKEY = 'S3kr1t'
-
-# RapidAPI key for favicons
-# https://rapidapi.com/realfavicongenerator/api/realfavicongenerator
-MASHAPE_API_KEY = 'your_MASHAPE_key'
-
-LOG_LOCATION = 'digimarks.log'
-#LOG_LOCATION = '/var/log/digimarks/digimarks.log'
-# How many logs to keep in log rotation:
-LOG_BACKUP_COUNT = 10

example_config/systemd/digimarks.service

@@ -0,0 +1,22 @@
+[Unit]
+Description=Gunicorn Daemon for digimarks FastAPI
+After=network.target
+
+[Service]
+User=YOUR_USER
+Group=YOUR_USER
+WorkingDirectory=/srv/www/marks.example.org/digimarks/src
+
+Environment="SYSTEM_KEY=RanDomSeCretKeyFoRAdmin"
+Environment="FAVICONS_DIR=/srv/www/marks.example.org/favicons"
+Environment="DATABASE_FILE=/srv/www/marks.example.org/bookmarks.db"
+Environment="STATIC_DIR=digimarks/static"
+Environment="TEMPLATE_DIR=digimarks/templates"
+
+ExecStart=/srv/www/marks.example.org/venv/bin/gunicorn -c /srv/www/marks.example.org/gunicorn_digimarks_conf.py digimarks.main:app
+
+StandardOutput=file:///var/log/digimarks/stdout.log
+StandardError=file:///var/log/digimarks/stderr.log
+
+[Install]
+WantedBy=multi-user.target

example_config/uwsgi.ini

@@ -1,15 +0,0 @@
-# Example supervisord configuration
-# Run with /srv/venv/bin/uwsgi --ini /srv/digimarks/uwsgi.ini:digimarks
-
-[digimarks]
-chdir = /srv/digimarks
-socket = /tmp/uwsgi_digimarks.sock
-module = wsgi
-threads = 4
-master = true
-processes = 5
-vacuum = true
-no-orphans = true
-chmod-socket = 666
-logger = main file:/var/log/webapps/digimarks.log
-logger = file:/var/log/webapps/digimarks_debug.log

migrations/README

@@ -0,0 +1 @@
+Generic single-database configuration with an async dbapi.

migrations/env.py

@@ -0,0 +1,94 @@
+"""Alembic environment file for SQLAlchemy."""
+
+import asyncio
+from logging.config import fileConfig
+
+from alembic import context
+from sqlalchemy import pool
+from sqlalchemy.engine import Connection
+from sqlalchemy.ext.asyncio import async_engine_from_config
+from sqlmodel import SQLModel
+
+from src.digimarks.models import Bookmark, PublicTag, User  # noqa
+
+# this is the Alembic Config object, which provides
+# access to the values within the .ini file in use.
+config = context.config
+
+# Interpret the config file for Python logging.
+# This line sets up loggers basically.
+if config.config_file_name is not None:
+    fileConfig(config.config_file_name)
+
+# add your model's MetaData object here
+# for 'autogenerate' support
+# from myapp import mymodel
+# target_metadata = mymodel.Base.metadata
+target_metadata = SQLModel.metadata
+
+# other values from the config, defined by the needs of env.py,
+# can be acquired:
+# my_important_option = config.get_main_option("my_important_option")
+# ... etc.
+
+
+def run_migrations_offline() -> None:
+    """Run migrations in 'offline' mode.
+
+    This configures the context with just a URL
+    and not an Engine, though an Engine is acceptable
+    here as well.  By skipping the Engine creation
+    we don't even need a DBAPI to be available.
+
+    Calls to context.execute() here emit the given string to the
+    script output.
+
+    """
+    url = config.get_main_option('sqlalchemy.url')
+    context.configure(
+        url=url,
+        target_metadata=target_metadata,
+        literal_binds=True,
+        dialect_opts={'paramstyle': 'named'},
+        render_as_batch=True,
+    )
+
+    with context.begin_transaction():
+        context.run_migrations()
+
+
+def do_run_migrations(connection: Connection) -> None:
+    """Run the migrations."""
+    context.configure(
+        connection=connection,
+        target_metadata=target_metadata,
+        render_as_batch=True,
+    )
+
+    with context.begin_transaction():
+        context.run_migrations()
+
+
+async def run_async_migrations() -> None:
+    """In this scenario we need to create an Engine and associate a connection with the context."""
+    connectable = async_engine_from_config(
+        config.get_section(config.config_ini_section, {}),
+        prefix='sqlalchemy.',
+        poolclass=pool.NullPool,
+    )
+
+    async with connectable.connect() as connection:
+        await connection.run_sync(do_run_migrations)
+
+    await connectable.dispose()
+
+
+def run_migrations_online() -> None:
+    """Run migrations in 'online' mode."""
+    asyncio.run(run_async_migrations())
+
+
+if context.is_offline_mode():
+    run_migrations_offline()
+else:
+    run_migrations_online()

migrations/script.py.mako

@@ -0,0 +1,29 @@
+"""${message}
+
+Revision ID: ${up_revision}
+Revises: ${down_revision | comma,n}
+Create Date: ${create_date}
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+import sqlmodel
+${imports if imports else ""}
+
+# revision identifiers, used by Alembic.
+revision: str = ${repr(up_revision)}
+down_revision: Union[str, Sequence[str], None] = ${repr(down_revision)}
+branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
+depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
+
+
+def upgrade() -> None:
+    """Upgrade schema."""
+    ${upgrades if upgrades else "pass"}
+
+
+def downgrade() -> None:
+    """Downgrade schema."""
+    ${downgrades if downgrades else "pass"}

migrations/versions/115bcd2e1a38_initial_migration.py

@@ -0,0 +1,68 @@
+"""Initial migration.
+
+Revision ID: 115bcd2e1a38
+Revises:
+Create Date: 2025-09-12 16:06:16.479075
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = '115bcd2e1a38'
+down_revision: Union[str, Sequence[str], None] = None
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Upgrade schema."""
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.create_table(
+        'bookmark',
+        sa.Column('id', sa.Integer(), nullable=False),
+        sa.Column('userkey', sa.String(length=255), nullable=False),
+        sa.Column('title', sa.String(length=255), nullable=False),
+        sa.Column('url', sa.String(length=255), nullable=False),
+        sa.Column('created_date', sa.DateTime(), nullable=False),
+        sa.Column('url_hash', sa.String(length=255), nullable=False),
+        sa.Column('tags', sa.String(length=255), nullable=False),
+        sa.Column('http_status', sa.Integer(), nullable=False),
+        sa.Column('modified_date', sa.DateTime(), nullable=True),
+        sa.Column('favicon', sa.String(length=255), nullable=True),
+        sa.Column('starred', sa.Boolean(), server_default=sa.text('0'), nullable=True),
+        sa.Column('deleted_date', sa.DateTime(), server_default=sa.text('(null)'), nullable=True),
+        sa.Column('status', sa.Integer(), server_default=sa.text('0'), nullable=True),
+        sa.Column('note', sa.Text(), server_default=sa.text('(null)'), nullable=True),
+        sa.PrimaryKeyConstraint('id'),
+    )
+    op.create_table(
+        'publictag',
+        sa.Column('id', sa.Integer(), nullable=False),
+        sa.Column('tagkey', sa.String(length=255), nullable=False),
+        sa.Column('userkey', sa.String(length=255), nullable=False),
+        sa.Column('tag', sa.String(length=255), nullable=False),
+        sa.Column('created_date', sa.DateTime(), server_default=sa.text('(null)'), nullable=True),
+        sa.PrimaryKeyConstraint('id'),
+    )
+    op.create_table(
+        'user',
+        sa.Column('id', sa.Integer(), nullable=False),
+        sa.Column('username', sa.String(length=255), nullable=False),
+        sa.Column('key', sa.String(length=255), nullable=False),
+        sa.Column('created_date', sa.DateTime(), nullable=False),
+        sa.Column('theme', sa.String(length=20), server_default=sa.text("'green'"), nullable=True),
+        sa.PrimaryKeyConstraint('id'),
+    )
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    """Downgrade schema."""
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.drop_table('user')
+    op.drop_table('publictag')
+    op.drop_table('bookmark')
+    # ### end Alembic commands ###

migrations/versions/a8d8e45f60a1_migrate_to_sqlmodel.py

@@ -0,0 +1,96 @@
+"""Migrate to sqlmodel.
+
+Revision ID: a8d8e45f60a1
+Revises: 115bcd2e1a38
+Create Date: 2025-09-12 16:10:41.378716
+"""
+
+from datetime import UTC, datetime
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+import sqlmodel
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = 'a8d8e45f60a1'
+down_revision: Union[str, Sequence[str], None] = '115bcd2e1a38'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Upgrade schema."""
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table('bookmark', schema=None) as batch_op:
+        batch_op.alter_column(
+            'note',
+            existing_type=sa.TEXT(),
+            type_=sqlmodel.sql.sqltypes.AutoString(),
+            nullable=True,
+            existing_server_default=sa.text('(null)'),
+        )
+        batch_op.alter_column(
+            'starred', existing_type=sa.BOOLEAN(), nullable=False, existing_server_default=sa.text('0')
+        )
+        batch_op.alter_column('modified_date', existing_type=sa.DATETIME(), nullable=True)
+        batch_op.alter_column(
+            'deleted_date', existing_type=sa.DATETIME(), nullable=True, existing_server_default=sa.text('(null)')
+        )
+        batch_op.alter_column(
+            'status', existing_type=sa.INTEGER(), nullable=False, existing_server_default=sa.text('0')
+        )
+        batch_op.create_foreign_key('bookmark_user', 'user', ['userkey'], ['key'])
+    with op.batch_alter_table('publictag', schema=None) as batch_op:
+        batch_op.alter_column(
+            'created_date',
+            existing_type=sa.DATETIME(),
+            nullable=True,
+            existing_server_default=sa.text(str(datetime.now(UTC))),
+        )
+        batch_op.create_foreign_key('publictag_user', 'user', ['userkey'], ['key'])
+    with op.batch_alter_table('user', schema=None) as batch_op:
+        batch_op.alter_column(
+            'theme', existing_type=sa.VARCHAR(length=20), nullable=False, existing_server_default=sa.text("'green'")
+        )
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    """Downgrade schema."""
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.alter_column(
+        'user', 'theme', existing_type=sa.VARCHAR(length=20), nullable=True, existing_server_default=sa.text("'green'")
+    )
+    op.drop_constraint(None, 'publictag', type_='foreignkey')
+    op.alter_column(
+        'publictag',
+        'created_date',
+        existing_type=sa.DATETIME(),
+        nullable=True,
+        existing_server_default=sa.text('(null)'),
+    )
+    op.drop_constraint(None, 'bookmark', type_='foreignkey')
+    op.alter_column(
+        'bookmark', 'status', existing_type=sa.INTEGER(), nullable=True, existing_server_default=sa.text('0')
+    )
+    op.alter_column(
+        'bookmark',
+        'deleted_date',
+        existing_type=sa.DATETIME(),
+        nullable=True,
+        existing_server_default=sa.text('(null)'),
+    )
+    op.alter_column('bookmark', 'modified_date', existing_type=sa.DATETIME(), nullable=True)
+    op.alter_column(
+        'bookmark', 'starred', existing_type=sa.BOOLEAN(), nullable=True, existing_server_default=sa.text('0')
+    )
+    op.alter_column(
+        'bookmark',
+        'note',
+        existing_type=sqlmodel.sql.sqltypes.AutoString(),
+        type_=sa.TEXT(),
+        nullable=True,
+        existing_server_default=sa.text('(null)'),
+    )
+    # ### end Alembic commands ###

migrations/versions/b8cbc6957df5_renamed_keys.py

@@ -0,0 +1,50 @@
+"""Renamed keys.
+
+Revision ID: b8cbc6957df5
+Revises: a8d8e45f60a1
+Create Date: 2025-09-12 22:26:38.684120
+"""
+
+from typing import Sequence, Union
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = 'b8cbc6957df5'
+down_revision: Union[str, Sequence[str], None] = 'a8d8e45f60a1'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Upgrade schema."""
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table('bookmark', schema=None) as batch_op:
+        batch_op.drop_constraint(batch_op.f('bookmark_user'), type_='foreignkey')
+        batch_op.alter_column('userkey', new_column_name='user_key')
+        batch_op.create_foreign_key('bookmark_user', 'user', ['user_key'], ['key'])
+
+    with op.batch_alter_table('publictag', schema=None) as batch_op:
+        batch_op.drop_constraint(batch_op.f('publictag_user'), type_='foreignkey')
+        batch_op.alter_column('userkey', new_column_name='user_key')
+        batch_op.alter_column('tagkey', new_column_name='tag_key')
+        batch_op.create_foreign_key('publictag_user', 'user', ['user_key'], ['key'])
+
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    """Downgrade schema."""
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table('publictag', schema=None) as batch_op:
+        batch_op.drop_constraint(batch_op.f('publictag_user'), type_='foreignkey')
+        batch_op.alter_column('user_key', new_column_name='userkey')
+        batch_op.alter_column('tag_key', new_column_name='tagkey')
+        batch_op.create_foreign_key('publictag_user', 'user', ['userkey'], ['key'])
+
+    with op.batch_alter_table('bookmark', schema=None) as batch_op:
+        batch_op.drop_constraint(batch_op.f('bookmark_user'), type_='foreignkey')
+        batch_op.alter_column('user_key', new_column_name='userkey')
+        batch_op.create_foreign_key('bookmark_user', 'user', ['userkey'], ['key'])
+
+    # ### end Alembic commands ###

pylintrc

@@ -0,0 +1,22 @@
+[FORMAT]
+max-line-length=120
+
+
+[BASIC]
+# Good variable names which should always be accepted, separated by a comma.
+good-names=i,
+           j,
+           k,
+           e,
+           ex,
+           extra,
+           f,
+           fd,
+           fp,
+           logger,
+           Run,
+           q,
+           s,
+           x,
+           y,
+           _

pyproject.toml

@@ -0,0 +1,105 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "digimarks"
+version = "1.1.99"
+authors = [
+    { name = "Michiel Scholten", email = "michiel@diginaut.net" },
+]
+description = 'Simple bookmarking service, using a SQLite database to store bookmarks, supporting tags, automatic title fetching and REST API calls.'
+readme = "README.rst"
+requires-python = ">=3.11"
+keywords = ["bookmarks", "api"]
+license = { text = "Apache" }
+classifiers = [
+    "Framework :: FastAPI",
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: Apache Software License",
+]
+dependencies = [
+    "fastapi[all]",
+    "sqlmodel",
+    "alembic",
+    "aiosqlite",
+    "pydantic>2.0",
+    "httpx",
+    "beautifulsoup4",
+    "extract_favicon",
+    "feedgen",
+]
+
+[project.optional-dependencies]
+server = [
+    "uvicorn",
+]
+
+[dependency-groups]
+dev = [
+    { include-group = "lint" },
+    { include-group = "pub" },
+    { include-group = "test" }
+]
+test = [
+    "pytest>=7.0.0",
+    "pytest-cov>=4.0.0",
+]
+lint = [
+    "ruff>=0.1.0",
+    "pyrefly",
+]
+# Publishing on PyPI
+pub = [
+    "build",
+    "twine"
+]
+server = [
+    "gunicorn>=23.0.0",
+]
+# dynamic = ["version"]
+
+[project.scripts]
+my-script = "digimarks:app"
+
+[project.urls]
+"Homepage" = "https://codeberg.org/diginaut/digimarks"
+"Bug Tracker" = "https://codeberg.org/diginaut/digimarks/issues"
+
+[tool.black]
+line-length = 120
+
+[tool.ruff]
+exclude = [
+    ".git",
+    "__pycache__",
+    "docs/source/conf.py",
+    "build",
+    "dist",
+    "example_config/gunicorn_digimarks_conf.py",
+    "example_config/settings.py",
+]
+line-length = 120
+
+[tool.ruff.format]
+# Prefer single quotes over double quotes
+quote-style = "single"
+
+[tool.ruff.lint]
+ignore = ["D203", "D213"]
+select = [
+    "C9",
+    "D",
+    "E",
+    "F",
+    "I",
+    "W",
+]
+
+[tool.ruff.lint.flake8-quotes]
+docstring-quotes = "double"
+inline-quotes = "single"
+multiline-quotes = "double"
+
+[tool.ruff.lint.mccabe]
+max-complexity = 10

requirements-dev.in

@@ -1,3 +1,14 @@
 -r requirements.in
 
-pylint
+# Linting and fixing, including isort
+ruff
+# Typing
+pyrefly
+
+# Test suite
+pytest
+pytest-cov
+
+# Publishing on PyPI
+build
+twine

requirements-dev.txt

@@ -1,67 +1,11 @@
-#
+-r requirements.txt
-# This file is autogenerated by pip-compile with python 3.10
-# To update, run:
-#
-#    pip-compile requirements-dev.in
-#
-astroid==2.11.7
-    # via pylint
-beautifulsoup4==4.11.1
-    # via bs4
-bs4==0.0.1
-    # via -r requirements.in
-certifi==2022.6.15
-    # via requests
-charset-normalizer==2.1.0
-    # via requests
-click==8.1.3
-    # via flask
-dill==0.3.5.1
-    # via pylint
-feedgen==0.9.0
-    # via -r requirements.in
-flask==2.1.3
-    # via -r requirements.in
-idna==3.3
-    # via requests
-isort==5.10.1
-    # via pylint
-itsdangerous==2.1.2
-    # via flask
-jinja2==3.1.2
-    # via flask
-lazy-object-proxy==1.7.1
-    # via astroid
-lxml==4.9.1
-    # via feedgen
-markupsafe==2.1.1
-    # via jinja2
-mccabe==0.7.0
-    # via pylint
-peewee==3.15.1
-    # via -r requirements.in
-platformdirs==2.5.2
-    # via pylint
-pylint==2.14.5
-    # via -r requirements-dev.in
-python-dateutil==2.8.2
-    # via feedgen
-requests==2.28.1
-    # via -r requirements.in
-six==1.16.0
-    # via python-dateutil
-soupsieve==2.3.2.post1
-    # via beautifulsoup4
-tomli==2.0.1
-    # via pylint
-tomlkit==0.11.1
-    # via pylint
-urllib3==1.26.10
-    # via requests
-werkzeug==2.1.2
-    # via flask
-wrapt==1.14.1
-    # via astroid
 
-# The following packages are considered to be unsafe in a requirements file:
+# Linting and fixing, including isort
-# setuptools
+ruff
+
+# Test suite
+pytest
+
+# Publishing on PyPI
+build
+twine

requirements-server.in

@@ -0,0 +1,3 @@
+-r requirements.in
+
+gunicorn

requirements.in

@@ -1,10 +1,20 @@
 # Core application
-flask
+fastapi[all]
-peewee
+sqlmodel
+sqlalchemy[asyncio]
+pydantic
+pydantic_settings
+alembic
+aiosqlite
+
+# Fetch external resources
+httpx
 
 # Fetch title etc from links
-bs4
+beautifulsoup4
-requests
+
+# Fetch favicons
+extract_favicon
 
 # Generate (atom) feeds for tags and such
 feedgen

requirements.txt

@@ -1,44 +1,20 @@
-#
+# Core application
-# This file is autogenerated by pip-compile with python 3.10
+fastapi[all]
-# To update, run:
+sqlmodel
-#
+sqlalchemy
-#    pip-compile requirements.in
+pydantic
-#
+pydantic_settings
-beautifulsoup4==4.11.1
+alembic
-    # via bs4
+aiosqlite
-bs4==0.0.1
+
-    # via -r requirements.in
+# Fetch external resources
-certifi==2022.6.15
+httpx
-    # via requests
+
-charset-normalizer==2.1.0
+# Fetch title etc from links
-    # via requests
+beautifulsoup4
-click==8.1.3
+
-    # via flask
+# Fetch favicons
-feedgen==0.9.0
+extract_favicon
-    # via -r requirements.in
+
-flask==2.1.3
+# Generate (atom) feeds for tags and such
-    # via -r requirements.in
+feedgen
-idna==3.3
-    # via requests
-itsdangerous==2.1.2
-    # via flask
-jinja2==3.1.2
-    # via flask
-lxml==4.9.1
-    # via feedgen
-markupsafe==2.1.1
-    # via jinja2
-peewee==3.15.1
-    # via -r requirements.in
-python-dateutil==2.8.2
-    # via feedgen
-requests==2.28.1
-    # via -r requirements.in
-six==1.16.0
-    # via python-dateutil
-soupsieve==2.3.2.post1
-    # via beautifulsoup4
-urllib3==1.26.10
-    # via requests
-werkzeug==2.1.2
-    # via flask

setup.py

@@ -1,43 +1,7 @@
-"""
+#!/usr/bin/env python
-A setuptools based setup module.
+"""Install script for module installation. Compatibility stub because pyproject.toml is used."""
-See:
-https://packaging.python.org/en/latest/distributing.html
-https://github.com/pypa/sampleproject
-"""
 
-from setuptools import setup
+import setuptools
-# To use a consistent encoding
-from codecs import open as codecopen
-from os import path
 
-here = path.abspath(path.dirname(__file__))
+if __name__ == "__main__":
-
+    setuptools.setup()
-# Get the long description from the relevant file
-with codecopen(path.join(here, 'README.rst'), encoding='utf-8') as f:
-    long_description = f.read()
-
-setup(
-    name='digimarks', # pip install digimarks
-    description='Simple bookmarking service, using a SQLite database to store bookmarks, supporting tags, automatic title fetching and REST API calls.',
-    #long_description=open('README.md', 'rt').read(),
-    long_description=long_description,
-
-    # version
-    # third part for minor release
-    # second when api changes
-    # first when it becomes stable someday
-    version='1.1.99',
-    author='Michiel Scholten',
-    author_email='michiel@diginaut.net',
-
-    url='https://github.com/aquatix/digimarks',
-    license='Apache',
-
-    # as a practice no need to hard code version unless you know program wont
-    # work unless the specific versions are used
-    install_requires=['Flask', 'Peewee', 'Flask-Peewee', 'requests', 'bs4'],
-
-    py_modules=['digimarks'],
-
-    zip_safe=True,
-)

src/__init__.py

@@ -0,0 +1 @@
+"""digimarks main module."""

src/digimarks/__init__.py

@@ -0,0 +1,3 @@
+"""Top-level package for Digimarks."""
+
+__author__ = """Michiel Scholten"""

src/digimarks/bookmarks_service.py

@@ -0,0 +1,231 @@
+"""Bookmark helper functions, like content scrapers, favicon extractor, updater functions."""
+
+import logging
+from collections.abc import Sequence
+from datetime import UTC, datetime
+from typing import Annotated
+from urllib.parse import urlparse, urlunparse
+
+import bs4
+import httpx
+from extract_favicon import from_html
+from fastapi import Query, Request
+from fastapi.exceptions import HTTPException
+from pydantic import AnyUrl, ValidationError
+from sqlmodel import select
+
+from digimarks import tags_service, utils
+from digimarks.exceptions import BookmarkNotFound
+from digimarks.models import Bookmark, Visibility
+
+DIGIMARKS_USER_AGENT = 'digimarks/2.0.0-dev'
+
+logger = logging.getLogger('digimarks')
+
+
+def get_favicon(html_content: str, root_url: str) -> str:
+    """Fetch the favicon from `html_content` using `root_url`."""
+    favicons = from_html(html_content, root_url=root_url, include_fallbacks=True)
+    for favicon in favicons:
+        print(favicon.url, favicon.width, favicon.height)
+    # TODO: save the preferred image to file and return
+
+
+async def set_information_from_source(bookmark: Bookmark, request: Request) -> Bookmark:
+    """Request the title by requesting the source url."""
+    logger.info('Extracting information from url %s', bookmark.url)
+    try:
+        result = await request.app.state.requests_client.get(
+            str(bookmark.url), headers={'User-Agent': DIGIMARKS_USER_AGENT}
+        )
+        bookmark.http_status = result.status_code
+        logger.info('HTTP status code %s for %s', bookmark.http_status, bookmark.url)
+    except httpx.HTTPError as err:
+        # For example, "MissingSchema: Invalid URL 'abc': No schema supplied. Perhaps you meant http://abc?"
+        logger.error('Exception when trying to retrieve title for %s. Error: %s', bookmark.url, str(err))
+        bookmark.http_status = 404
+        bookmark.title = ''
+        return bookmark
+    if bookmark.http_status == 200 or bookmark.http_status == 202:
+        html_content = bs4.BeautifulSoup(result.text, 'html.parser')
+        try:
+            bookmark.title = html_content.title.text.strip()
+        except AttributeError as exc:
+            logger.error('Error while trying to extract title from URL %s: %s', str(bookmark.url), str(exc))
+            raise HTTPException(status_code=400, detail='Error while trying to extract title')
+
+        url_parts = urlparse(str(bookmark.url))
+        root_url = url_parts.scheme + '://' + url_parts.netloc
+        favicon = get_favicon(result.text, root_url)
+        # filename = os.path.join(settings.media_dir, 'favicons/', domain + file_extension)
+        # with open(filename, 'wb') as out_file:
+        #     shutil.copyfileobj(response.raw, out_file)
+
+        # Extraction was successful
+        logger.info('Extracting information was successful')
+    return bookmark
+
+
+def strip_url_params(url: str) -> str:
+    """Strip URL params from URL.
+
+    :param url: URL to strip URL params from.
+    :return: clean URL
+    :rtype: str
+    """
+    parsed = urlparse(url)
+    return urlunparse((parsed.scheme, parsed.netloc, parsed.path, parsed.params, '', parsed.fragment))
+
+
+async def update_bookmark_with_info(bookmark: Bookmark, request: Request, strip_params: bool = False):
+    """Automatically update title, favicon, etc."""
+    if isinstance(bookmark.url, str):
+        # If type of the url is a 'simple' string, ensure it to be an AnyUrl
+        bookmark.url = AnyUrl(bookmark.url)
+
+    if not bookmark.title:
+        # Title was empty, automatically fetch it from the url, will also update the status code
+        await set_information_from_source(bookmark, request)
+
+    if strip_params:
+        # Strip URL parameters, e.g., tracking params
+        bookmark.url = AnyUrl(strip_url_params(str(bookmark.url)))
+
+    # Sort and deduplicate tags
+    tags_service.set_tags(bookmark, bookmark.tags)
+
+
+async def list_bookmarks_for_user(
+    session,
+    user_key: str,
+    offset: int = 0,
+    limit: Annotated[int, Query(le=10000)] = 100,
+) -> Sequence[Bookmark]:
+    """List all bookmarks in the database. By default, 100 items are returned.
+
+    There is a limit of 10000 items.
+    """
+    result = await session.exec(
+        select(Bookmark)
+        .where(Bookmark.user_key == user_key, Bookmark.status != Visibility.DELETED)
+        .offset(offset)
+        .limit(limit)
+    )
+    bookmarks = result.all()
+    return bookmarks
+
+
+async def get_bookmark_for_user_with_url_hash(session, user_key: str, url_hash: str) -> Bookmark:
+    """Get a bookmark from the database by its URL hash."""
+    result = await session.exec(
+        select(Bookmark).where(
+            Bookmark.user_key == user_key, Bookmark.url_hash == url_hash, Bookmark.status != Visibility.DELETED
+        )
+    )
+    if not result.first():
+        raise BookmarkNotFound(f'url_hash: {url_hash}')
+    return result.first()
+
+
+async def autocomplete_bookmark(
+    session,
+    request: Request,
+    user_key: str,
+    bookmark: Bookmark,
+    strip_params: bool = False,
+) -> Bookmark:
+    """Autofill some fields for this (new) bookmark for user `user_key`."""
+    bookmark.user_key = user_key
+
+    # Auto-fill title, fix tags etc.
+    try:
+        await update_bookmark_with_info(bookmark, request, strip_params)
+    except ValidationError as exc:
+        logger.error('ValidationError while autocompleting bookmark with URL %s', bookmark.url)
+        logger.error('Error was: %s', str(exc))
+        raise HTTPException(status_code=400, detail='Error while autocompleting, likely the URL contained an error')
+
+    url_hash = utils.generate_hash(str(bookmark.url))
+    result = await session.exec(
+        select(Bookmark).where(
+            Bookmark.user_key == user_key, Bookmark.url_hash == url_hash, Bookmark.status != Visibility.DELETED
+        )
+    )
+    bookmark_db = result.first()
+    if bookmark_db:
+        # Bookmark with this URL already exists, provide the hash so the frontend can look it up and the user can
+        # merge them if so wanted
+        bookmark.url_hash = url_hash
+
+    return bookmark
+
+
+async def add_bookmark(
+    session,
+    request: Request,
+    user_key: str,
+    bookmark: Bookmark,
+    strip_params: bool = False,
+) -> Bookmark:
+    """Add new bookmark for user `user_key`."""
+    bookmark.user_key = user_key
+
+    # Auto-fill title, fix tags etc.
+    await update_bookmark_with_info(bookmark, request, strip_params)
+    bookmark.url_hash = utils.generate_hash(str(bookmark.url))
+    logger.info('Adding bookmark %s for user %s', bookmark.url_hash, user_key)
+
+    session.add(bookmark)
+    await session.commit()
+    await session.refresh(bookmark)
+    return bookmark
+
+
+async def update_bookmark(
+    session,
+    request: Request,
+    user_key: str,
+    bookmark: Bookmark,
+    url_hash: str,
+    strip_params: bool = False,
+) -> Bookmark:
+    """Update existing bookmark `bookmark_key` for user `user_key`."""
+    result = await session.exec(
+        select(Bookmark).where(
+            Bookmark.user_key == user_key, Bookmark.url_hash == url_hash, Bookmark.status != Visibility.DELETED
+        )
+    )
+    bookmark_db = result.first()
+    if not bookmark_db:
+        raise BookmarkNotFound(message='Bookmark with hash {url_hash} not found')
+
+    bookmark.modified_date = datetime.now(UTC)
+
+    # 'patch' endpoint, which means that you can send only the data that you want to update, leaving the rest intact
+    bookmark_data = bookmark.model_dump(exclude_unset=True)
+    # Merge the changed fields into the existing object
+    bookmark_db.sqlmodel_update(bookmark_data)
+
+    # Autofill title, fix tags, etc. where (still) needed
+    await update_bookmark_with_info(bookmark, request, strip_params)
+
+    session.add(bookmark_db)
+    await session.commit()
+    await session.refresh(bookmark_db)
+    return bookmark_db
+
+
+async def delete_bookmark(
+    session,
+    user_key: str,
+    url_hash: str,
+) -> None:
+    """(Soft)Delete bookmark `bookmark_key` for user `user_key`."""
+    result = await session.get(Bookmark, {'url_hash': url_hash, 'user_key': user_key})
+    bookmark = result
+    if not bookmark:
+        raise BookmarkNotFound(message='Bookmark with hash {url_hash} not found')
+    bookmark.deleted_date = datetime.now(UTC)
+    bookmark.status = Visibility.DELETED
+    session.add(bookmark)
+    await session.commit()

src/digimarks/exceptions.py

@@ -0,0 +1,25 @@
+"""Exceptions that could be encountered managing digimarks."""
+
+
+class BookmarkNotFound(Exception):
+    """A bookmark was not found."""
+
+    def __init__(self, message: str ='Bookmark not found'):
+        """Initialise the exception.
+
+        :param str message: The message for the exception
+        """
+        super().__init__(message)
+        self.message: str = message
+
+
+class BookmarkAlreadyExists(Exception):
+    """A bookmark already exists for this URL and this user."""
+
+    def __init__(self, message: str ='Bookmark already exists'):
+        """Initialise the exception.
+
+        :param str message: The message for the exception
+        """
+        super().__init__(message)
+        self.message: str = message

src/digimarks/extract.py

@@ -0,0 +1,6 @@
+from pydantic import AnyUrl
+
+
+def extract_contents(title: str, url: AnyUrl, note: str):
+    """Extract contents from a URL."""
+    return

src/digimarks/main.py

@@ -0,0 +1,385 @@
+"""digimarks main module."""
+
+import logging
+from collections.abc import Sequence
+from contextlib import asynccontextmanager
+from datetime import UTC, datetime
+from typing import Annotated, AsyncGenerator, cast
+
+import httpx
+from fastapi import Depends, FastAPI, HTTPException, Query, Request
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi.responses import HTMLResponse
+from fastapi.staticfiles import StaticFiles
+from fastapi.templating import Jinja2Templates
+from pydantic import DirectoryPath, FilePath
+from pydantic_settings import BaseSettings
+from sqlalchemy.ext.asyncio import create_async_engine
+from sqlalchemy.orm import sessionmaker
+from sqlmodel import desc, select
+from sqlmodel.ext.asyncio.session import AsyncSession
+
+from digimarks import bookmarks_service, tags_service
+from digimarks.exceptions import BookmarkNotFound
+from digimarks.models import DEFAULT_THEME, Bookmark, User, Visibility
+
+DIGIMARKS_VERSION = '2.0.0a1'
+
+
+class Settings(BaseSettings):
+    """Configuration needed for digimarks to find its database, favicons, API integrations."""
+
+    # outside the codebase
+    database_file: FilePath
+    favicons_dir: DirectoryPath
+
+    # inside the codebase
+    # static_dir: DirectoryPath = Path('digimarks/static')
+    # template_dir: DirectoryPath = Path('digimarks/templates')
+    static_dir: DirectoryPath = DirectoryPath('digimarks/static')
+    template_dir: DirectoryPath = DirectoryPath('digimarks/templates')
+
+    media_url: str = '/static/'
+
+    system_key: str
+
+    debug: bool = False
+
+
+settings = Settings()
+print(settings.model_dump())
+
+engine = create_async_engine(f'sqlite+aiosqlite:///{settings.database_file}', connect_args={'check_same_thread': False})
+
+
+async def get_session() -> AsyncSession:
+    """SQLAlchemy session factory."""
+    async_session = sessionmaker(bind=engine, class_=AsyncSession, expire_on_commit=False)
+    async with async_session() as session:
+        yield session
+
+
+# Shorter way of getting the DB session in an endpoint
+SessionDep = Annotated[AsyncSession, Depends(get_session)]
+
+
+@asynccontextmanager
+async def lifespan(the_app: FastAPI) -> AsyncGenerator[None, None]:
+    """Upon start, initialise an AsyncClient and assign it to an attribute named requests_client on the app object."""
+    async with httpx.AsyncClient() as requests_client:
+        the_app.state.requests_client = requests_client
+        yield
+        await the_app.state.requests_client.aclose()
+
+
+async def get_requests_client(request: Request) -> httpx.AsyncClient:
+    """Get the httpx client from the application object."""
+    return cast(httpx.AsyncClient, request.app.state.requests_client)
+
+
+# Shorter way of getting the httpx client in an endpoint
+RequestsDep = Annotated[AsyncSession, Depends(get_requests_client)]
+
+app = FastAPI(lifespan=lifespan)
+app.mount('/static', StaticFiles(directory=settings.static_dir), name='static')
+app.mount('/content/favicons', StaticFiles(directory=settings.favicons_dir), name='favicons')
+templates = Jinja2Templates(directory=settings.template_dir)
+
+# Set up logging
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
+    datefmt='%Y-%m-%d %H:%M:%S',
+)
+logger = logging.getLogger('digimarks')
+if settings.debug:
+    logger.setLevel(logging.DEBUG)
+
+# CORS configuration
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=['*'],  # Allow requests from everywhere
+    allow_credentials=True,
+    allow_methods=['*'],
+    allow_headers=['*'],
+)
+
+
+def file_type(filename: str) -> str:
+    """Try to determine the file type for the file in `filename`.
+
+    :param str filename: path to file to check
+    :return: zip file type
+    :rtype: str
+    """
+    magic_dict = {b'\x1f\x8b\x08': 'gz', b'\x42\x5a\x68': 'bz2', b'\x50\x4b\x03\x04': 'zip'}
+
+    max_len = max(len(x) for x in magic_dict)
+
+    with open(filename, 'rb') as f:
+        file_start = f.read(max_len)
+    for magic, filetype in magic_dict.items():
+        if file_start.startswith(magic):
+            return filetype
+    return 'no match'
+
+
+@app.get('/', response_class=HTMLResponse)
+@app.head('/', response_class=HTMLResponse)
+def index(request: Request):
+    """Homepage, point visitors to project page."""
+    logger.info('Root page requested')
+    return templates.TemplateResponse(
+        request=request,
+        name='index.html',
+        context={'language': 'en', 'version': DIGIMARKS_VERSION, 'theme': DEFAULT_THEME},
+    )
+
+
+@app.get('/api/v1/admin/{system_key}/users/{user_id}', response_model=User)
+async def get_user(session: SessionDep, system_key: str, user_id: int) -> User:
+    """Show user information."""
+    logger.info('User %d requested', user_id)
+    if system_key != settings.system_key:
+        logger.error('User %s requested but incorrect system key %s provided', user_id, system_key)
+        raise HTTPException(status_code=404)
+
+    result = await session.get(User, user_id)
+    user = result
+    if not user:
+        logger.error('User %s not found', user_id)
+        raise HTTPException(status_code=404, detail='User not found')
+    return user
+
+
+# @app.get('/admin/{system_key}/users/', response_model=list[User])
+@app.get('/api/v1/admin/{system_key}/users/')
+async def list_users(
+    session: SessionDep,
+    system_key: str,
+    offset: int = 0,
+    limit: Annotated[int, Query(le=100)] = 100,
+) -> Sequence[User]:
+    """List all users in the database.
+
+    :param SessionDep session:
+    :param str system_key: secrit key
+    :param int offset: [Optional] offset of pagination
+    :param int limit: [Optional] limits the number of users to return, defaults to 100
+    :return: list of users in the system
+    :rtype: list[User]
+    """
+    logger.info('User listing requested')
+    if system_key != settings.system_key:
+        logger.error('User listing requested but incorrect system key %s provided', system_key)
+        raise HTTPException(status_code=404)
+
+    result = await session.exec(select(User).offset(offset).limit(limit))
+    return result.all()
+
+
+@app.get('/api/v1/{user_key}/bookmarks/')
+async def list_bookmarks(
+    session: SessionDep,
+    user_key: str,
+    offset: int = 0,
+    limit: Annotated[int, Query(le=10000)] = 100,
+) -> Sequence[Bookmark]:
+    """List all bookmarks in the database. By default, 100 items are returned."""
+    logger.info('List bookmarks for user %s with offset %d, limit %d', user_key, offset, limit)
+    return await bookmarks_service.list_bookmarks_for_user(session, user_key, offset, limit)
+
+
+@app.get('/api/v1/{user_key}/bookmarks/{url_hash}')
+async def get_bookmark(
+    session: SessionDep,
+    user_key: str,
+    url_hash: str,
+) -> Bookmark:
+    """Show bookmark details."""
+    logger.info('Bookmark details for user %s with url_hash %s', user_key, url_hash)
+    try:
+        return await bookmarks_service.get_bookmark_for_user_with_url_hash(session, user_key, url_hash)
+    except BookmarkNotFound as exc:
+        logger.error('Bookmark not found: %s', exc)
+        raise HTTPException(status_code=404, detail=f'Bookmark not found: {exc.message}')
+
+
+@app.post('/api/v1/{user_key}/autocomplete_bookmark/', response_model=Bookmark)
+async def autocomplete_bookmark(
+    session: SessionDep,
+    request: Request,
+    user_key: str,
+    bookmark: Bookmark,
+    strip_params: bool = False,
+) -> Bookmark:
+    """Autofill some fields for this (new) bookmark for user `user_key`."""
+    logger.info('Autocompleting bookmark %s for user %s', bookmark.url_hash, user_key)
+    return await bookmarks_service.autocomplete_bookmark(session, request, user_key, bookmark, strip_params)
+
+
+@app.post('/api/v1/{user_key}/bookmarks/', response_model=Bookmark)
+async def add_bookmark(
+    session: SessionDep,
+    request: Request,
+    user_key: str,
+    bookmark: Bookmark,
+    strip_params: bool = False,
+):
+    """Add new bookmark for user `user_key`."""
+    logger.info('Adding bookmark %s for user %s', bookmark.url, user_key)
+    return await bookmarks_service.add_bookmark(session, request, user_key, bookmark, strip_params)
+
+
+@app.patch('/api/v1/{user_key}/bookmarks/{url_hash}', response_model=Bookmark)
+async def update_bookmark(
+    session: SessionDep,
+    request: Request,
+    user_key: str,
+    bookmark: Bookmark,
+    url_hash: str,
+    strip_params: bool = False,
+):
+    """Update existing bookmark `bookmark_key` for user `user_key`."""
+    logger.info('Updating bookmark %s for user %s', url_hash, user_key)
+    try:
+        return await bookmarks_service.update_bookmark(session, request, user_key, bookmark, url_hash, strip_params)
+    except Exception:
+        logger.exception('Failed to update bookmark %s', bookmark.id)
+        raise HTTPException(status_code=404, detail='Bookmark not found')
+
+
+@app.delete('/api/v1/{user_key}/bookmarks/{url_hash}', response_model=Bookmark)
+async def delete_bookmark(
+    session: SessionDep,
+    user_key: str,
+    url_hash: str,
+):
+    """(Soft)Delete bookmark `bookmark_key` for user `user_key`."""
+    logger.info('Deleting bookmark %s for user %s', url_hash, user_key)
+    try:
+        _ = await bookmarks_service.delete_bookmark(session, user_key, url_hash)
+        return {'ok': True}
+    except Exception:
+        logger.exception('Failed to delete bookmark %s', url_hash)
+        raise HTTPException(status_code=404, detail='Bookmark not found')
+
+
+@app.get('/api/v1/{user_key}/latest_changes/')
+async def bookmarks_changed_since(
+    session: SessionDep,
+    user_key: str,
+):
+    """Last update on server, so the (browser) client knows whether to fetch an update."""
+    logger.info('Retrieving latest changes for user %s', user_key)
+    result = await session.exec(
+        select(Bookmark)
+        .where(Bookmark.user_key == user_key, Bookmark.status != Visibility.DELETED)
+        .order_by(desc(Bookmark.modified_date))
+    )
+    latest_modified_bookmark = result.first()
+    result = await session.exec(
+        select(Bookmark)
+        .where(Bookmark.user_key == user_key, Bookmark.status != Visibility.DELETED)
+        .order_by(desc(Bookmark.created_date))
+    )
+    latest_created_bookmark = result.first()
+
+    # There needs to be at least one bookmark of course
+    if latest_created_bookmark:
+        latest_created_datetime = latest_created_bookmark.created_date
+    else:
+        latest_created_datetime = datetime.min
+
+    # We only have a modified datetime when at least one has been edited
+    if latest_modified_bookmark:
+        latest_modified_datetime = latest_modified_bookmark.modified_date
+    else:
+        latest_modified_datetime = datetime.min
+
+    latest_modification = max(latest_modified_datetime, latest_created_datetime)
+
+    return {
+        'current_time': datetime.now(UTC),
+        'latest_change': latest_modified_datetime,
+        'latest_created': latest_created_datetime,
+        'latest_modification': latest_modification,
+    }
+
+
+@app.get('/api/v1/{user_key}/tags/')
+async def list_tags_for_user(
+    session: SessionDep,
+    user_key: str,
+) -> list[str]:
+    """List all tags in use by the user."""
+    return await tags_service.list_tags_for_user(session, user_key)
+
+
+@app.get('/api/v1/{user_key}/tags/{tag_key}')
+async def list_bookmarks_for_tag_for_user(
+    session: SessionDep,
+    user_key: str,
+    tag_key: str,
+) -> list[str]:
+    """List all tags in use by the user."""
+    logger.info('List bookmarks for tag "%s" by user %s', tag_key, user_key)
+    return await tags_service.list_bookmarks_for_tag_for_user(session, user_key, tag_key)
+
+
+@app.get('/{user_key}', response_class=HTMLResponse)
+async def page_user_landing(
+    session: SessionDep,
+    request: Request,
+    user_key: str,
+) -> HTMLResponse:
+    """HTML page with the main view for the user."""
+    result = await session.exec(select(User).where(User.key == user_key))
+    user = result.first()
+    if not user:
+        raise HTTPException(status_code=404, detail='User not found')
+    language = 'en'
+    return templates.TemplateResponse(
+        request=request,
+        name='user_index.html',
+        context={'language': language, 'version': DIGIMARKS_VERSION, 'user_key': user_key},
+    )
+
+
+# def tags_page(userkey):
+#     """Overview of all tags used by user"""
+#     tags = get_cached_tags(userkey)
+#     alltags = []
+#     for tag in tags:
+#         try:
+#             publictag = PublicTag.get(PublicTag.userkey == userkey, PublicTag.tag == tag)
+#         except PublicTag.DoesNotExist:
+#             publictag = None
+#
+#         total = (
+#             Bookmark.select()
+#             .where(Bookmark.userkey == userkey, Bookmark.tags.contains(tag), Bookmark.status == Bookmark.VISIBLE)
+#             .count()
+#         )
+#         alltags.append({'tag': tag, 'publictag': publictag, 'total': total})
+#     totaltags = len(alltags)
+#     totalbookmarks = Bookmark.select().where(Bookmark.userkey == userkey, Bookmark.status == Bookmark.VISIBLE).count()
+#     totalpublic = PublicTag.select().where(PublicTag.userkey == userkey).count()
+#     totalstarred = Bookmark.select().where(Bookmark.userkey == userkey, Bookmark.starred).count()
+#     totaldeleted = Bookmark.select().where(Bookmark.userkey == userkey, Bookmark.status == Bookmark.DELETED).count()
+#     totalnotes = Bookmark.select().where(Bookmark.userkey == userkey, Bookmark.note != '').count()
+#     totalhttperrorstatus = Bookmark.select().where(Bookmark.userkey == userkey, Bookmark.http_status != 200).count()
+#     theme = get_theme(userkey)
+#     return render_template(
+#         'tags.html',
+#         tags=alltags,
+#         totaltags=totaltags,
+#         totalpublic=totalpublic,
+#         totalbookmarks=totalbookmarks,
+#         totaldeleted=totaldeleted,
+#         totalstarred=totalstarred,
+#         totalhttperrorstatus=totalhttperrorstatus,
+#         totalnotes=totalnotes,
+#         userkey=userkey,
+#         theme=theme,
+#     )

src/digimarks/models.py

@@ -0,0 +1,107 @@
+"""Models for digimarks.
+
+Contains the bookmarks administration, users, tags, public tags and more.
+"""
+
+from datetime import UTC, datetime
+from http import HTTPStatus
+from typing import TypeVar
+
+from pydantic import AnyUrl, computed_field
+from sqlmodel import AutoString, Field, SQLModel
+
+DEFAULT_THEME = 'freshgreen'
+
+
+class User(SQLModel, table=True):
+    """User account."""
+
+    id: int = Field(primary_key=True)
+    username: str
+    key: str
+    theme: str = Field(default=DEFAULT_THEME)
+    created_date: datetime
+
+
+class Visibility:
+    """Options for visibility of an object."""
+
+    VISIBLE: int = 0
+    DELETED: int = 1
+    HIDDEN: int = 2
+
+
+# Type var used for building custom types for the DB
+T = TypeVar('T')
+
+
+def build_custom_type(internal_type: type[T]) -> type[AutoString]:
+    """Create a type that is compatible with the database.
+
+    Based on https://github.com/fastapi/sqlmodel/discussions/847
+    """
+
+    class CustomType(AutoString):
+        def process_bind_param(self, value, dialect) -> str | None:
+            if value is None:
+                return None
+
+            if isinstance(value, str):
+                # Test if value is valid to avoid `process_result_value` failing
+                try:
+                    internal_type(value)  # type: ignore[call-arg]
+                except ValueError as e:
+                    raise ValueError(f'Invalid value for {internal_type.__name__}: {e}') from e
+
+            return str(value)
+
+        def process_result_value(self, value, dialect) -> T | None:
+            if value is None:
+                return None
+
+            return internal_type(value)  # type: ignore[call-arg]
+
+    return CustomType
+
+
+class Bookmark(SQLModel, table=True):
+    """Bookmark object."""
+
+    id: int = Field(primary_key=True)
+    user_key: str = Field(foreign_key='user.key', nullable=False)
+    title: str = Field(default='', nullable=False)
+    url: AnyUrl = Field(default='', sa_type=build_custom_type(AnyUrl))
+    note: str = Field(default='', nullable=True)
+    # image: str = Field(default='')
+    url_hash: str = Field(default='', nullable=False)
+    tags: str = Field(default='')
+    starred: bool = Field(default=False)
+
+    favicon: str | None = Field(default=None)
+
+    http_status: int = Field(default=HTTPStatus.OK)
+
+    created_date: datetime = Field(default=datetime.now(UTC))
+    modified_date: datetime = Field(default=None, nullable=True)
+    deleted_date: datetime = Field(default=None, nullable=True)
+
+    status: int = Field(default=Visibility.VISIBLE)
+
+    @computed_field
+    @property
+    def tag_list(self) -> list[str]:
+        """The tags but as a proper list."""
+        if self.tags:
+            return self.tags.split(',')
+        # Not tags, return empty list instead of [''] that split returns in that case
+        return []
+
+
+class PublicTag(SQLModel, table=True):
+    """Public tag object."""
+
+    id: int = Field(primary_key=True)
+    tag_key: str
+    user_key: str = Field(foreign_key='user.key')
+    tag: str
+    created_date: datetime = Field(default=datetime.now(UTC))

src/digimarks/static/css/digimarks.css

@@ -0,0 +1,33 @@
+/**
+ * digimarks styling
+ *
+ * Overrides on and additions to the digui styling
+ */
+
+/* Star, error, note etc */
+.card .statuses {
+    display: flex;
+    flex-direction: column;
+}
+
+.star {
+    color: #ffeb3b;
+}
+
+.thumbnail {
+    /*width: 80px;*/
+    width: 66px;
+}
+
+.thumbnail img {
+    /*width: 72px;*/
+    width: 60px;
+}
+
+#bookmarkEditForm fieldset {
+    border: none;
+}
+
+#bookmarkEditForm fieldset input, #bookmarkEditForm textarea, #bookmarkEditForm select, #bookmarkEditForm label {
+    width: 100%;
+}

src/digimarks/static/css/digui.css

@@ -0,0 +1,503 @@
+/**
+ * digui structure and theming
+ * v0.0.2
+ *
+ * Created by: Michiel Scholten
+ * Source: https://codeberg.org/diginaut/digui
+ */
+
+/** Colours and themes */
+
+:root {
+    --padding: .5rem;
+
+    color-scheme: only light;
+
+    /* Default is nebula-light */
+    --font-family: sans-serif;
+    --background-color: #fff;
+    --background-color-secondary: #ccc;
+    --button-color: #eee;
+    --button-text: var(--text-color);
+    --text-color: #121416d8;
+    --text-color-secondary: #121416d8;
+    --text-color-muted: #d5d9d9;
+    --link-color: #543fd7;
+
+    --nav-background-color: #FFF;
+    --nav-color: var(--text-color);
+
+    --border-color: #d5d9d9;
+    --border-width: 1px;
+    --border-radius: 8px;
+    --chip-border-radius: 2rem;
+    --shadow-color: rgba(213, 217, 217, .5);
+    --global-theme-toggle-content: ' 🌞';
+
+    /* E.g., an active button */
+    --color-highlight: #fb8c00;
+    /* Generic colors */
+    /*--color-danger: #e03131;*/
+    --color-danger: var(--color-red);
+    --color-warning: var(--color-yellow);
+    --color-error: var(--color-danger);
+    /*--color-ok: #31e031;*/
+    --color-ok: var(--color-green);
+
+    /* Argonaut colours */
+    --color-black: #000000;
+    --color-red: #FF000F;
+    --color-green: #8CE10B;
+    --color-yellow: #FFB900;
+    --color-blue: #008DF8;
+    --color-purple: #6D43A6;
+    --color-cyan: #00D8EB;
+    --color-white: #FFFFFF;
+}
+
+html[data-theme='nebula'] {
+    /* Default theme, see :root element */
+}
+
+html[data-theme='nebula-dark'] {
+    color-scheme: dark;
+
+    --background-color: #29292c;
+    --background-color-secondary: #29292c;
+    --button-color: #29292c;
+    --button-text: var(--text-color);
+    --text-color: #F7F8F8;
+    --text-color-secondary: #ddd;
+    --text-color-muted: #F7F8F8;
+    --link-color: #ffe7a3;
+    --color-highlight: #e03131;
+
+    --nav-background-color: #FF9800;
+    --nav-color: var(--text-color);
+
+    --border-color: #333;
+    --border-width: 1px;
+    --border-radius: 8px;
+    --chip-border-radius: 2rem;
+    --shadow-color: rgba(3, 3, 3, .5);
+    --global-theme-toggle-content: ' 🌝';
+}
+
+html[data-theme='bbs'] {
+    --font-family: monospace;
+    --background-color: #FFF;
+    --background-color-secondary: #ccc;
+    --button-color: #FFFFFF;
+    --button-text: var(--text-color);
+    --text-color: #000;
+    --text-color-secondary: #000;
+    --text-color-muted: #000;
+    --link-color: #543fd7;
+    --color-highlight: #e03131;
+
+    /*--nav-background-color: #ccc;*/
+    /*--nav-color: var(--text-color);*/
+
+    --border-color: #333;
+    --border-width: 2px;
+    --border-radius: 0;
+    --chip-border-radius: 0;
+    --global-theme-toggle-content: ' 🖥️';
+}
+
+html[data-theme='silo'] {
+    --font-family: monospace;
+    /*--background-color: #003eaa;*/
+    --background-color: #1d212c;
+    --background-color-secondary: var(--color-highlight);
+    --button-color: #FFFFFF;
+    --button-text: var(--text-color);
+    --text-color: #FFF;
+    --text-color-secondary: #29292c;
+    --text-color-muted: #FFF;
+    --link-color: #FF9800;
+    --color-highlight: #23B0FF;
+
+    /*--nav-background-color: #003eaa;*/
+    /*--nav-background-color: #23B0FF;*/
+    /*--nav-color: var(--text-color);*/
+    --nav-background-color: var(--background-color);
+
+    --border-color: #23B0FF;
+    /*--border-color: #003eaa;*/
+    --border-width: 2px;
+    --border-radius: 0;
+    --chip-border-radius: 0;
+    --global-theme-toggle-content: ' ⌨️';
+}
+
+/* AlpineJS blip-preventer */
+[x-cloak] {
+    display: none !important;
+}
+
+/** Main structure */
+
+body {
+    background: var(--background-color);
+    color: var(--text-color);
+    height: 125vh;
+    font-family: var(--font-family), sans-serif;
+    margin-top: 3rem;
+    /*padding: 30px;*/
+}
+
+main {
+    color: var(--text-color);
+    padding-top: .5em;
+}
+
+/* Navigation */
+
+header {
+    background-color: var(--nav-background-color);
+    position: fixed;
+    top: 0;
+    left: 0;
+    right: 0;
+    height: 3rem;
+    display: flex;
+    align-items: center;
+}
+
+[data-theme='nebula'] header,
+[data-theme='nebula-dark'] header {
+    /*box-shadow: 0 0 5px 0 rgba(213, 217, 217, .5);*/
+    /*box-shadow: 0 0 5px 0 #999;*/
+    box-shadow: 0 0 5px var(--shadow-color);
+}
+
+header * {
+    display: inline;
+}
+
+header li {
+    /*margin: 10px;*/
+}
+
+header li a {
+    color: black;
+    text-decoration: none;
+}
+
+header li h1 {
+    font-weight: bold;
+    font-size: 1.2rem;
+    vertical-align: middle;
+    margin-right: 3rem;
+}
+
+[data-theme='silo'] header nav::after {
+    content: '';
+    background: repeating-linear-gradient(90deg, #23B0FF, #23B0FF 2px, transparent 0, transparent 10px);
+    display: block;
+    width: 100%;
+    right: 10px;
+}
+
+[data-theme='silo'] header {
+    border-bottom: 3px dotted #23B0FF;
+}
+
+/** Generic elements */
+
+/* https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/Heading_Elements#specifying_a_uniform_font_size_for_h1 */
+h1 {
+    margin-block: 0.67em;
+    font-size: 2em;
+}
+
+a, a:hover, a:visited, a:active, a.button, a.button:hover, a.button:active, a.button:visited {
+    text-decoration: none;
+}
+
+a {
+    color: var(--link-color);
+}
+
+a:hover {
+    text-decoration: underline;
+    filter: brightness(80%);
+}
+
+ol li::marker, ul li::marker {
+    color: var(--text-color-muted);
+}
+
+/* Active element, e.g. a button */
+.active {
+    background-color: var(--color-highlight);
+    color: var(--text-color);
+    transition-duration: 0.2s;
+}
+
+/* Special button */
+.theme-toggle::after {
+    content: var(--global-theme-toggle-content);
+}
+
+/* Buttons */
+
+button, .button, input, select, textarea {
+    box-sizing: border-box;
+    color: var(--text-color-secondary);
+    background-color: var(--button-color);
+    cursor: pointer;
+    display: inline-block;
+    font-size: 13px;
+    /*line-height: 29px;*/
+    /*padding: 0 10px 0 11px;*/
+    position: relative;
+    text-align: left;
+    text-decoration: none;
+    user-select: none;
+    -webkit-user-select: none;
+    touch-action: manipulation;
+    vertical-align: middle;
+    transition-duration: 0.2s;
+}
+
+button, .button, input, select, textarea, table {
+    border: var(--border-width) solid var(--border-color);
+    border-radius: var(--border-radius);
+    padding: .5rem .5rem;
+}
+
+button:hover, .button:hover {
+    /*background-color: #f7fafa;*/
+    /*background-color: #d57803;*/
+    background-color: var(--color-highlight);
+    filter: brightness(80%);
+    transition-duration: 0.2s;
+}
+
+button:focus, .button:focus {
+    /*border-color: #008296;*/
+    /*box-shadow: rgba(213, 217, 217, .5) 0 2px 5px 0;*/
+    outline: 0;
+    /*border-color: #d57803;*/
+}
+
+.btn-dangerous {
+    background: var(--color-danger);
+}
+
+.btn-dangerous:hover {
+    background: var(--color-danger);
+    filter: brightness(80%);
+}
+
+.btn-warning {
+    background-color: var(--color-warning);
+}
+
+.btn-warning:hover {
+    background-color: var(--color-warning);
+    filter: brightness(80%);
+}
+
+.btn-ok {
+    background: var(--color-ok);
+}
+
+.btn-ok:hover {
+    background: var(--color-ok);
+    filter: brightness(80%);
+}
+
+/* Toggle buttons */
+
+.button-group {
+    display: inline-flex;
+    overflow: hidden;
+    border: var(--border-width) solid var(--border-color);
+    border-radius: var(--border-radius);
+}
+
+.button-group button {
+    /* Reset borders because the buttons are mashed together and the group has its own border */
+    border: none;
+    border-radius: 0;
+    box-shadow: none !important;
+}
+
+/* Table */
+
+th {
+    text-align: left;
+}
+
+th, td {
+    padding: 0 0.3rem;
+}
+
+/* Cards */
+
+.cards {
+    display: grid;
+    grid-column-gap: 1rem;
+    grid-row-gap: 1rem;
+    grid-template-columns: repeat(auto-fit, minmax(400px, 1fr));
+}
+
+.card {
+    display: inline-grid;
+    border-radius: var(--border-radius);
+    border: var(--border-width) solid var(--border-color);
+    /*margin: 1em;*/
+    /*padding: 1em;*/
+}
+
+[data-theme='nebula'] :modal,
+[data-theme='nebula'] .card,
+[data-theme='nebula'] button,
+[data-theme='nebula'] .button,
+[data-theme='nebula'] .button-group,
+[data-theme='nebula'] input,
+[data-theme='nebula'] select,
+[data-theme='nebula'] textarea,
+[data-theme='nebula'] table,
+[data-theme='nebula-dark'] :modal,
+[data-theme='nebula-dark'] .card,
+[data-theme='nebula-dark'] button,
+[data-theme='nebula-dark'] .button,
+[data-theme='nebula-dark'] .button-group,
+[data-theme='nebula-dark'] input,
+[data-theme='nebula-dark'] select,
+[data-theme='nebula-dark'] textarea,
+[data-theme='nebula-dark'] table {
+    box-shadow: var(--shadow-color) 0 2px 5px 0;
+}
+
+.card .card-header {
+
+}
+
+.card-body {
+    display: flex;
+    flex-direction: row;
+    flex-wrap: nowrap;
+    gap: 1rem;
+    padding: 1em;
+}
+
+.card-body > * {
+    /*padding-left: 1em;*/
+}
+
+.card .card-image {
+    width: 100%;
+}
+
+.card .card-image img {
+    width: 100%;
+    border-radius: var(--border-radius);
+}
+
+.card .card-thumb {
+    width: 72px;
+    /*min-width: 60px;*/
+    /*max-width: 100px;*/
+    /*position: relative;*/
+    /*box-sizing: inherit;*/
+}
+
+.card .card-thumb img {
+    width: 72px;
+}
+
+.card .card-action {
+    padding: .5em;
+}
+
+.card .meta {
+    filter: brightness(80%);
+    color: var(--text-color);
+}
+
+.card .card-footer {
+    display: flex;
+    flex-direction: row-reverse;
+    gap: 1rem;
+    align-items: center;
+
+    padding: .3em;
+}
+
+.card-footer h1, .card-footer h2, .card-footer h3, .card-footer h4, .card-footer h5, .card-footer h6 {
+    margin: 0;
+}
+
+/*
+.card button {
+    border: none;
+    background: none;
+}
+*/
+
+/* Tags/chips */
+
+.chip {
+    font-size: .8rem;
+    border-radius: var(--chip-border-radius);
+    background-color: var(--background-color-secondary);
+    color: var(--text-color-secondary);
+    /*color: var(--text-color);*/
+    padding: .2rem .5rem;
+    margin-left: .5rem;
+}
+
+.chip .button {
+    border-radius: var(--chip-border-radius);
+}
+
+/* Status */
+.error {
+    color: var(--color-error);
+}
+
+
+/** Modal, e.g. for showing info or filling in a form; on top of the other content */
+
+dialog:modal {
+    color: var(--text-color);
+    background-color: var(--background-color);
+    /*background-color: var(--nav-background-color);*/
+    border: 2px solid var(--border-color);
+    border-radius: var(--border-radius);
+    width: 90%;
+    /*height: 90%;*/
+    overflow: auto;
+}
+
+/* The umwelt of the modal, on top of the regular content */
+dialog::backdrop {
+    background-color: rgba(0, 0, 0, 0.5);
+}
+
+/** Footer */
+
+footer {
+    /*background-color: var(--secondary-background-color);*/
+    margin-top: 1rem;
+    padding: 2rem 1rem;
+
+    display: grid;
+    grid-column-gap: 1rem;
+    grid-row-gap: 1rem;
+    grid-template-columns: repeat(auto-fit, minmax(400px, 1fr));
+}
+
+footer .column {
+    display: inline-grid;
+}
+
+footer h2 {
+    text-align: left;
+    margin-bottom: 0;
+}

src/digimarks/static/js/digimarks.js

@@ -0,0 +1,310 @@
+document.addEventListener('alpine:init', () => {
+    Alpine.store('digimarks', {
+        /** Main digimarks application, state etc */
+        userKey: -1,
+        /* cache consists of cache[userKey] = {'bookmarks': [], 'tags': [], ??} */
+        cache: Alpine.$persist({}).as('cache'),
+
+        bookmarks: [],
+
+        /* nebula (drop-shadows), bbs (monospace, right lines), silo (like bbs but dark) ?? */
+        themes: ['nebula', 'nebula-dark', 'bbs', 'silo'],
+        theme: Alpine.$persist('nebula').as('theme'),
+
+        showBookmarks: Alpine.$persist(true).as('showBookmarks'),
+        showBookmarksList: Alpine.$persist(true).as('showBookmarksList'),
+        showBookmarksCards: Alpine.$persist(false).as('showBookmarksCards'),
+        showTags: Alpine.$persist(false).as('showTags'),
+        /* Bookmark that is being edited, used to fill the form, etc. */
+        bookmarkToEdit: Alpine.$persist({}).as('bookmarkToEdit'),
+        bookmarkToEditError: null,
+        bookmarkToEditVisible: false,
+
+        /* Loading indicator */
+        loading: false,
+
+        /* Search filter */
+        search: '',
+        /* Show bookmarks with this tag/these tags */
+        tagsFilter: [],
+        /* Hide bookmarks with these tags */
+        tagsToHide: Alpine.$persist([]).as('tags_to_hide'),
+
+        /* Sort on ~ */
+        sortTitleAsc: Alpine.$persist(false).as('sortTitleAsc'),
+        sortTitleDesc: Alpine.$persist(false).as('sortTitleDesc'),
+        sortCreatedAsc: Alpine.$persist(false).as('sortCreatedAsc'),
+        sortCreatedDesc: Alpine.$persist(false).as('sortCreatedDesc'),
+
+        async init() {
+            /** Initialise the application after loading */
+            document.documentElement.setAttribute('data-theme', this.theme);
+            console.log('Set theme', this.theme);
+
+            /* Make sure the edit/add bookmark form has a fresh empty object */
+            this.resetEditBookmark();
+            /* Bookmarks are refreshed through the getBookmarks() call in the HTML page */
+            /* await this.getBookmarks(); */
+            setInterval(() => {
+                // Update counter to next game (midnight UTC, fetched from API) every second
+                // this.countDownTimer();
+            }, 1000);
+        },
+
+        async loopToNextTheme() {
+            /* Loop through themes */
+            let currentThemeIndex = this.themes.indexOf(this.theme);
+            if (currentThemeIndex + 1 >= this.themes.length) {
+                currentThemeIndex = 0
+            } else {
+                currentThemeIndex++;
+            }
+            this.theme = this.themes[currentThemeIndex];
+            console.log('Switching to theme', this.theme)
+            document.documentElement.setAttribute('data-theme', this.theme);
+            /* Optionally, change the theme CSS file too */
+            // document.getElementById('theme-link').setAttribute('href', 'digui-theme-' + this.theme + '.css');
+        },
+
+        async loadCache() {
+            /* Load bookmarks and tags from cache */
+            if (this.userKey in this.cache) {
+                console.log('Loading bookmarks from cache for user "' + this.userKey + '"');
+                this.filterBookmarksByTags();
+            }
+        },
+        async getBookmarks() {
+            /** Get the bookmarks from the backend */
+            this.loading = true;
+            if (!(this.userKey in this.cache)) {
+                /* There is no cache for this userKey yet, create on */
+                console.log('Creating cache for user "' + this.userKey + '"');
+                this.cache[this.userKey] = {'bookmarks': [], 'latest_changes': {}};
+            }
+
+            let latestStatusResponse = await fetch('/api/v1/' + this.userKey + '/latest_changes/');
+            let latestStatusResult = await latestStatusResponse.json();
+            let shouldFetch = false;
+            let latestModificationInCache = this.cache[this.userKey].latest_changes.latest_modification || "0000-00-00";
+            shouldFetch = latestStatusResult.latest_modification > latestModificationInCache;
+            this.cache[this.userKey].latest_changes = latestStatusResult;
+
+            if (!shouldFetch) {
+                console.log('Cache is up-to-date');
+                this.loading = false;
+                return;
+            }
+
+            console.log('Fetching latest bookmarks from backend for user "' + this.userKey + '"...');
+            /* At the moment, request 'a lot' bookmarks; likely all of them in one go; paging tbd if needed */
+            let response = await fetch('/api/v1/' + this.userKey + '/bookmarks/?limit=10000');
+            /* Cache the bookmarks to Local Storage */
+            this.cache[this.userKey]['bookmarks'] = await response.json();
+
+            let tagsResponse = await fetch('/api/v1/' + this.userKey + '/tags/');
+            this.cache[this.userKey]['tags'] = await tagsResponse.json();
+
+            /* Filter bookmarks by (blacklisted) tags */
+            this.filterBookmarksByTags();
+            this.loading = false;
+        },
+
+        hasTag(tagList, filterList) {
+            /* Looks for the items in filterList and returns True when one appears on the tagList */
+            if (tagList === undefined) {
+                return false;
+            }
+            for (let tag in filterList) {
+                if (tagList.includes(tag))
+                    return true;
+            }
+            return false;
+        },
+        filterBookmarksByTags() {
+            /* Filter away bookmarks with a tag on the 'blacklist' */
+
+            /* First make a shallow copy of all bookmarks */
+            let prefilteredBookmarks = [...this.cache[this.userKey]['bookmarks']] || [];
+            if (this.tagsToHide.length > 0) {
+                console.log('Filtering away bookmarks containing blacklisted tags');
+                this.bookmarks = prefilteredBookmarks.filter(
+                    i => !this.hasTag(i.tag_list, this.tagsToHide)
+                )
+            } else {
+                this.bookmarks = prefilteredBookmarks;
+            }
+            this.sortBookmarks();
+        },
+        get filteredBookmarks() {
+            /* Get the bookmarks, optionally filtered by search text or tag black-/whitelists */
+
+            /* Use 'bookmarks' and not the cache, as it can already be pre-filtered */
+            if (this.search === '') {
+                /* No need to filter, quickly return the set */
+                return this.bookmarks;
+            }
+            return this.bookmarks.filter(
+                i => i.title.match(new RegExp(this.search, "i"))
+            )
+        },
+        get filteredTags() {
+            if (this.cache[this.userKey].tags === undefined) {
+                console.log('Tags not yet cached');
+                return [];
+            }
+            /* Search in the list of all tags */
+            return this.cache[this.userKey].tags.filter(
+                i => i.match(new RegExp(this.search, "i"))
+            )
+        },
+
+        sortBookmarks() {
+            /* Sort the bookmarks according to the setting */
+            if (this.sortTitleAsc) {
+                this.bookmarks.sort((a, b) => a.title.localeCompare(b.title));
+            } else if (this.sortTitleDesc) {
+                this.bookmarks.sort((a, b) => b.title.localeCompare(a.title));
+            } else if (this.sortCreatedAsc) {
+                this.bookmarks.sort((a, b) => a.created_date.localeCompare(b.created_date));
+            } else if (this.sortCreatedDesc) {
+                this.bookmarks.sort((a, b) => b.created_date.localeCompare(a.created_date));
+            }
+        },
+        async sortAlphabetically(order = 'asc') {
+            /* Sort the bookmarks (reverse) alphabetically, based on 'asc' or 'desc' */
+            this.loading = true;
+            this.sortCreatedAsc = false;
+            this.sortCreatedDesc = false;
+            this.sortTitleAsc = false;
+            this.sortTitleDesc = false;
+            if (order === 'desc') {
+                this.sortTitleDesc = true;
+            } else {
+                this.sortTitleAsc = true;
+            }
+            this.sortBookmarks();
+            this.loading = false;
+        },
+        async sortCreated(order = 'asc') {
+            /* Sort the bookmarks (reverse) chronologically, based on 'asc' or 'desc' */
+            this.loading = true;
+            this.sortCreatedAsc = false;
+            this.sortCreatedDesc = false;
+            this.sortTitleAsc = false;
+            this.sortTitleDesc = false;
+            if (order === 'desc') {
+                this.sortCreatedDesc = true;
+            } else {
+                this.sortCreatedAsc = true;
+            }
+            this.sortBookmarks();
+            this.loading = false;
+        },
+
+        async toggleTagPage() {
+            /* Show or hide the tag page instead of the bookmarks */
+            this.showBookmarks = !this.showBookmarks;
+            this.showTags = !this.showBookmarks;
+        },
+        async toggleListOrGrid() {
+            /* Toggle between 'list' or 'grid' (cards) view */
+            this.showBookmarksList = !this.showBookmarksList;
+            this.showBookmarksCards = !this.showBookmarksList;
+        },
+
+        resetEditBookmark() {
+            this.bookmarkToEdit = {
+                'url_hash': '',
+                'url': '',
+                'title': '',
+                'note': '',
+                'tags': '',
+                'http_status': 0,
+                'strip_params': false
+            }
+        },
+        async startAddingBookmark() {
+            /* Open 'add bookmark' page */
+            console.log('Open "add bookmark" modal');
+            this.resetEditBookmark();
+            // this.show_bookmark_details = true;
+            const editFormDialog = document.getElementById("editFormDialog");
+            this.bookmarkToEditVisible = true;
+            editFormDialog.showModal();
+        },
+        async bookmarkURLChanged() {
+            console.log('Bookmark URL changed');
+            // let response = await fetch('/api/v1/' + this.userKey + '/autocomplete_bookmark/');
+            try {
+                const response = await fetch('/api/v1/' + this.userKey + '/autocomplete_bookmark/?strip_params=' + this.bookmarkToEdit.strip_params, {
+                    method: 'POST',
+                    headers: {
+                        'Content-Type': 'application/json'
+                    },
+                    body: JSON.stringify({
+                        // Bookmark form data
+                        url_hash: this.bookmarkToEdit.url_hash,
+                        url: this.bookmarkToEdit.url,
+                        title: this.bookmarkToEdit.title,
+                        note: this.bookmarkToEdit.note,
+                        tags: this.bookmarkToEdit.tags
+                    })
+                });
+                const data = await response.json();
+                // TODO: update form fields if needed (auto-fetched title for example
+                console.log('Got response');
+                console.log(response);
+                console.log(data);
+                if (response.ok) {
+                    this.bookmarkToEdit.url_hash = data.url_hash;
+                    this.bookmarkToEdit.url = data.url;
+                    this.bookmarkToEdit.title = data.title;
+                    this.bookmarkToEdit.note = data.note;
+                    this.bookmarkToEdit.tags = data.tags;
+                    this.bookmarkToEdit.http_status = data.http_status;
+                } else {
+                    console.log('Error occurred');
+                    this.bookmarkToEditError = data.detail;
+                }
+            } catch (error) {
+                // enter logic for when there is an error (ex. error toast)
+                console.log('error occurred');
+                console.log(error);
+                this.bookmarkToEditError = error.detail;
+                console.log('yesssh?');
+            }
+        },
+        async saveBookmark() {
+            console.log('Saving bookmark');
+            // this.bookmarkToEditVisible = false;
+            // this.show_bookmark_details = false;
+        },
+        async addBookmark() {
+            /* Post new bookmark to the backend */
+            console.log('Adding bookmark');
+            try {
+                const response = await fetch('/api/v1/' + this.userKey + '/add_bookmark/?strip_params=' + this.bookmarkToEdit.strip_params, {
+                    method: 'POST',
+                    headers: {
+                        'Content-Type': 'application/json'
+                    },
+                    body: JSON.stringify({
+                        // Bookmark form data
+                        url: this.bookmarkToEdit.url,
+                        title: this.bookmarkToEdit.title,
+                        note: this.bookmarkToEdit.note,
+                        tags: this.bookmarkToEdit.tags
+                    })
+                });
+                const data = await response.json();
+                // TODO: update form fields if needed (auto-fetched title for example
+                console.log(data);
+                // this.bookmarkToEditError = 'lolwut';
+            } catch (error) {
+                // enter your logic for when there is an error (ex. error toast)
+
+                console.log(error)
+            }
+        }
+    })
+});

src/digimarks/tags_service.py

@@ -0,0 +1,102 @@
+"""Helper functions for tags used with Bookmark models."""
+
+from sqlalchemy import Sequence
+from sqlmodel import select
+
+from digimarks.models import Bookmark, Visibility
+
+
+def i_filter_false(predicate, iterable):
+    """Filter an iterable if predicate returns True.
+
+    i_filter_false(lambda x: x%2, range(10)) --> 0 2 4 6 8
+    """
+    if predicate is None:
+        predicate = bool
+    for x in iterable:
+        if not predicate(x):
+            yield x
+
+
+def unique_ever_seen(iterable, key=None):
+    """List unique elements, preserving order. Remember all elements ever seen.
+
+    unique_ever_seen('AAAABBBCCDAABBB') --> A B C D
+    unique_ever_seen('ABBCcAD', str.lower) --> A B C D
+    """
+    seen = set()
+    seen_add = seen.add
+    if key is None:
+        for element in i_filter_false(seen.__contains__, iterable):
+            seen_add(element)
+            yield element
+    else:
+        for element in iterable:
+            k = key(element)
+            if k not in seen:
+                seen_add(k)
+                yield element
+
+
+def clean_tags(tags_list: list) -> list[str]:
+    """Generate a unique list of the tags.
+
+    :param list tags_list: List with all tags
+    :return: deduplicated list of the tags, without leading or trailing whitespace
+    :rtype: list
+    """
+    tags_res = [x.strip() for x in tags_list]
+    tags_res = list(unique_ever_seen(tags_res))
+    tags_res.sort()
+    if tags_res and tags_res[0] == '':
+        del tags_res[0]
+    return tags_res
+
+
+def list_tags_for_bookmarks(bookmarks: Sequence[Bookmark]) -> list[str]:
+    """Generate a unique list of the tags from the list of bookmarks.
+
+    :param Sequence[Bookmark] bookmarks: List of bookmarks to create the list of tags from
+    """
+    tags = []
+    for bookmark in bookmarks:
+        tags += bookmark.tag_list
+    return clean_tags(tags)
+
+
+def set_tags(bookmark: Bookmark, new_tags: str) -> None:
+    """Set tags from `tags`, strip and sort them.
+
+    :param Bookmark bookmark: Bookmark to modify
+    :param str new_tags: New tags to sort and set.
+    """
+    tags_split = new_tags.split(',')
+    tags_clean = clean_tags(tags_split)
+    bookmark.tags = ','.join(tags_clean)
+
+
+async def list_tags_for_user(
+    session,
+    user_key: str,
+) -> list[str]:
+    """List all tags in use by the user."""
+    result = await session.exec(
+        select(Bookmark).where(Bookmark.user_key == user_key, Bookmark.status != Visibility.DELETED)
+    )
+    bookmarks = result.all()
+    tags = []
+    for bookmark in bookmarks:
+        tags += bookmark.tag_list
+    return clean_tags(tags)
+
+
+async def list_bookmarks_for_tag_for_user(
+    session,
+    user_key: str,
+    tag_key: str,
+) -> list[str]:
+    """List all tags in use by the user."""
+    result = await session.exec(select(Bookmark).where(Bookmark.user_key == user_key))
+    # TODO: filter on tag_key
+    bookmarks = result.all()
+    return list_tags_for_bookmarks(bookmarks)

src/digimarks/templates/404.html

@@ -0,0 +1,6 @@
+{% extends "base.html" %}
+{% block title %}404: Page not found{% endblock %}
+{% block page_header %}404: Page not found{% endblock %}
+{% block page_content %}
+    The page you requested was not found.
+{% endblock %}

src/digimarks/templates/base.html

@@ -0,0 +1,35 @@
+<!doctype html>
+<html lang="{{ language }}">
+<head>
+    <title>{% block title %}{% endblock %} - digimarks</title>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css">
+    <link rel="stylesheet" href="/static/css/digui.css?v={{ version }}">
+    <link rel="stylesheet" href="/static/css/digimarks.css?v={{ version }}">
+    <link id="favicon" rel="icon" type="image/x-icon" href="/static/images/favicon.ico">
+    <link rel="apple-touch-icon" sizes="180x180" href="/static/images/apple-touch-icon.png">
+    <link rel="icon" type="image/png" sizes="32x32" href="/static/images/favicon-32x32.png">
+    <link rel="icon" type="image/png" sizes="16x16" href="/static/images/favicon-16x16.png">
+    <link rel="manifest" href="/static/images/site.webmanifest">
+    <script src="https://cdn.jsdelivr.net/npm/@alpinejs/persist@3.x.x/dist/cdn.min.js"></script>
+{#-
+    <script src="https://cdn.jsdelivr.net/npm/@ryangjchandler/alpine-clipboard@2.x.x/dist/alpine-clipboard.js"
+            defer></script>
+#}
+    <script defer src="https://cdn.jsdelivr.net/npm/alpinejs@3.x.x/dist/cdn.min.js"></script>
+</head>
+<body>
+{% block page_content %}
+    <div id="container" x-data="">
+
+
+    </div>
+{% endblock %}
+
+<!-- Scripts -->
+<script src="/static/js/digimarks.js?v={{ version }}"></script>
+{% block extrajs %}{% endblock %}
+
+</body>
+</html>

src/digimarks/templates/bookmarks.html

@@ -0,0 +1,127 @@
+{% extends "base.html" %}
+{% if not action %}
+    {% set action = 'Bookmarks' %}
+{% endif %}
+{% block title %}{{ action }}{% endblock %}
+{% block page_header %}{{ action }}{% endblock %}
+{% block page_content %}
+
+    {% if tag and not publictag %}
+        <div class="row">
+            <div class="col s12">
+                <a href="{{ url_for('addpublictag', userkey=userkey, tag=tag) }}">Create public page <i
+                        class="material-icons right">tag</i></a>
+            </div>
+        </div>
+    {% endif %}
+
+    {% if tag and publictag %}
+        <div class="row">
+            <div class="col s12"><a href="{{ url_for('publictag_page', tagkey=publictag.tagkey) }}">Public link</a>
+            </div>
+        </div>
+    {% endif %}
+
+    {% if message %}
+        <div class="row">
+            <div class="col s12">
+                <div class="card-panel {{ theme.MESSAGE_BACKGROUND }}">
+            <span class="{{ theme.MESSAGE_TEXT }}">
+                {{ message|safe }}
+            </span>
+                </div>
+            </div>
+        </div>
+    {% endif %}
+
+    <div class="row">
+        <form action="{{ url_for('bookmarks_page', userkey=userkey) }}" name="filterForm" method="POST"
+              autocomplete="off">
+            <div class="input-field col l9 m9 s8">
+                <input placeholder="search text" type="text" name="filter_text" id="filter_text" class="autocomplete"
+                       value="{{ filter_text }}" autocomplete="false"/>
+            </div>
+
+            <div class="input-field col l3 m3 s4">
+                <p class="right-align">
+                    <button class="btn waves-effect waves-light" type="submit" name="submitBtn" title="Find"><i
+                            class="material-icons">search</i></button>
+                    {% if show_as and show_as == 'list' %}
+                        <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod=filtermethod, sortmethod=sortmethod, show_as=None) }}"
+                           class="waves-effect waves-light btn" title="Show as cards"><i class="material-icons">apps</i></a>
+                    {% else %}
+                        <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod=filtermethod, sortmethod=sortmethod, show_as='list') }}"
+                           class="waves-effect waves-light btn" title="Show as list"><i
+                                class="material-icons">reorder</i></a>
+                    {% endif %}
+                </p>
+            </div>
+        </form>
+    </div>
+
+    {% if tags %}
+        <div class="row">
+            <div class="col s12">
+                <ul class="collapsible" data-collapsible="expandable">
+                    <li>
+                        <div class="collapsible-header"><i class="material-icons">label</i>Filter on
+                            star/problem/comment/tag
+                        </div>
+                        <div class="collapsible-body" style="padding: 10px;">
+                            <div class="chip">
+                                <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod='starred') }}"><i
+                                        class="tiny material-icons {{ theme.STAR }}">star</i></a>
+                            </div>
+                            <div class="chip">
+                                <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod='broken') }}"><i
+                                        class="tiny material-icons {{ theme.PROBLEM }}">report_problem</i></a>
+                            </div>
+                            <div class="chip">
+                                <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod='note') }}"><i
+                                        class="tiny material-icons {{ theme.COMMENT }}">comment</i></a>
+                            </div>
+                            {% for tag in tags %}
+                                <div class="chip">
+                                    <a href="{{ url_for('tag_page', userkey=userkey, tag=tag) }}">{{ tag }}</a>
+                                </div>
+                            {% endfor %}
+                    </li>
+                </ul>
+            </div>
+        </div>
+    {% endif %}
+
+    {% if show_as and show_as == 'list' %}
+        {% include 'list.html' %}
+    {% else %}
+        {% include 'cards.html' %}
+    {% endif %}
+
+    <div class="fixed-action-btn" style="bottom: 20px; right: 20px;">
+        <a class="btn-floating btn-large {{ theme.FAB }}" href="{{ url_for('addbookmark', userkey=userkey) }}">
+            <i class="large material-icons">add</i>
+        </a>
+    </div>
+{% endblock %}
+{% block extrajs %}
+    <script>
+        function submitFilter() {
+            document.filterForm.submit();
+        }
+
+        /* Search filter autocomplete */
+        var options = {
+            onAutocomplete: submitFilter,
+            minLength: 3,
+            limit: 10,
+            data: {},
+        }
+        var elem = document.querySelector('.autocomplete');
+        var instance = M.Autocomplete.init(elem, options);
+        /* TODO: fetch from API
+        instance.updateData({
+        });
+        */
+    </script>
+    <script src="{{ url_for('bookmarks_js', userkey=userkey) }}"></script>
+{% endblock %}

src/digimarks/templates/index.html

@@ -0,0 +1,30 @@
+{% extends "base.html" %}
+{% block title %}digimarks{% endblock %}
+{% block page_header %}digimarks{% endblock %}
+{% block page_content %}
+    <article>
+
+        <header>
+            <nav class="menu">
+                <ul>
+                    <li><h1>digimarks</h1></li>
+                    <li>
+                        <a class="button" href="https://codeberg.org/diginaut/digimarks">digimarks project page</a>
+                    </li>
+                </ul>
+            </nav>
+        </header>
+
+        <main>
+            <h1>Welcome to digimarks, your online bookmarking and notes tool</h1>
+
+            <p>Please visit your personal url, or <a href="https://codeberg.org/diginaut/digimarks">see the digimarks
+                project page</a>.</p>
+
+            <p>If you forgot/lost your personal url, contact your digimarks
+                administrator.{# On startup, the personal codes are printed to the standard output (so should be findable in a log). Of course, bookmarks.db contains the user information too.#}
+            </p>
+        </main>
+
+    </article>
+{% endblock %}

src/digimarks/templates/publicbookmarks.html

@@ -0,0 +1,30 @@
+{% extends "base.html" %}
+{% if not action %}
+    {% set action = 'Bookmarks' %}
+{% endif %}
+{% block title %}{{ action }}{% endblock %}
+{% block page_header %}{{ action }}{% endblock %}
+{% block page_content %}
+
+    {% if message %}
+        <div class="row">
+            <div class="col s12">
+                <div class="card-panel orange lighten-2">
+            <span class="white-text">
+                {{ message }}
+            </span>
+                </div>
+            </div>
+        </div>
+    {% endif %}
+
+    <div class="row">
+        <div class="col s12">
+            <a href="{{ url_for('publictag_feed', tagkey=tagkey) }}"><i class="material-icons tiny">rss_feed</i>
+                feed</a>
+        </div>
+    </div>
+
+    {% include 'cards.html' %}
+
+{% endblock %}

src/digimarks/templates/tags.html

@@ -0,0 +1,68 @@
+{% extends "base.html" %}
+{% block title %}Tags{% endblock %}
+{% block page_header %}Tags{% endblock %}
+{% block page_content %}
+    <div class="row">
+
+        <div class="col s12">
+            <table class="centered">
+                <thead>
+                <tr>
+                    <th><i class="material-icons" title="Unique labels">label</i></th>
+                    <th><i class="material-icons green-text" title="Public tag pages">present_to_all</i></th>
+                    <th><i class="material-icons" title="Total bookmarks">turned_in</i></th>
+                    <th><i class="material-icons" title="Bookmarks with notes">comment</i></th>
+                    <th><i class="material-icons yellow-text" title="Starred bookmarks">star</i></th>
+                    <th><i class="material-icons orange-text" title="HTTP status is not 200 OK">warning</i></th>
+                    <th><i class="material-icons red-text" title="Deleted bookmarks">delete</i></th>
+                </tr>
+                </thead>
+                <tbody>
+                <tr>
+                    <td>{{ totaltags }}</td>
+                    <td>{{ totalpublic }}</td>
+                    <td>{{ totalbookmarks }}</td>
+                    <td>{{ totalnotes }}</td>
+                    <td>{{ totalstarred }}</td>
+                    <td>{{ totalhttperrorstatus }}</td>
+                    <td>{{ totaldeleted }}</td>
+                </tr>
+                </tbody>
+            </table>
+
+            <br/><br/>
+
+            <table>
+                <thead>
+                <tr>
+                    <th>Tag</th>
+                    <th>Public link</th>
+                    <th>Number of bookmarks</th>
+                </tr>
+                </thead>
+                <tbody>
+                {% for tag in tags %}
+                    <tr>
+                        <td>
+                            <a href="{{ url_for('tag_page', userkey=userkey, tag=tag['tag']) }}">{{ tag['tag'] }}</a>
+                        </td>
+                        <td>
+                            {% if tag['publictag'] %}
+                                <a href="{{ url_for('publictag_page', tagkey=tag['publictag'].tagkey) }}">Public
+                                    link</a> (
+                                <a href="{{ url_for('removepublictag', tag=tag['tag'], tagkey=tag['publictag'].tagkey, userkey=userkey) }}">Delete</a>
+                                <i class="tiny material-icons red-text">warning</i>)
+                            {% else %}
+                                <a href="{{ url_for('addpublictag', userkey=userkey, tag=tag['tag']) }}">Create</a>
+                            {% endif %}
+                        </td>
+                        <td>
+                            {{ tag['total'] }}
+                        </td>
+                    </tr>
+                {% endfor %}
+                </tbody>
+            </table>
+        </div>
+    </div>
+{% endblock %}

src/digimarks/templates/user_index.html

@@ -0,0 +1,237 @@
+{% extends "base.html" %}
+{% block title %}Bookmarks{% endblock %}
+{% block page_header %}Bookmarks{% endblock %}
+{% block page_content %}
+    <article
+            x-init="$store.digimarks.userKey = '{{ user_key }}'; $store.digimarks.loadCache(); $store.digimarks.getBookmarks()"
+            x-data="">
+
+        <header>
+            <nav class="menu">
+                <ul>
+                    <li><h1>digimarks</h1></li>
+                    <li>
+                        <div class="button-group">
+                            <button x-data @click="$store.digimarks.toggleTagPage()"
+                                    :class="!$store.digimarks.showTags && 'active'">bookmarks
+                            </button>
+                            <button x-data @click="$store.digimarks.toggleTagPage()"
+                                    :class="$store.digimarks.showTags && 'active'">tags
+                            </button>
+                        </div>
+                    </li>
+                    <li>
+                        <button @click="$store.digimarks.startAddingBookmark()">add bookmark</button>
+                    </li>
+                    <li>
+                        <button @click="$store.digimarks.loopToNextTheme()" class="theme-toggle">theme</button>
+                    </li>
+                    <li><input x-model="$store.digimarks.search" placeholder="Search/filter..."></li>
+                    <li x-show="$store.digimarks.loading"><i class="fa-solid fa-rotate-right fa-spin"></i></li>
+                </ul>
+            </nav>
+        </header>
+
+        <main>
+            <section x-cloak x-show="$store.digimarks.showBookmarks" x-transition.opacity>
+                <h1 x-bind:title="$store.digimarks.userKey">Bookmarks</h1>
+
+                <p>
+                <div class="button-group">
+                    <button @click="$store.digimarks.sortAlphabetically()"
+                            :class="$store.digimarks.sortTitleAsc && 'active'">a-z &darr;
+                    </button>
+                    <button @click="$store.digimarks.sortAlphabetically('desc')"
+                            :class="$store.digimarks.sortTitleDesc && 'active'">z-a &uarr;
+                    </button>
+                    <button @click="$store.digimarks.sortCreated()"
+                            :class="$store.digimarks.sortCreatedAsc && 'active'">date &darr;
+                    </button>
+                    <button @click="$store.digimarks.sortCreated('desc')"
+                            :class="$store.digimarks.sortCreatedDesc && 'active'">date &uarr;
+                    </button>
+                </div>
+                <div class="button-group">
+                    <button @click="$store.digimarks.toggleListOrGrid()"
+                            :class="$store.digimarks.showBookmarksCards && 'active'">grid
+                    </button>
+                    <button @click="$store.digimarks.toggleListOrGrid()"
+                            :class="!$store.digimarks.showBookmarksCards && 'active'">list
+                    </button>
+                </div>
+                </p>
+
+                <table x-cloak x-show="$store.digimarks.showBookmarksList">
+                    <thead>
+                    <tr>
+                        <th colspan="2">&nbsp;</th>
+                        <th>Title</th>
+                        <th>Note</th>
+                        <th>Tags</th>
+                    </tr>
+                    </thead>
+                    <tbody>
+                    <template x-for="bookmark in $store.digimarks.filteredBookmarks" :key="bookmark.id">
+                        <tr>
+                            <td class="thumbnail">
+                                <div class="card-thumb" x-show="bookmark.favicon"><img
+                                        x-bind:src="'/content/favicons/' + bookmark.favicon"></div>
+                            </td>
+                            <td>
+                                <div x-show="bookmark.starred" class="star"><i class="fa-fw fa-solid fa-star"></i>
+                                </div>
+                                <div x-show="bookmark.http_status !== 200 && bookmark.http_status !== 304"
+                                     class="error"><i
+                                        class="fa-fw fa-solid fa-triangle-exclamation"></i>
+                                </div>
+                            </td>
+                            <td><a x-text="bookmark.title" x-bind:href="bookmark.url" target="_blank"></a></td>
+                            <td x-text="bookmark.note"></td>
+                            <td>
+                                <template x-for="tag in bookmark.tag_list">
+                                    <span x-text="tag" class="chip"></span>
+                                </template>
+                            </td>
+                        </tr>
+                    </template>
+                    </tbody>
+                </table>
+                {#
+                <ul x-cloak x-show="$store.digimarks.show_bookmarks_list">
+                    <template x-for="bookmark in $store.digimarks.filteredBookmarks" :key="bookmark.id">
+                        <li><a x-text="bookmark.title" x-bind:href="bookmark.url" target="_blank"></a></li>
+                    </template>
+                </ul>
+#}
+                <section x-cloak x-show="$store.digimarks.showBookmarksCards" class="cards">
+                    <template x-for="bookmark in $store.digimarks.filteredBookmarks" :key="bookmark.id">
+                        <div class="card">
+                            <div class="card-body">
+                                <div class="card-thumb" x-show="bookmark.favicon"><img
+                                        x-bind:src="'/content/favicons/' + bookmark.favicon"></div>
+                                <div class="statuses">
+                                    <div x-show="bookmark.starred" class="star"><i
+                                            class="fa-fw fa-solid fa-star"></i>
+                                    </div>
+                                    <div x-show="bookmark.http_status !== 200 && bookmark.http_status !== 304"
+                                         class="error"><i
+                                            class="fa-fw fa-solid fa-triangle-exclamation"></i>
+                                    </div>
+                                    <div x-show="bookmark.note"><i class="fa-fw fa-regular fa-note-sticky"></i>
+                                    </div>
+                                </div>
+                                <div><a x-text="bookmark.title" x-bind:href="bookmark.url" target="_blank"></a>
+                                </div>
+                            </div>
+                            <div class="card-footer">
+                                <button title="show actions"><i class="fa-solid fa-square-caret-down"></i></button>
+                                <div class="meta">
+                                    <template x-for="tag in bookmark.tag_list">
+                                        <span x-text="tag" class="chip"></span>
+                                    </template>
+                                </div>
+                                {#                                <div x-text="bookmark.created_date" class="meta"></div>#}
+                            </div>
+                        </div>
+                    </template>
+                </section>
+            </section>
+
+            <section x-cloak x-show="$store.digimarks.showTags" x-transition.opacity>
+                <h1>Tags</h1>
+
+                <table>
+                    <thead>
+                    <tr>
+                        <th>Tag</th>
+                        <th>Public link</th>
+                        <th>Number of bookmarks</th>
+                    </tr>
+                    </thead>
+                    <tbody>
+                    <template x-for="tag in $store.digimarks.filteredTags" :key="tag">
+                        <tr>
+                            <td x-text="tag"></td>
+                            <td></td>
+                            <td></td>
+                        </tr>
+                    </template>
+                    </tbody>
+                </table>
+            </section>
+
+            <dialog x-cloak id="editFormDialog"
+                    x-transition:enter="modal-enter"
+                    x-transition:enter-start="modal-enter"
+                    x-transition:enter-end="modal-enter-active"
+                    x-transition:leave="modal-leave-active"
+                    x-transition:leave-start="modal-enter-active"
+                    x-transition:leave-end="modal-enter">
+                <h1>Add/Edit bookmark</h1>
+                {#
+                <div class="card-panel {{ theme.ERRORMESSAGE_BACKGROUND }}">
+            <span class="error">
+                {% if bookmark.http_status == 404 %}
+                    <i class="material-icons">report_problem</i>&nbsp;&nbsp;URL not found (404), broken/outdated link?
+                {% elif bookmark.http_status == 301 %}
+                    <i class="material-icons">report_problem</i>&nbsp;&nbsp;HTTP status (301), moved permanently. Use button for new target
+                {% elif bookmark.http_status == 302 %}
+                    <i class="material-icons">report_problem</i>&nbsp;&nbsp;HTTP status (302), moved temporarily. Use button for new target
+                {% elif bookmark.http_status == bookmark.HTTP_CONNECTIONERROR %}
+                    <i class="material-icons">report_problem</i>&nbsp;&nbsp;Connection error, server might have been offline at the time of last edit
+                {% else %}
+                    <i class="material-icons">report_problem</i>&nbsp;&nbsp;HTTP status {{ bookmark.http_status }}
+                {% endif %}
+            </span>
+                </div>
+#}
+                <template x-if="$store.digimarks.bookmarkToEditVisible">
+                    <form method="dialog" id="bookmarkEditForm">
+                        <fieldset class="form-group">
+                            <label for="bookmark_url">URL</label>
+                            <input id="bookmark_url" type="text" name="bookmark_url" placeholder="url"
+                                   x-on:change.debounce="$store.digimarks.bookmarkURLChanged()"
+                                   x-model="$store.digimarks.bookmarkToEdit.url">
+                            <p x-show="$store.digimarks.bookmarkToEdit.http_status > 202"
+                               x-text="'HTTP statuscode: ' + $store.digimarks.bookmarkToEdit.http_status" x-cloak
+                               class="error"></p>
+                            <p>
+                        </fieldset>
+                        <fieldset class="form-group">
+                            <label for="bookmark_title">Title</label>
+                            <input id="bookmark_title" type="text" name="bookmark_title"
+                                   placeholder="title (leave empty for autofetch)"
+                                   x-model="$store.digimarks.bookmarkToEdit.title">
+                        </fieldset>
+                        <fieldset class="form-group">
+                            <label for="bookmark_note">Note</label>
+                            <textarea id="bookmark_note" type="text" name="bookmark_note"
+                                      x-model="$store.digimarks.bookmarkToEdit.note">
+                        </textarea>
+                        </fieldset>
+                        <fieldset class="form-group">
+                            <label for="bookmark_tags">Tags</label>
+                            <input id="bookmark_tags" type="text" name="bookmark_tags"
+                                   placeholder="tags, divided bij comma's"
+                                   x-model="$store.digimarks.bookmarkToEdit.tags">
+                        </fieldset>
+                        <p x-show="$store.digimarks.bookmarkToEditError"
+                           x-text="$store.digimarks.bookmarkToEditError" x-cloak class="error"></p>
+                        <p>
+                            <label>
+                                <input type="checkbox" x-model="$store.digimarks.bookmarkToEdit.strip_params"/>
+                                <span>Strip parameters from url (like <em>?utm_source=social</em> - can break the link!)</span>
+                            </label>
+                        </p>
+                        <div>
+                            <button value="cancel">Cancel</button>
+                            <button @click="$store.digimarks.saveBookmark()">Save</button>
+                            <button @click="$store.digimarks.addBookmark()">Add</button>
+                        </div>
+                    </form>
+                </template>
+            </dialog>
+        </main>
+
+    </article>
+{% endblock %}

src/digimarks/utils.py

@@ -0,0 +1,15 @@
+"""General utility functions."""
+
+import binascii
+import hashlib
+import os
+
+
+def generate_hash(input_text: str) -> str:
+    """Generate a hash from string `input`, e.g., for a URL."""
+    return hashlib.md5(input_text.encode('utf-8')).hexdigest()
+
+
+def generate_key() -> str:
+    """Generate a key to be used for a user or tag."""
+    return str(binascii.hexlify(os.urandom(24)))

static/css/digimarks.css

@@ -1,67 +0,0 @@
-/**
- * digimarks styling
- */
-
-/** Navigation **/
-
-nav .sidenav-trigger
-{
-    /* Fix for misalignment of hamburger icon */
-    margin: 0;
-}
-
-nav .sidenav-trigger i
-{
-    /* Make the hamburger icon great again */
-    font-size: 2.7rem;
-}
-
-/** Cards and tags **/
-
-.card .card-content,
-.card .card-reveal
-{
-    padding: 12px;
-}
-
-.card.tiny
-{
-    height: 140px;
-    overflow: hidden;
-}
-
-.card.tiny .card-title
-{
-    font-size: 18px;
-}
-
-.card .card-reveal .digimark-card-header,
-.card .digimark-card-header.activator,
-.chip.clickable
-{
-    cursor: pointer;
-    /*display: block;*/
-}
-
-.card .digimark-card-header-tags
-{
-    padding-top: 10px;
-}
-
-.card-image
-{
-    min-width: 60px;
-}
-
-.card-image i,
-.list-image i
-{
-    padding: 5px 0 0 15px;
-}
-
-.card.horizontal .card-image img.favicon,
-.list-image img.favicon
-{
-    height: 60px;
-    width: 60px;
-}

static/js/init.js

@@ -1,11 +0,0 @@
-/* global M */
-
-var options = {};
-var elem = document.querySelector(".sidenav");
-var instance = M.Sidenav.init(elem, options);
-
-elem = document.querySelector(".collapsible");
-instance = M.Collapsible.init(elem, {
-    // inDuration: 1000,
-    // outDuration: 1000
-});

templates/404.html

@@ -1,6 +0,0 @@
-{% extends "base.html" %}
-{% block title %}404: Page not found{% endblock %}
-{% block pageheader %}404: Page not found{% endblock %}
-{% block pagecontent %}
-The page you requested was not found.
-{% endblock %}

templates/base.html

@@ -1,126 +0,0 @@
-<!doctype html>
-<html>
-    <head>
-        <title>{% block title %}{% endblock %} - digimarks</title>
-
-        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
-        <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0"/>
-        <link rel="shortcut icon" href="{{ url_for('static', filename='favicon.ico') }}"/>
-
-        <!-- Chrome, Firefox OS and Opera -->
-        <meta name="theme-color" content="{{ theme.BROWSERCHROME }}" />
-        <!-- Windows Phone -->
-        <meta name="msapplication-navbutton-color" content="{{ theme.BROWSERCHROME }}">
-        <!-- iOS Safari -->
-        <meta name="apple-mobile-web-app-capable" content="yes">
-        <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
-
-        <meta name="referrer" content="never">
-        <meta name="referrer" content="no-referrer">
-
-        <link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">
-        <link href='https://fonts.googleapis.com/css?family=Roboto+Mono&subset=latin,latin-ext' rel='stylesheet' type='text/css'>
-        <link href="https://cdnjs.cloudflare.com/ajax/libs/materialize/1.0.0/css/materialize.min.css" type="text/css" rel="stylesheet" media="screen,projection"/>
-        <style>
-            a
-            {
-                color: {{ theme.LINK_TEXT }};
-            }
-            .card-content a
-            {
-                color: {{ theme.CARD_LINK }};
-            }
-            .chip a
-            {
-                color: {{ theme.CHIP_TEXT }};
-            }
-            /* label color */
-            .input-field .prefix ~ input, .input-field .prefix ~ textarea, .input-field .prefix ~ label, .input-field .prefix ~ .validate ~ label, .input-field .prefix ~ .autocomplete-content, .input-field input[type=text]
-            {
-                color: {{ theme.TEXTHEX }};
-            }
-            /* label focus color */
-            .input-field input[type=text]:focus + label,
-            .input-field .prefix ~ input[type=text]:focus + label
-            {
-                color: {{ theme.BUTTON }};
-            }
-            /* label underline focus color */
-            .input-field input[type=text]:focus,
-            .input-field .prefix ~ input[type=text]:focus,
-            .input-field input[type=text].autocomplete:focus
-            {
-                border-bottom: 1px solid {{ theme.BUTTON }};
-                box-shadow: 0 1px 0 0 {{ theme.BUTTON }};
-            }
-            /* icon prefix focus color */
-            .input-field .prefix.active
-            {
-                color: {{ theme.BUTTON }};
-            }
-            .btn, .btn:visited
-            {
-                background-color: {{ theme.BUTTON }};
-            }
-            .btn:hover, .btn:active
-            {
-                background-color: {{ theme.BUTTON_ACTIVE }};
-            }
-            .deletebtn
-            {
-                background-color: red;
-            }
-            .deletebtn:hover
-            {
-                background-color: #ef5350; /* red lighten-1 */
-            }
-        </style>
-        <link href="{{ url_for('static', filename='css/digimarks.css') }}?20180330" type="text/css" rel="stylesheet" media="screen,projection"/>
-{% if not sortmethod %}
-    {% set sortmethod = None %}
-{% endif %}
-{% if not show_as %}
-    {% set show_as = None %}
-{% endif %}
-    </head>
-    <body class="{{ theme.BODY }} {{ theme.TEXT }}">
-        <nav class="{{ theme.NAV }}" role="navigation">
-            <div class="nav-wrapper container"><a id="logo-container" href="{% if userkey %}{{ url_for('bookmarks_page', userkey=userkey, sortmethod=sortmethod, show_as=show_as) }}{% else %}{{ url_for('index') }}{% endif %}" class="brand-logo">digimarks</a>
-                <ul class="right hide-on-med-and-down">
-                    {% if userkey %}
-                    <li><a href="{{ url_for('tags_page', userkey=userkey) }}" class="waves-effect waves-light btn"><i class="material-icons left">label</i>Tags</a></li>
-                    <li><a href="{{ url_for('addbookmark', userkey=userkey) }}" class="waves-effect waves-light btn"><i class="material-icons left">add</i>Add bookmark</a></li>
-                    {% endif %}
-                </ul>
-
-                {% if userkey %}
-                <ul id="nav-mobile" class="sidenav">
-                    <li><a class="waves-effect" href="{{ url_for('bookmarks_page', userkey=userkey) }}"><i class="material-icons left">turned_in</i>Home</a></li>
-                    <li><a class="waves-effect" href="{{ url_for('tags_page', userkey=userkey) }}"><i class="material-icons left">label</i>Tags</a></li>
-                    <li><a class="waves-effect" href="{{ url_for('addbookmark', userkey=userkey) }}"><i class="material-icons left">add</i>Add bookmark</a></li>
-                </ul>
-                <a href="#" data-target="nav-mobile" class="sidenav-trigger"><i class="material-icons">menu</i></a>
-                {% endif %}
-            </div>
-        </nav>
-        <div class="section no-pad-bot" id="index-banner">
-            <div class="container">
-                <div class="header {{ theme.PAGEHEADER }}">
-                    <h1>{% block pageheader %}Bookmarks{% endblock %}</h1>
-                </div>
-            </div>
-        </div>
-        <div class="container">
-            <div class="section">
-                {% block pagecontent %}
-                {% endblock %}
-            </div>
-        </div>
-
-        <!-- Scripts -->
-        <script src="https://cdnjs.cloudflare.com/ajax/libs/materialize/1.0.0/js/materialize.min.js"></script>
-        <script src="{{ url_for('static', filename='js/init.js') }}?20180309"></script>
-
-        {% block extrajs %}{% endblock %}
-    </body>
-</html>

templates/bookmarks.html

@@ -1,113 +0,0 @@
-{% extends "base.html" %}
-{% if not action %}
-{% set action = 'Bookmarks' %}
-{% endif %}
-{% block title %}{{ action }}{% endblock %}
-{% block pageheader %}{{ action }}{% endblock %}
-{% block pagecontent %}
-
-{% if tag and not publictag %}
-<div class="row">
-    <div class="col s12">
-        <a href="{{ url_for('addpublictag', userkey=userkey, tag=tag) }}">Create public page <i class="material-icons right">tag</i></a>
-    </div>
-</div>
-{% endif %}
-
-{% if tag and publictag %}
-<div class="row">
-    <div class="col s12"><a href="{{ url_for('publictag_page', tagkey=publictag.tagkey) }}">Public link</a></div>
-</div>
-{% endif %}
-
-{% if message %}
-<div class="row">
-    <div class="col s12">
-        <div class="card-panel {{ theme.MESSAGE_BACKGROUND }}">
-            <span class="{{ theme.MESSAGE_TEXT }}">
-                {{ message|safe }}
-            </span>
-        </div>
-    </div>
-</div>
-{% endif %}
-
-<div class="row">
-    <form action="{{ url_for('bookmarks_page', userkey=userkey) }}" name="filterForm" method="POST" autocomplete="off">
-        <div class="input-field col l9 m9 s8">
-            <input placeholder="search text" type="text" name="filter_text" id="filter_text" class="autocomplete" value="{{ filter_text }}" autocomplete="false" />
-        </div>
-
-        <div class="input-field col l3 m3 s4">
-            <p class="right-align"><button class="btn waves-effect waves-light" type="submit" name="submitBtn" title="Find"><i class="material-icons">search</i></button>
-            {% if show_as and show_as == 'list' %}
-                <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod=filtermethod, sortmethod=sortmethod, show_as=None) }}" class="waves-effect waves-light btn" title="Show as cards"><i class="material-icons">apps</i></a>
-            {% else %}
-                <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod=filtermethod, sortmethod=sortmethod, show_as='list') }}" class="waves-effect waves-light btn" title="Show as list"><i class="material-icons">reorder</i></a>
-            {% endif %}
-            </p>
-        </div>
-    </form>
-</div>
-
-{% if tags %}
-<div class="row">
-    <div class="col s12">
-        <ul class="collapsible" data-collapsible="expandable">
-            <li>
-              <div class="collapsible-header"><i class="material-icons">label</i>Filter on star/problem/comment/tag</div>
-              <div class="collapsible-body" style="padding: 10px;">
-                <div class="chip">
-                    <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod='starred') }}"><i class="tiny material-icons {{ theme.STAR }}">star</i></a>
-                </div>
-                <div class="chip">
-                    <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod='broken') }}"><i class="tiny material-icons {{ theme.PROBLEM }}">report_problem</i></a>
-                </div>
-                <div class="chip">
-                    <a href="{{ url_for('bookmarks_page', userkey=userkey, filtermethod='note') }}"><i class="tiny material-icons {{ theme.COMMENT }}">comment</i></a>
-                </div>
-                {% for tag in tags %}
-                <div class="chip">
-                    <a href="{{ url_for('tag_page', userkey=userkey, tag=tag) }}">{{ tag }}</a>
-                </div>
-                {% endfor %}
-            </li>
-        </ul>
-    </div>
-</div>
-{% endif %}
-
-{% if show_as and show_as == 'list' %}
-    {% include 'list.html' %}
-{% else %}
-    {% include 'cards.html' %}
-{% endif %}
-
-<div class="fixed-action-btn" style="bottom: 20px; right: 20px;">
-    <a class="btn-floating btn-large {{ theme.FAB }}" href="{{ url_for('addbookmark', userkey=userkey) }}">
-      <i class="large material-icons">add</i>
-    </a>
-</div>
-{% endblock %}
-{% block extrajs %}
-<script>
-    function submitFilter() {
-        document.filterForm.submit();
-    }
-    /* Search filter autocomplete */
-    var options = {
-        onAutocomplete: submitFilter,
-        minLength: 3,
-        limit: 10,
-        data: {
-        },
-    }
-    var elem = document.querySelector('.autocomplete');
-    var instance = M.Autocomplete.init(elem, options);
-    /* TODO: fetch from API
-    instance.updateData({
-    });
-    */
-</script>
-<script src="{{ url_for('bookmarks_js', userkey=userkey) }}" ></script>
-{% endblock %}

templates/bookmarks.js

@@ -1,11 +0,0 @@
-var elem = document.querySelector('.autocomplete');
-var instance = M.Autocomplete.getInstance(elem);
-instance.updateData({
-    {% for bookmark in bookmarks %}
-        {% if bookmark.favicon %}
-            "{{ bookmark.title | replace('"', '\\"') | replace('\n', '') | replace('\r', '') }}": "{{ url_for('static', filename='favicons/' + bookmark.favicon) }}",
-        {% else %}
-            "{{ bookmark.title | replace('"', '\\"') | replace('\n', '') | replace('\r', '') }}": null,
-        {% endif %}
-    {% endfor %}
-});

templates/index.html

@@ -1,17 +0,0 @@
-{% extends "base.html" %}
-{% block title %}digimarks{% endblock %}
-{% block pageheader %}digimarks{% endblock %}
-{% block pagecontent %}
-    <p>Please visit your personal url, or <a href="https://github.com/aquatix/digimarks">see the digimarks project page</a>.</p>
-
-<div class="row">
-    <div class="col s12">
-        <div class="card-panel orange lighten-2">
-            <span class="black-text">
-                If you forgot/lost your personal url, contact your digimarks administrator. On startup, the personal codes are printed to the standard output (so should be findable in a log). Of course, bookmarks.db contains the user information too.
-            </span>
-        </div>
-    </div>
-</div>
-
-{% endblock %}

templates/publicbookmarks.html

@@ -1,29 +0,0 @@
-{% extends "base.html" %}
-{% if not action %}
-{% set action = 'Bookmarks' %}
-{% endif %}
-{% block title %}{{ action }}{% endblock %}
-{% block pageheader %}{{ action }}{% endblock %}
-{% block pagecontent %}
-
-{% if message %}
-<div class="row">
-    <div class="col s12">
-        <div class="card-panel orange lighten-2">
-            <span class="white-text">
-                {{ message }}
-            </span>
-        </div>
-    </div>
-</div>
-{% endif %}
-
-<div class="row">
-    <div class="col s12">
-        <a href="{{ url_for('publictag_feed', tagkey=tagkey) }}"><i class="material-icons tiny">rss_feed</i> feed</a>
-    </div>
-</div>
-
-{% include 'cards.html' %}
-
-{% endblock %}

templates/tags.html

@@ -1,65 +0,0 @@
-{% extends "base.html" %}
-{% block title %}Tags{% endblock %}
-{% block pageheader %}Tags{% endblock %}
-{% block pagecontent %}
-<div class="row">
-
-    <div class="col s12">
-    <table class="centered">
-        <thead>
-            <tr>
-                <th><i class="material-icons" title="Unique labels">label</i></th>
-                <th><i class="material-icons green-text" title="Public tag pages">present_to_all</i></th>
-                <th><i class="material-icons" title="Total bookmarks">turned_in</i></th>
-                <th><i class="material-icons" title="Bookmarks with notes">comment</i></th>
-                <th><i class="material-icons yellow-text" title="Starred bookmarks">star</i></th>
-                <th><i class="material-icons orange-text" title="HTTP status is not 200 OK">warning</i></th>
-                <th><i class="material-icons red-text" title="Deleted bookmarks">delete</i></th>
-            </tr>
-        </thead>
-        <tbody>
-            <tr>
-                <td>{{ totaltags }}</td>
-                <td>{{ totalpublic }}</td>
-                <td>{{ totalbookmarks }}</td>
-                <td>{{ totalnotes }}</td>
-                <td>{{ totalstarred }}</td>
-                <td>{{ totalhttperrorstatus }}</td>
-                <td>{{ totaldeleted }}</td>
-            </tr>
-        </tbody>
-    </table>
-
-    <br /><br />
-
-    <table>
-        <thead>
-            <tr>
-                <th>Tag</th>
-                <th>Public link</th>
-                <th>Number of bookmarks</th>
-            </tr>
-        </thead>
-        <tbody>
-            {% for tag in tags %}
-            <tr>
-                <td>
-                    <a href="{{ url_for('tag_page', userkey=userkey, tag=tag['tag']) }}">{{ tag['tag'] }}</a>
-                </td>
-                <td>
-                    {% if tag['publictag'] %}
-                        <a href="{{ url_for('publictag_page', tagkey=tag['publictag'].tagkey) }}">Public link</a> (<a href="{{ url_for('removepublictag', tag=tag['tag'], tagkey=tag['publictag'].tagkey, userkey=userkey) }}">Delete</a> <i class="tiny material-icons red-text">warning</i>)
-                    {% else %}
-                        <a href="{{ url_for('addpublictag', userkey=userkey, tag=tag['tag']) }}">Create</a>
-                    {% endif %}
-                </td>
-                <td>
-                    {{ tag['total'] }}
-                </td>
-            </tr>
-            {% endfor %}
-        </tbody>
-    </table>
-    </div>
-</div>
-{% endblock %}

tox.ini

@@ -0,0 +1,21 @@
+[flake8]
+ignore = D203, W503
+exclude =
+    .git,
+    __pycache__,
+    docs/source/conf.py,
+    build,
+    dist,
+    example_config/gunicorn_webhaak_conf.py,
+    example_config/rq_settings.example.py,
+    example_config/settings.py,
+max-line-length = 120
+max-complexity = 10
+
+[pycodestyle]
+max_line_length = 120
+ignore = E501, W503
+
+[isort]
+line_length = 120
+multi_line_output = 3

wsgi.py

@@ -1,16 +0,0 @@
-# Activate virtualenv
-import settings
-activate_this = getattr(settings, 'VENV', None)
-# FIXME: python 2 *and* python 3 compatibility
-# Python 2
-#if activate_this:
-#    execfile(activate_this, dict(__file__=activate_this))
-# Python 3
-with open(activate_this) as file_:
-    exec(file_.read(), dict(__file__=activate_this))
-
-from digimarks import app as application
-
-if __name__ == "__main__":
-    # application is ran standalone
-    application.run(debug=settings.DEBUG)