feat: integrate Paperless-ngx document workflows

.gitignore

@@ -7,5 +7,7 @@ uploads/*
 !README.md
 !CONTRIBUTING.md
 !CHANGELOG.md
+!docs/
+!docs/**/*.md
 *.db
-*.db*
+*.db*

CHANGELOG.md

@@ -2,6 +2,12 @@
 
 ## Unreleased
 
+- Added a Paperless-ngx integration for linking or copying existing archive documents to vehicles
+- Added explicit copy and move workflows for sending MotoMate uploads to Paperless-ngx without deleting remote documents
+- Added authenticated inline document previews, downloads, Paperless thumbnails, and generated/cached thumbnails for local PDFs and raster images
+- Added reusable document relationships for maintenance and spending events, including sale/purchase relationships and migration of existing attachment arrays
+- Added encrypted storage for Paperless API tokens and background sync jobs with retry/error state
+- Clarified storage ownership and removal behavior throughout the document UI
 - You can now convert an existing vehicle between kilometers and miles (not compatible with hours, ofcourse)
 - Your account currency and distance preferences are now applied consistently across, while each entry keeps the currency it was originally recorded in
 - Spending totals now group by currency when a vehicle has entries in more than one currency, instead of adding different currencies into a single incorrect total
@@ -81,4 +87,4 @@
 - Fix: reminder entries are now correctly saved alongside your service logs
 - Fix: filters on the maintenance view no longer reset when you navigate away from the page
 - Fix: reminders are now properly scheduled with cron (#33)
-- Fix: alerts like odometer reminders no longer repeat every day once they have already been sent (#35)
+- Fix: alerts like odometer reminders no longer repeat every day once they have already been sent (#35)

README.md

@@ -14,7 +14,7 @@ View our [**demo-instance here**](https://motomate.mijnmotorparkeren.nl) (hosted
 
 <img width="100%" alt="MotoMate screenshot" src=".github/images/example.webp" />
 
-We want to make it incredibly simple for riders and vehicle enthusiasts to host their own maintenance journals. Unlike more complex systems such as [LubeLogger](https://lubelogger.com/?ref=github.com/hawkinslabdev/motomate), MotoMate is designed to strip your tracking down to the absolute essentials. 
+We want to make it incredibly simple for riders and vehicle enthusiasts to host their own maintenance journals. Unlike more complex systems such as [LubeLogger](https://lubelogger.com/?ref=github.com/hawkinslabdev/motomate), MotoMate is designed to strip your tracking down to the absolute essentials.
 
 ## Getting Started
 
@@ -43,10 +43,25 @@ services:
 
 After downloading the image and starting the container, the application will be ready in a few seconds once database migrations complete.
 
+### Paperless-ngx integration
+
+MotoMate can connect to an existing Paperless-ngx library so documents can be linked or copied to a vehicle, and local uploads can be copied or moved into Paperless. Browser previews and document thumbnails are available without exposing the Paperless API token to the browser.
+
+Set a separate 32-byte encryption key before creating the connection:
+
+```yaml
+environment:
+  - INTEGRATION_ENCRYPTION_KEY=replace-with-output-from-openssl-rand-hex-32
+```
+
+Then open **Settings > Developer > Paperless-ngx** and enter the Paperless base URL and an API token. The MotoMate container must be able to reach that URL.
+
+See the [Paperless-ngx integration guide](docs/paperless-integration.md) for action semantics, deployment notes, security details, and troubleshooting.
+
 ## Donate
 
-[![Buy Me A Coffee](https://img.shields.io/badge/Buy_me_a_coffee-fdd734?\&logo=buy-me-a-coffee\&logoColor=black\&style=for-the-badge)](https://coff.ee/hawkinslabdev)
-[![GitHub Sponsors](https://img.shields.io/badge/GitHub_Sponsors-30363d?style=for-the-badge\&logo=github\&logoColor=white)](https://github.com/sponsors/hawkinslabdev)
+[![Buy Me A Coffee](https://img.shields.io/badge/Buy_me_a_coffee-fdd734?&logo=buy-me-a-coffee&logoColor=black&style=for-the-badge)](https://coff.ee/hawkinslabdev)
+[![GitHub Sponsors](https://img.shields.io/badge/GitHub_Sponsors-30363d?style=for-the-badge&logo=github&logoColor=white)](https://github.com/sponsors/hawkinslabdev)
 
 Want to support MotoMate? Drop a star on GitHub, or consider supporting development via GitHub Sponsors or Buy Me a Coffee.
 
@@ -57,6 +72,7 @@ This project is licensed under the **AGPL 3.0** license. See [LICENSE](LICENSE)
 ## Contributors
 
 Made possible thanks to the following people:
+
 <!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
 <!-- prettier-ignore-start -->
 <!-- markdownlint-disable -->

docs/paperless-integration.md

@@ -0,0 +1,111 @@
+# Paperless-ngx integration
+
+MotoMate can use Paperless-ngx as an external document library while retaining clear ownership boundaries. The integration supports selecting an existing Paperless document, keeping a local copy when desired, and transferring new MotoMate uploads into Paperless.
+
+MotoMate never deletes a document from Paperless-ngx. Every destructive action in MotoMate is limited to the MotoMate record, its local object, or its generated thumbnail cache.
+
+## Requirements
+
+- A Paperless-ngx instance whose base URL is reachable from the MotoMate container
+- A Paperless API token with access to the documents the MotoMate user should see
+- A dedicated 32-byte `INTEGRATION_ENCRYPTION_KEY`
+
+Generate the encryption key with:
+
+```sh
+openssl rand -hex 32
+```
+
+Add it to the MotoMate service environment:
+
+```yaml
+services:
+  motomate:
+    environment:
+      INTEGRATION_ENCRYPTION_KEY: "your-64-character-hex-value"
+      PAPERLESS_SYNC_INTERVAL_SECONDS: "30"
+```
+
+Do not reuse `AUTH_SECRET` as the integration key. Keep the configured value stable: changing it makes previously saved Paperless tokens unreadable until the original key is restored or the connection is recreated.
+
+The sync interval defaults to 30 seconds and controls how frequently MotoMate polls Paperless for completion of asynchronous uploads.
+
+## Connect Paperless-ngx
+
+1. In Paperless-ngx, create or copy an API token for the account MotoMate will use.
+2. In MotoMate, open **Settings > Developer**.
+3. Under **Paperless-ngx**, enter a connection name, the Paperless base URL, and the API token.
+4. Select **Connect and test**. MotoMate saves the token only after the API test succeeds.
+
+The saved token is encrypted with AES-256-GCM before it is stored in the MotoMate database. It is used only by authenticated server-side requests and is never returned to the browser.
+
+## Document actions
+
+| Action                  | MotoMate content                                                                    | Paperless content                                          | What removal from MotoMate does                                            |
+| ----------------------- | ----------------------------------------------------------------------------------- | ---------------------------------------------------------- | -------------------------------------------------------------------------- |
+| **Link from Paperless** | Metadata and a secure reference only                                                | Existing document remains authoritative                    | Removes only the MotoMate reference                                        |
+| **Copy from Paperless** | Stores an independent local copy and the Paperless reference                        | Existing document remains unchanged                        | Removes the MotoMate record and local copy only                            |
+| **Copy to Paperless**   | Keeps the local MotoMate file                                                       | Creates a new Paperless document                           | Removes the MotoMate record and local copy; the Paperless document remains |
+| **Move to Paperless**   | Keeps the local file until Paperless confirms successful ingestion, then removes it | Creates a new Paperless document and becomes authoritative | Removes only the MotoMate reference                                        |
+| **Preview**             | No ownership change                                                                 | No ownership change                                        | Nothing; this is read-only                                                 |
+| **Download**            | No ownership change                                                                 | No ownership change                                        | Nothing; this is read-only                                                 |
+
+“Copy to Paperless” is the former mirror concept expressed as an explicit action: there are two durable copies after Paperless confirms the upload. “Move to Paperless” is asynchronous and does not remove the local file until Paperless reports success. Failed jobs retain the local content and expose their error state for retry.
+
+Disconnecting a Paperless connection is blocked while MotoMate documents still reference it. Remove or copy those references first. Disconnecting never calls a Paperless delete API.
+
+## Previews and thumbnails
+
+Previewable PDFs, raster images, and plain-text files open in a new browser tab through an authenticated MotoMate endpoint. Download is a separate explicit action and returns an attachment response.
+
+- Linked Paperless documents use Paperless's generated thumbnail endpoint through an authenticated MotoMate proxy.
+- Local PDFs render their first page as a WebP thumbnail on first view.
+- Local JPEG, PNG, GIF, WebP, and AVIF files are resized and converted to WebP.
+- Generated thumbnails are cached under `thumbnails/<user>/<document>.webp` in the configured local or S3 storage adapter.
+- SVG, HTML, and other active or unsupported formats are not rendered as thumbnails. The UI displays a neutral document placeholder instead.
+
+The thumbnail and content routes verify the signed-in user owns the document. Responses use private browser caching and `nosniff`; Paperless credentials and direct internal URLs are not embedded in the page.
+
+## Event attachments and migration
+
+Documents are attached to maintenance and spending events through the `document_links` table instead of event-specific JSON arrays. The relation field distinguishes normal attachments from purchase and sale documents. The same model also supports vehicle-level links and gives future event types a consistent extension point, while the underlying document can be local, Paperless-linked, or present in both systems.
+
+Migration `0009_cheerful_catseye.sql` creates the Paperless connection, sync job, and document link tables; adds source/sync fields to documents; and migrates existing maintenance and spending attachment arrays into document links. MotoMate applies migrations automatically when the container starts. Back up the SQLite database before upgrading.
+
+## Backup and restore
+
+Back up all of the following together:
+
+- MotoMate database (`/app/data` with the default container paths)
+- MotoMate local uploads (`/app/uploads`) when using local storage
+- `INTEGRATION_ENCRYPTION_KEY` in a secrets manager or encrypted configuration backup
+- Paperless-ngx using its normal backup process
+
+A MotoMate backup cannot restore linked-only Paperless content. Conversely, a Paperless backup does not include MotoMate event relationships.
+
+## Troubleshooting
+
+### Connection test fails
+
+- Confirm the URL is the Paperless base URL, not an individual API endpoint.
+- Test DNS, TLS, and routing from inside the MotoMate container. A URL reachable from a desktop browser may not be reachable from Docker.
+- Confirm the token is current and the Paperless account can list documents.
+- If the error mentions encryption, verify `INTEGRATION_ENCRYPTION_KEY` decodes to exactly 32 bytes.
+
+### A transfer remains queued or processing
+
+- Check that the MotoMate process is running its scheduler.
+- Verify `PAPERLESS_SYNC_INTERVAL_SECONDS` is a positive number.
+- Open the document list to inspect its transfer status and error detail.
+- Confirm Paperless workers are processing document ingestion tasks.
+
+### Preview or thumbnail returns 502
+
+- Test the Paperless connection again for linked-only documents.
+- Confirm the original MotoMate object still exists in local or S3 storage for local documents.
+- Check MotoMate logs for `Document content retrieval failed` or `Document thumbnail retrieval failed`.
+- For S3, verify MotoMate has get, put, and delete access to both document and `thumbnails/` object keys.
+
+### Paperless shows duplicates
+
+Copy and move create new Paperless documents. Wait for an in-progress action to finish before starting another transfer. MotoMate prevents multiple active jobs for the same document, but retrying after Paperless accepted an upload and before its success state became visible can require manual duplicate cleanup in Paperless.

motomate/.env.example

@@ -1,6 +1,8 @@
 # Required
 DATABASE_URL=./data/motomate.db
 AUTH_SECRET=change-me-in-production-min-32-chars   # openssl rand -hex 32
+# Separate 32-byte key for encrypting external integration credentials (openssl rand -hex 32)
+INTEGRATION_ENCRYPTION_KEY=
 
 # Auth
 AUTH_COOKIE_SECURE=false
@@ -41,6 +43,7 @@ PUBLIC_DEMO_ENABLED=false
 
 # Scheduler
 CRON_INTERVAL_HOURS=1
+PAPERLESS_SYNC_INTERVAL_SECONDS=30
 
 # Push notifications (VAPID)
 VAPID_PUBLIC_KEY=

motomate/drizzle/0009_cheerful_catseye.sql

@@ -0,0 +1,94 @@
+CREATE TABLE `document_links` (
+	`id` text PRIMARY KEY NOT NULL,
+	`vehicle_id` text NOT NULL,
+	`document_id` text NOT NULL,
+	`target_type` text NOT NULL,
+	`target_id` text NOT NULL,
+	`relation` text DEFAULT 'attachment' NOT NULL,
+	`created_at` text DEFAULT (datetime('now')) NOT NULL,
+	FOREIGN KEY (`vehicle_id`) REFERENCES `vehicles`(`id`) ON UPDATE no action ON DELETE cascade,
+	FOREIGN KEY (`document_id`) REFERENCES `documents`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+CREATE INDEX `idx_document_links_target` ON `document_links` (`target_type`,`target_id`);--> statement-breakpoint
+CREATE INDEX `idx_document_links_vehicle` ON `document_links` (`vehicle_id`);--> statement-breakpoint
+CREATE INDEX `idx_document_links_document` ON `document_links` (`document_id`);--> statement-breakpoint
+CREATE UNIQUE INDEX `idx_document_links_unique` ON `document_links` (`target_type`,`target_id`,`document_id`,`relation`);--> statement-breakpoint
+CREATE TABLE `document_sync_jobs` (
+	`id` text PRIMARY KEY NOT NULL,
+	`document_id` text NOT NULL,
+	`user_id` text NOT NULL,
+	`mode` text NOT NULL,
+	`state` text DEFAULT 'queued' NOT NULL,
+	`paperless_task_id` text,
+	`attempt_count` integer DEFAULT 0 NOT NULL,
+	`next_attempt_at` text,
+	`last_error` text,
+	`created_at` text DEFAULT (datetime('now')) NOT NULL,
+	`updated_at` text DEFAULT (datetime('now')) NOT NULL,
+	FOREIGN KEY (`document_id`) REFERENCES `documents`(`id`) ON UPDATE no action ON DELETE cascade,
+	FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+CREATE INDEX `idx_document_sync_jobs_document` ON `document_sync_jobs` (`document_id`);--> statement-breakpoint
+CREATE INDEX `idx_document_sync_jobs_due` ON `document_sync_jobs` (`state`,`next_attempt_at`);--> statement-breakpoint
+CREATE TABLE `paperless_integrations` (
+	`id` text PRIMARY KEY NOT NULL,
+	`user_id` text NOT NULL,
+	`name` text DEFAULT 'Paperless-ngx' NOT NULL,
+	`base_url` text NOT NULL,
+	`encrypted_token` text NOT NULL,
+	`enabled` integer DEFAULT true NOT NULL,
+	`last_tested_at` text,
+	`last_error` text,
+	`created_at` text DEFAULT (datetime('now')) NOT NULL,
+	`updated_at` text DEFAULT (datetime('now')) NOT NULL,
+	FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+CREATE UNIQUE INDEX `idx_paperless_integrations_user_url` ON `paperless_integrations` (`user_id`,`base_url`);--> statement-breakpoint
+PRAGMA foreign_keys=OFF;--> statement-breakpoint
+CREATE TABLE `__new_documents` (
+	`id` text PRIMARY KEY NOT NULL,
+	`vehicle_id` text NOT NULL,
+	`user_id` text NOT NULL,
+	`name` text NOT NULL,
+	`title` text,
+	`doc_type` text DEFAULT 'service' NOT NULL,
+	`source` text DEFAULT 'motomate' NOT NULL,
+	`storage_key` text,
+	`mime_type` text NOT NULL,
+	`size_bytes` integer NOT NULL,
+	`paperless_integration_id` text,
+	`paperless_document_id` integer,
+	`sync_status` text DEFAULT 'none' NOT NULL,
+	`sync_error` text,
+	`last_synced_at` text,
+	`expires_at` text,
+	`created_at` text DEFAULT (datetime('now')) NOT NULL,
+	FOREIGN KEY (`vehicle_id`) REFERENCES `vehicles`(`id`) ON UPDATE no action ON DELETE cascade,
+	FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE cascade,
+	FOREIGN KEY (`paperless_integration_id`) REFERENCES `paperless_integrations`(`id`) ON UPDATE no action ON DELETE set null
+);
+--> statement-breakpoint
+INSERT INTO `__new_documents`("id", "vehicle_id", "user_id", "name", "title", "doc_type", "source", "storage_key", "mime_type", "size_bytes", "paperless_integration_id", "paperless_document_id", "sync_status", "sync_error", "last_synced_at", "expires_at", "created_at") SELECT "id", "vehicle_id", "user_id", "name", "title", "doc_type", 'motomate', "storage_key", "mime_type", "size_bytes", NULL, NULL, 'none', NULL, NULL, "expires_at", "created_at" FROM `documents`;--> statement-breakpoint
+DROP TABLE `documents`;--> statement-breakpoint
+ALTER TABLE `__new_documents` RENAME TO `documents`;--> statement-breakpoint
+PRAGMA foreign_keys=ON;--> statement-breakpoint
+CREATE INDEX `idx_documents_vehicle_created` ON `documents` (`vehicle_id`,`created_at`);--> statement-breakpoint
+CREATE UNIQUE INDEX `idx_documents_paperless_document` ON `documents` (`paperless_integration_id`,`paperless_document_id`) WHERE "documents"."paperless_document_id" IS NOT NULL;--> statement-breakpoint
+INSERT OR IGNORE INTO `document_links` (`id`, `vehicle_id`, `document_id`, `target_type`, `target_id`, `relation`, `created_at`)
+SELECT 'dl_' || lower(hex(randomblob(16))), service_logs.vehicle_id, attachment.value, 'service_log', service_logs.id, 'attachment', service_logs.created_at
+FROM service_logs
+JOIN json_each(CASE WHEN json_valid(service_logs.attachments) THEN service_logs.attachments ELSE '[]' END) AS attachment
+JOIN documents ON documents.id = attachment.value
+	AND documents.vehicle_id = service_logs.vehicle_id
+WHERE attachment.type = 'text';--> statement-breakpoint
+INSERT OR IGNORE INTO `document_links` (`id`, `vehicle_id`, `document_id`, `target_type`, `target_id`, `relation`, `created_at`)
+SELECT 'dl_' || lower(hex(randomblob(16))), finance_transactions.vehicle_id, attachment.value, 'finance_transaction', finance_transactions.id, 'attachment', finance_transactions.created_at
+FROM finance_transactions
+JOIN json_each(CASE WHEN json_valid(finance_transactions.attachments) THEN finance_transactions.attachments ELSE '[]' END) AS attachment
+JOIN documents ON documents.id = attachment.value
+	AND documents.vehicle_id = finance_transactions.vehicle_id
+	AND documents.user_id = finance_transactions.user_id
+WHERE attachment.type = 'text';