Paperless-ngx: Turn Paper Piles into a Searchable Online Archive

Drawers of receipts, folders of invoices, shoeboxes of tax records - physical paper is where information goes to become unfindable. Paperless-ngx, a Python document management system with over 45,000 stars and a GPL-3.0 license, solves this by turning scanned documents into a searchable online archive: it watches an inbox, runs OCR on every page, suggests tags and correspondents automatically, and files the result where you can find it again in seconds. As the official successor to the original Paperless and Paperless-ng projects, it is maintained by a dedicated team rather than a single author, and it has quietly become the default choice for people who want their documents on their own hardware. This post walks through how the system actually works, from the moment a scanner drops a PDF into the consume folder to the moment it appears, fully indexed, in the web UI.

Why You Need This

Cloud document services charge monthly fees to hold your most sensitive files, train their models on your privacy, and sometimes delete accounts without warning. Paperless-ngx takes the opposite trade: it runs on a machine you control, stores everything in plain formats you can back up, and never phones home. The project itself is blunt about the responsibility that comes with this - documents are stored unencrypted, so it belongs on a trusted host in your home, a stance it shares with other local-first systems like Home Assistant.

The second problem it solves is manual labor. A scan is just an image until someone OCRs it, names it, tags it, and files it - and nobody does that consistently for thousands of pages. Paperless-ngx automates the entire chain. Mail attachments arrive on their own, barcode separator pages split multi-document scans, an OCR layer makes every word searchable, and a classifier trained on your own archive suggests tags, correspondents, and document types for each new arrival. What used to be an evening of filing becomes a notification. For developers who want to extract structured text from PDFs programmatically, we previously covered LiteParse; Paperless-ngx applies similar ideas to the archive you actually live in.

How It Works

The diagram below maps the repository’s main components and how they connect.

Intake. Documents enter through the consumer flow in src/documents/consumer.py, triggered by the workflows engine whenever a file lands in the consume folder, an email arrives, or an API upload completes. Anything the engine accepts - a watched folder, a drag-and-drop upload, or a webhook from a scanner app - ends up in the same queue with the same treatment. The mail importer in src/paperless_mail connects to your mail accounts over IMAP with OAuth support, applies per-account rules to decide which attachments matter, and hands them to the same flow. Celery tasks, queued through a Valkey broker, keep every step asynchronous so the web UI stays responsive while ingestion runs.

Processing. Each document then flows through parsing stages: barcode detection splits scans on separator sheets, and the parser layer extracts text and metadata from whatever format arrived. Scanned images are recognized by the bundled OCR stack built on OCRmyPDF and Tesseract; office documents are delegated to companion containers - Apache Tika pulls out text and metadata, and Gotenberg renders office formats to PDF. After extraction, the classifier requests tag, correspondent, and document-type predictions, backed by the matching engine that combines simple rules with a machine-learning model trained on documents you have already filed. Finally, the file layout manager renames the archived file from your own templates and stores it alongside the document record in the database.

The web layer. An Angular frontend talks to a Django REST API for browsing, faceted filtering, and bulk edits, while WebSocket updates push live ingestion progress into the browser. Search answers instantly because the text index is maintained continuously as documents arrive, not rebuilt on demand. A newer AI module extends the archive further: an AI-assisted classifier refines automatic tagging, and a vector store plus document chat lets you ask questions of your own papers in plain language. The result is an archive that files itself.

Operations. Scheduled tasks handle index maintenance, classifier retraining, and email polling; a sanity checker verifies archive integrity on demand; and a plugin hook system lets custom code join the ingestion stages without forking the core.

Advantages

The architecture earns its keep in several ways. Storage stays boring and durable: original files keep their exact bytes on disk next to a database record, so exports and backups remain simple even years later. The ingestion flow is asynchronous end to end, so a hundred-document scan batch never blocks the UI. Automation is adaptive rather than static - the classifier improves as your tagging habits accumulate, and the matching engine lets you mix hand-written regex rules with learned suggestions. The whole stack is light enough for a low-power home server or a small virtual machine, which keeps running costs near zero. Deployment is honest about complexity: one Docker Compose file brings up the web server, PostgreSQL or MariaDB, Valkey, Tika, and Gotenberg together, with SQLite available for smaller setups. And the plugin boundaries mean the ingestion stages are extensible in Python without patching the core.

Benefits

In practice this adds up to an archive that behaves like infrastructure. You stop paying rent on your own paperwork, and your records remain usable even if the project vanished tomorrow, because the source files are plain PDFs with an embedded text layer - readable in any ordinary PDF viewer, not just inside paperless. Finding a five-year-old invoice becomes a search box instead of a folder excavation. Multi-user support with per-document permissions makes it viable for families and small offices, not just solo enthusiasts. Bulk editing turns reorganization into a few clicks. And the new AI chat gives the archive a conversational front door - ask where the car insurance policy is and get an answer drawn from your own documents, computed locally rather than by a third party.

Usage

The fastest path is the install script, which configures a Docker Compose environment for you; alternatively, grab a compose file from the repository’s docker/compose directory and point it at the official GitHub Container Registry image. Set a secret key and database credentials, start the stack, and create your admin account on first login. From there, point your scanner’s save folder at the consume directory, add a mail account for automatic attachments, and upload a first batch of documents. Within minutes you will have an OCR’d, searchable archive that tags new arrivals as you teach it. A built-in exporter command produces a portable backup of every document and setting, so migration to new hardware is a copy, not a rebuild. Full setup guides, backup instructions, and troubleshooting live in the documentation folder of the repository, and a public demo exists for a risk-free look before you deploy.

Conclusion

Paperless-ngx is that rare self-hosted project where the engineering depth is invisible in daily use: documents go in, answers come out, and the machinery - Celery queues, OCR stages, classifiers, companion containers - hums underneath. Its discipline about plain-file storage and local control makes it a safe foundation for genuinely important records, and its new AI features show the project still moving forward. If paper keeps arriving in your life, this is the tool that makes it stop mattering.

Enjoyed this post? Never miss out on future posts by following us