Implementing Durable Workflows with PostgreSQL Alone

PostgreSQL, a widely-used database, has been shown by engineer Raman Varma to be capable of serving as the foundation for running workflows while safely maintaining processing state without relying on external workflow management tools. This represents a noteworthy approach for development teams seeking to keep their system architecture simple.

What exactly is a "durable workflow"? In short, it is a mechanism that ensures processing state does not disappear even if a server suddenly crashes, and the system can correctly resume from where it left off after restarting. Examples include multi-step processes waiting for human approval or "sleep-enabled processing" that resumes after minutes or hours. Traditionally, to meet these requirements, teams would set up a dedicated orchestrator software as a separate component.

In Varma's approach, PostgreSQL alone assumes this role. Specifically, it leverages PostgreSQL's SKIP LOCKED feature, which prevents multiple processes from redundantly acquiring the same rows when querying the same queue simultaneously, enabling safe parallel processing. Next, checkpoint management using primary keys guarantees idempotency—ensuring that executing the same process twice produces the same result. Furthermore, by employing leases (a mechanism where if a process fails to demonstrate its existence within a set time period, another process takes over), the approach handles crash recovery.

Additionally, workflow "sleeps" (pauses) and states where the system awaits human approval are stored as database records. This ensures that even if the application itself is restarted, the state is not lost and processing can resume accurately. In essence, the core of this technique lies in continuously recording "how far the process has progressed" on the database rather than in memory.

The practical significance of this approach should not be underestimated. Implementing an external orchestrator incurs costs for operations, monitoring, and version management, and increases overall system complexity. If a team can leverage PostgreSQL—a database that many organizations already operate—they can achieve durable workflows without introducing new infrastructure. This becomes a realistic option, particularly for teams with smaller headcounts or in scenarios where maximizing an existing PostgreSQL environment is desired.

However, it is important to note that this technique is not universally optimal for all use cases. In environments requiring large-scale throughput or where comprehensive centralized visualization of complex workflows is desired, dedicated tools may be more appropriate. This approach should be positioned as targeting scenarios where cost is a consideration under the assumption that PostgreSQL is already in place, with the goal of minimizing additional expenditure.

This technique, which revisits the question of what can be accomplished with a single database, aligns with the engineering trend emphasizing infrastructure simplicity. In recent years, PostgreSQL has expanded beyond its traditional RDBMS scope, with applications in vector search and time-series data processing, and this approach can be seen as a continuation of that evolution. Going forward, how widely such techniques can be applied across different scales and use cases will be the focal point of practical validation.