12.7. Apply Worker and Transaction Replay

The apply worker is the core component responsible for replaying logical changes received from the publisher onto the subscriber’s local tables.

This section first reviews the fundamental operation of the apply worker, followed by an analysis of advanced modes involving the incremental processing of large transactions and the coordinated use of parallel apply workers.

Section Contents

12.7.1. Process Overview (streaming = off)

The apply worker on the subscriber node performs the following primary tasks:

  • Origin Filtering: Inspects the origin metadata of the incoming message to determine whether the transaction should be replayed or skipped to prevent loops.

  • LSN Skip Check (Idempotency): Compares the incoming transaction’s LSN with the remote_lsn persisted in the pg_replication_origin_status view. If the transaction has already been applied, the worker skips it to ensure data consistency.

  • Dispatch: Routes messages to their respective handlers based on the message type, such as INSERT, UPDATE, or DELETE.

  • Conflict Detection: Identifies operational conflicts, such as attempting to update a missing tuple. By default, the worker reports these conflicts and halts replication to prevent divergence.

12.7.1.1. Origin Filtering and LSN Tracking

As discussed in Section 12.1.4, the decision to apply a transaction depends on the origin parameter and the presence of an origin_id.

In current implementations, if origin = ’none’ is configured and the origin_id is greater than zero, the apply worker aborts the replay; otherwise, the worker proceeds.

12.7.1.2. Message Dispatching and Replay Flow

Upon receiving a stream of logical decoding messages, the apply worker processes them sequentially based on their type. Consider a scenario where a single tuple is inserted into tbl_a on the publisher:

Message Stream Sample:

[B] lsn=0/1CA96C0  ts=2026-03-29T16:55:05  txid=845
[R] oid=16456  "public"."tbl_a"  'd'  3cols      [id:int4(key)][name:text][data:int4]
[I] oid=16456  N      [t"3"][t"Candy"][t"3"]
[C] flags=0  commit=0/1CA9E00  end=0/1CA9F00  ts=2026-03-29T16:55:15

The processing flow for this INSERT transaction is as follows:

  1. [B] Begin Message: The apply worker initiates a local transaction and captures the publisher’s commit LSN and timestamp. The worker sets the session_replication_role to replica, ensuring that local triggers and constraints follow their defined replication roles.

  2. [R] Relation Message: The apply worker receives the table definition. The worker updates the RelationSyncCache, mapping the publisher-side OID (e.g., 16456) to the local table’s OID based on the schema and table name.

  3. [I] Insert Message: The worker converts the binary tuple data into the local table format. The apply worker executes an internal insert (via ExecSimpleRelationInsert), which also updates any associated indexes. This operation generates its own WAL records on the subscriber.

  4. [C] Commit Message: The local transaction commits. The apply worker appends the origin_id corresponding to the publisher to the commit WAL record. The worker then updates pg_replication_origin to store the latest applied LSN and sends an acknowledgment (ACK) to the walsender, confirming that the data has flushed to disk.

While INSERT is the primary example here, other DML operations such as UPDATE and DELETE follow a similar dispatch logic.

The commit WAL record contains appended metadata from the source publisher: specifically, the origin_id, the final_lsn as the origin_lsn, and the commit_timestamp as the origin_timestamp.

As mentioned in Section 12.4.3, UPDATE and DELETE operations involve an additional step: the apply worker must execute direct tuple lookups — utilizing index or sequential scans via functions such as RelationFindReplTupleByIndex — to uniquely identify and modify the target data before applying the change.

12.7.1.3. Optimization for REPLICA IDENTITY FULL

In version 15 or earlier, tables configured with REPLICA IDENTITY FULL necessitated a sequential scan to identify the target tuple for UPDATE or DELETE operations, leading to significant performance overhead on large tables.

From version 16 onwards, the apply worker can leverage existing indexes to locate tuples even under REPLICA IDENTITY FULL. The function FindLogicalRepLocalIndex() identifies suitable non-partial B-tree indexes that can uniquely identify the row, substantially reducing the reliance on costly sequential scans.

12.7.2. Streaming Mode (streaming = on)

When the streaming parameter is set to on, the apply worker receives in-progress transactions from the publisher in multiple segments before the final commit.

Consider a scenario involving two concurrent transactions: txid 842, a large transaction sent across two streamed segments, and txid 843, a smaller transaction sent as a standard block-based message. See Figure 12.28.

Figure 12.28. Replay process of logical changes when streaming is 'on'.

The processing sequence illustrated in Figure 12.28 is as follows:

  • (1) First Segment of txid 842: The apply worker receives the initial stream segment and accumulates the changes in memory using a StringInfoData structure (internally, the original_msg buffer).
  • (2) Message for txid 843: A standard non-streamed block for txid 843 arrives. The apply worker immediately decodes and applies these changes to the local table.
  • (3) Second Segment of txid 842: The subsequent stream segment for txid 842 arrives and is appended to the existing buffer in memory.
  • (4) Commit of txid 842: Upon receiving the Stream Commit message, the apply worker decodes all accumulated changes for txid 842 and replays them sequentially.

In summary, the apply worker processes standard messages immediately while storing streamed segments in a buffer until the commit message triggers the replay of the entire transaction.

12.7.2.1. Memory Pressure and Spilling to Disk

The logical_decoding_work_mem parameter governs the memory allocated for buffering streamed changes. If the volume of accumulated changes exceeds this limit, the apply worker spills the data to temporary files.

These temporary files are located in the $PGDATA/base/pgsql_tmp/ directory. The naming conventions are structured as follows:

  • Directory Format: $PGDATA/base/pgsql_tmp/pgsql_tmp[PID].[FilesetSerial].fileset/
  • File Format: [SubID]-[txid].changes

If a subscription with OID 16403 processes a streamed transaction with txid 767 using an apply worker with PID 2212, the file path is as follows:

base/pgsql_tmp/pgsql_tmp2212.0.fileset/16403-767.changes

Once the transaction commits or aborts, PostgreSQL automatically removes these temporary files.

12.7.3. Parallel Apply Worker Mode (streaming = parallel)

Introduced in version 16, parallel apply enhances logical replication performance by allowing multiple transactions to be replayed concurrently on the subscriber. Unlike the standard streaming = 'on' mode, which buffers streamed changes until a commit arrives, parallel apply enables assigned workers to begin replaying changes immediately as they are received.

Note

Parallel apply only applies to streamed transactions — those sent as a sequence of segments rather than as a single message. Therefore, non-streamed transactions are still received and applied directly by the leader apply worker itself, exactly as in streaming = 'off' mode; no parallel apply worker is involved in that case.

The architecture and constraints of this mode are summarized below:

  • Leader Apply Worker: Maintains the connection with the walsender and dispatches parallel apply workers for incoming changes.
  • Parallel Apply Workers: Launched by the leader apply worker to apply changes concurrently.
  • Worker Limit: The number of concurrent parallel apply workers is governed by the max_parallel_apply_workers_per_subscription parameter.
  • Fallback Mechanism: If the number of active streamed transactions exceeds the configured limit, the leader apply worker handles additional large transactions by falling back to the streaming = ‘on’ behavior (buffering to memory or disk).

The following sections illustrate the processing overview of parallel apply workers using several scenarios.

12.7.3.1. Replay Sequence in Parallel Mode

Consider the simplest scenario: two transactions, txid 845 and txid 846, are both large transactions that insert many rows into tables tbl_a and tbl_b, respectively. See Figure 12.29.

Figure 12.29. Transaction replay sequence in parallel apply mode.

As illustrated in Figure 12.29, the processing flow occurs as follows:

  • (1) Arrival Segment of txid 845: The leader apply worker identifies this as a streamed transaction and launches (or assigns) a parallel apply worker.
  • (2) Immediate Replay: The leader apply worker dispatches the segment to the parallel apply worker, which immediately starts replaying the changes to the local table.
  • (3) Arrival Segment of txid 846: The leader apply worker launches (or assigns) a parallel apply worker.
  • (4) Immediate Replay: The leader apply worker dispatches the segment to another parallel apply worker, which immediately replays the changes to the local table.
  • (5) Arrival Stream Commit of txid 845: The leader apply worker receives the Stream Commit message for txid 845 and forwards it to the assigned parallel apply worker.
  • (6) Finalization of txid 845: The parallel apply worker commits the local transaction and returns to a ready state.
  • (7) Leader Waits: While txid 845’s commit is being finalized, the leader apply worker pauses and does not read further changes — including txid 846’s Stream Commit message — from the publisher. Only after txid 845’s commit completes does the leader apply worker resume reading, receive txid 846’s Stream Commit message, and forward it to the parallel apply worker handling txid 846. The reason for this wait is explained in Section 12.7.3.2 (see also pa_wait_for_xact_finish()).
  • (8) Finalization of txid 846: The parallel apply worker commits the local transaction and returns to a ready state.

The leader apply worker dynamically manages task distribution based on available system resources:

  • If a parallel apply worker is already assigned to the transaction, the leader apply worker forwards the segment to that worker for immediate replay.

  • If no worker is assigned and the number of active parallel apply workers is below the max_parallel_apply_workers_per_subscription threshold, the leader apply worker launches a new worker and delegates the transaction.

  • Fallback Mechanism: If the parallel apply worker limit is reached, the leader apply worker reverts to the streaming = ‘on’ behavior, accumulating incoming segments in memory or on disk until the commit message arrives.

12.7.3.2. Avoiding Failures Due to Transaction Dependencies

This subsection explains why the leader apply worker pauses reading and applying further changes while an in-progress transaction’s commit is being finalized.

In non-parallel mode, changes from multiple transactions are serialized on the publisher by MVCC, and the subscriber’s apply worker replays them in that exact order, preventing correctness issues.

In parallel mode, however, separate parallel apply workers apply changes concurrently. Consequently, the transaction order serialized on the publisher is no longer automatically preserved on the subscriber.

To resolve this, the leader apply worker pauses reading new changes while finalizing a commit. The following example illustrates this behavior.

Consider table tbl_a, which is initially empty. Two transactions, txid 851 and txid 852, perform an INSERT and an UPDATE on tbl_a, respectively. See Figure 12.30.

Figure 12.30. Leader apply worker pausing to preserve transaction order.

On the publisher, txid 851 inserts rows and commits; only then does txid 852 update those rows in tbl_a.

  • (1) Arrival Segment of txid 851: The leader apply worker receives the segment for txid 851 and forwards it to parallel apply worker 1.
  • (2) Immediate Replay: Parallel apply worker 1 immediately begins replaying the INSERT changes to the local table.
  • (3) Arrival Commit of txid 851, Leader Pauses: The leader apply worker receives the Stream Commit message for txid 851, forwards it to parallel apply worker 1, and pauses reading further messages from the publisher until this commit completes.
  • (4) Segment of txid 852 Arrives, But Is Not Read: While the leader apply worker is paused, the publisher sends the segment for txid 852. The leader apply worker does not retrieve this message yet. As Figure 12.30 illustrates, if the leader apply worker forwarded it immediately and parallel apply worker 2 applied it, the UPDATE would find no matching rows on the subscriber because txid 851’s INSERT is not yet committed and visible.
  • (5) Commit of txid 852 Also Arrives, Still Unread: The Stream Commit message for txid 852 likewise reaches the subscriber, but remains unread for the same reason.
  • (6) Leader Resumes and Forwards txid 852: Once parallel apply worker 1 finishes committing txid 851 locally, the leader apply worker resumes reading. It retrieves both the segment and the Stream Commit message for txid 852, forwarding them to parallel apply worker 2.
  • (7) Immediate Replay and Finalization: Parallel apply worker 2 replays the UPDATE changes and commits, during which the leader apply worker pauses again as in step (3).

By pausing message reception from the moment a commit message is received until local commit finalization, the leader apply worker prevents out-of-order execution and preserves the transaction sequence established on the publisher.