Regarding possibly wasteful io in neon page server's wal application procedure

General
1

Hi, I went through the WAL redo code in the neon page server and came across the code where we updated the page to the latest lsn (apply_wal_records).
This code starts up a pg process and calls RmgrTable[record->xl_rmid].rm_redo

Even though the job of this function apply_wal_records is to apply wal records of a particular page, rm_redo would read and update all pages involved in the wal record.

for ex: A heap_update wal record will have a block to apply on the page in which the tuple is deleted and another block to apply on the page where the tuple is inserted.

Am I getting this right? Wouldn’t it be wasteful to apply WAL for all pages and then discard the unnecessary pages that where modified? Is there some hack we could do so we avoid reading unneeded pages while calling RmgrTable[record->xl_rmid].rm_redo and save on the i/o ??

2

That’s partially correct. Yes, we reuse the rm_redo code from PostgreSQL, and that does try to apply WAL on all pages. But it also only tries to apply WAL to pages that still need to be modified.

In the wal-redo PostgreSQL instance hosted by Pageserver, all pages are read through XLogReadBufferForRedo, which returns BLK_DONE (“block is already up-to-date”) for all pages, except the page that Pageserver needs to reconstruct.
A similar mechanism in read-only nodes makes sure that we only have to apply changes to pages that were already present in Postgres’ buffers when we’re applying the new WAL that arrives from the primary instance.

1 Like