This behavior appears to be a regression new to UE5, as I tested in 4.25 and the issue does not occur.
Interestingly, the issue clears up if you switch to No, FXAA, or MSAA antialiasing methods. The issue appears to only occur with TSR and TAA.
The issue does not seem to occur if you switch the PPM to BeforeTonemapping (although then the world position values get tonemapped which may be undesirable).
There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-164501 in the post.