Found by licensee. After debugging, it appears that the Slate input is consuming the KeyDown event which adds the apostrophe, but it passes the KeyUp event back to Player input. The gameplay debugger catches the KeyUp event which then calls to toggle the gameplay debugger.
Expected result would be that input would either be fully consumed by Slate input or that the GameplayDebugger would check for some value set in the KeyDown event which runs a separate function before toggling the GameplayDebugger.
There's no existing public thread on this issue, so head over to AnswerHub just mention UE-131402 in the post.