The problem seems to be that in HandleRefreshTestCallback in AutomationCommandline.cpp, it enables all the tests, but then in SAutomationWindow::OnRefreshTestCallback it turns off all the test that don't have checkboxes (typically all of them).
I'm not sure if it should always ignore that callback when I'm still running test from the commandline, or only if I have the "-unattended" flag (which I also had), but at least it should probably make this error state clearer.
Leave the Session Frontend "Automation" tab open in the editor and close the editor.
Run the editor from the command-line with the automation command "Run All".
This results in no tests running without a warning or error.
There's no existing public thread on this issue, so head over toAnswerHub just mention UE-55523 in the post.