Using the Editor Utility Widget to add components to objects in UE does not appear to function as expected. I reproduced this 8/8 times in UE 5.1, 5.0, and 4.27, confirming that this is not a regression. I was also able to reproduce the issue on Linux and Windows (I have not tested on Mac).
Result: A collision box is added to the actor in the viewport, and there are no changes to the actor in the details panel.
Expected: A collision box is added to the actor in the viewport and as a component to the actor in the details panel.
There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-170085 in the post.