Create two cubes and have each use a different material.
Open the merge actors tool
Merge method: Merge
Enable "Merge Materials"
Tooltip says that "LOD Selection Type" must be set to use the lowest lod. 5.2 required this to enable Merge Materials, but 5.3 doesn't (might be another issue here).
Run the merge.
Add the merged actor to the scene.
Change view mode to "Visibility Collision"
Notice the merged actor does not appear.
Try these steps again without "Merge Materials" and notice the merged actor does appear.
Have Comments or More Details?
There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-199444 in the post.