Changing a project's Mac game icon does not persist through Mac development packages of the project. However, Shipping packages will show the correct icon.
This only occurs on Mac. This was not reproducible on Windows.
Works: 4.12.5 3039270 (Can regress further upon request)
Result: The .app's icon is the default Unreal icon instead of the icon set in the project settings
There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-34581 in the post.