In short, the trick storing a pointer to the packed index of the primitive does not work as it is simply a pointer into the same object as
the cached primitive already stored. If the object pointer is invalid, the index pointer will also be invalid.
See UDN post.
There's no existing public thread on this issue, so head over toAnswerHub just mention UE-68094 in the post.