The modules file that is generated when packaging a plugin appends a number to the end of the .dll reference, similar to hot reload modules. This causes the reference to refer to a nonexistent file. Attempting to open a project with this packaged plugin included prompts the user to recompile the plugin, due to the dll not being found.
Regression?: Yes
This did not occur in 4.21
There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-72487 in the post.