Description

Regression testing done on //UE4/Release-4.27.2 CL-18427296 and this issue did occur.

[Link Removed]
[Link Removed]

Below is snips of customers description and proposed workaround:

Importing alembic as skeletal mesh takes too much memory, causes TArray overflow and crashes import
...
I'm working in the team that's working on virtual movie production in unreal engine and currently we think that alembic is the best format that fit our needs
...

Currently we hit a wall with our 2 million normals mesh importing it with 800 frames, it results in critical error 
....
I've tried to rewrite AlembicImporter myself to make it fit to 64-bit size arrays and it worked, But there's issue number two - importing alembic takes too much memory. In our real life case our mesh has size of 3 GB, it has 2 million normals and 800 frames of animation. The only way we were able to import it is by increasing our RAM to 64 GB and virtual memory to 300 GB.
...

Workaround found but at a cost of Memory usage:

What i've learn investigating the code - it's lines like that need to have TArray replaced with TArray 64.
https://github.com/EpicGames/UnrealEngine/blob/46544fa5e0aa9e6740c19b44b0628b72e7bbd5ce/Engine/Plugins/Importers/AlembicImporter/Source/AlembicLibrary/Private/AbcImporter.cpp#L1168
But then again you'll need to also cast all 32 bit integers to 64 bit when for example in this line `NormalsMatrices[MeshIndex].AddZeroed(AverageNormalData[MeshIndex].Num() * 3 * NumSamples);` -> NormalsMatrices[MeshIndex].AddZeroed((int64) AverageNormalData[MeshIndex].Num() * 3 * NumSamples);
And then to fix the issue you'll need to rewrite all those function definitions like `AbcImporterUtilities::GenerateDeltaFrameDataMatrix` that interact somehow with TArray64. 
Steps to Reproduce
  1. Open any project
  2. In content Browser click Import
  3. import attached .abc file
  4. in import options window, change Import Type to Skeletal
  5. Click Import

Expected Results: File imports without errors

Actual Results: File imports with errors and crashes the editor.

Callstack
UnrealEditor-AlembicLibrary!AbcImporterUtilities::GenerateDeltaFrameDataMatrix(TArray<UE::Math::TVector<float>,TSizedDefaultAllocator<32> > const &,TArray<UE::Math::TVector<float>,TSizedDefaultAllocator<32> > const &,TArray<UE::Math::TVector<float>,TSizedDefaultAllocator<32> > const &,TArray<UE::Math::TVector<float>,TSizedDefaultAllocator<32> > const &,int,int,int,UE::Math::TVector<double> const &,TArray<float,TSizedDefaultAllocator<32> > &,TArray<float,TSizedDefaultAllocator<32> > &) [AbcImportUtilities.cpp:1215]
UnrealEditor-AlembicLibrary!static void <lambda_e302fdf8be2f45152e103006e42fd95d>::operator() [AbcImporter.cpp:1229]
UnrealEditor-AlembicLibrary!void UE::Core::Private::Function::TFunctionRefCaller<TFunction<void __cdecl(int,FAbcFile *) [Function.h:549]
UnrealEditor-AlembicLibrary!static void <lambda_e2c6f0a54d6a56cc0b09565c56cc3fc2>::operator() [AbcFile.cpp:565]
UnrealEditor-AlembicLibrary!`ParallelForImpl::NewParallelForInternal<TFunctionRef<void >,<lambda_600b85aa2a81f4761389a82b02631227>,std::nullptr_t>(int,TFunctionRef<void >,<lambda_600b85aa2a81f4761389a82b02631227>,EParallelForFlags,TArrayView<std::nullptr_t,int> const &) [ParallelFor.h:566]
UnrealEditor-AlembicLibrary!LowLevelTasks::TTaskDelegate<void ,48>::TTaskDelegateImpl<<lambda_a7d909f875cf8fde887b62459e73acd4>,0>::CallAndMove(LowLevelTasks::TTaskDelegate<void ,48> &,void *,unsigned int,bool) [TaskDelegate.h:171]
UnrealEditor-Core!LowLevelTasks::FTask::ExecuteTask() [Task.h:426]
UnrealEditor-Core!LowLevelTasks::FScheduler::TryExecuteTaskFrom<&LowLevelTasks::TLocalQueueRegistry<1024>::TLocalQueue::DequeueGlobal(bool,bool) [Scheduler.cpp:280]
UnrealEditor-Core!LowLevelTasks::FScheduler::WorkerMain(LowLevelTasks::FSleepEvent *,LowLevelTasks::TLocalQueueRegistry<1024>::TLocalQueue *,unsigned int,bool) [Scheduler.cpp:320]
UnrealEditor-Core!static void UE::Core::Private::Function::TFunctionRefCaller<<lambda_776551d1e343b6cb7eaf0e5521c77c81>,void __cdecl(void) [Function.h:549]
UnrealEditor-Core!FThreadImpl::Run() [Thread.cpp:69]
UnrealEditor-Core!FRunnableThreadWin::Run() [WindowsRunnableThread.cpp:146]
UnrealEditor-Core!FRunnableThreadWin::GuardedRun() [WindowsRunnableThread.cpp:76]
KERNEL32!7ffe2fe60000  + 17034
ntdll!7ffe30410000  + 52651

Have Comments or More Details?

There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-150964 in the post.

0
Login to Vote

Fixed
Fix Commit21816098
Main Commit21927572
Release Commit21926141
CreatedMay 3, 2022
ResolvedSep 6, 2022
UpdatedSep 20, 2022
View Jira Issue