Niagara Scalability Crash When Processing Significance

Method FNiagaraScalabilityManager::ProcessSignificance() iterates on indices from Context.SignificanceIndices and uses them to access the State and ManagedComponents arrays. It can then decide to create or destroy cull proxies for Niagara Components based on its settings. During this process, it is possible, although unlikely, that CreateCullProxy() triggers a call chain that results in recently completed components unregistering themselves from the scalability manager. In that situation, FNiagaraScalabilityManager::UnregisterAt() ends up removing elements from the State and ManagedComponents arrays while they are being iterated on, causing invalid indices to be accessed.

The crash scenario was examined and described by an engine licensee on the linked EPS case. Callstacks are included for both the crash site and the moment where components get unregistered from the scalability manager during the iteration performed by ProcessSignificance(). The chain of events that lead to this can be described as follows:

1. During ProcessSignificance, a component is detected that needs to be culled, so CreateCullProxy is called.
2. When the NiagaraCullProxyComponent is registered, it is specifically told to unpause via SetPaused
3. This unpause call gets propagated up a few layers through the UNiagaraComponent, FNiagaraSystemInstanceController, FNiagaraSystemInstance, and eventually to FNiagaraSystemSimulation.
4. As part of FNiagaraSystemSimulation::UnpauseInstance, there is a call to WaitForInstancesTickComplete. This blocks the game thread until all tick operations finish.
5. One of those tick operations can result in a component being completed, possibly at the end of its lifetime. Consequently, it immediately invokes FNiagaraSystemInstance::HandleCompletion and begins cleaning itself up.
6. As part of that completion process, the NiagaraComponent is reclaimed by the pooling system, so it gets unregistered from the scalability manager.

The linked case also includes some suggestions about how to handle the situation.

Preliminary work on this issue included an attempt to generate a repro project (UE 5.4). Unfortunately, however, repro attempts were unsuccessful so far. The project is provided as a possible starting point for further investigation.

The issue occurred when rapidly spawning several short-lived Niagara Systems (for bullet impact effects) with component pooling and aggressive culling using the "Kill" reaction type. The following repro steps were suggested:

1. Create a Niagara system with a short lifetime, less than a second
2. Set it to use a new NiagaraEffectType. Inside it, set the Cull Reaction type to "Kill" and the System Scalability settings to limit the max instances to a low number and the cull proxy mode to "Instanced Rendered".
3. Use the level blueprint to rapidly spawn these niagara systems over time, making sure to enable pooling for the components on spawn.
4. When playing, a crash can occur when an array is accessed with an out-of-bounds index.

The provided repro project attempt (which does not repro the issue in its current state) spawns one system per tick at 120fps. Each system spawns a burst of particles that last a random time from 0.05s to 0.5s. The Scalability Settings for the Effect Type limit the number of instances to 30 and uses "age" for relevance.

-------------------------------------

-------------------------------------

Note: Line numbers for FNiagaraScalabilityManager may not be accurate due to debug code added to generate this callstack.

UnrealEditor-Niagara.dll! FNiagaraScalabilityManager:: UnregisterAt::__l2::<lambda_4>::operator()() Line 169 C++
UnrealEditor-Niagara.dll! FNiagaraScalabilityManager:: UnregisterAt(int IndexToRemove) Line 169 C++
UnrealEditor-Niagara.dll! FNiagaraWorldManager:: UnregisterWithScalabilityManager(UNiagaraComponent * Component, UNiagaraEffectType * EffectType) Line 1614 C++
UnrealEditor-Niagara.dll! UNiagaraComponent:: UnregisterWithScalabilityManager() Line 1626 C++
UnrealEditor-Niagara.dll! UNiagaraComponent:: DeactivateImmediateInternal(bool bIsScalabilityCull) Line 1564 C++
UnrealEditor-Niagara.dll! FNCPool:: Reclaim(UNiagaraComponent * Component, const double CurrentTimeSeconds) Line 135 C++
UnrealEditor-Niagara.dll! UNiagaraComponentPool:: ReclaimWorldParticleSystem(UNiagaraComponent * Component) Line 499 C++
UnrealEditor-Niagara.dll! UNiagaraComponent:: OnSystemComplete(bool bExternalCompletion) Line 1741 C++
UnrealEditor-Niagara.dll! FNiagaraSystemInstance:: Complete(bool bExternalCompletion) Line 707 C++
UnrealEditor-Niagara.dll! FNiagaraSystemInstance:: HandleCompletion() Line 2435 C++
UnrealEditor-Niagara.dll! FNiagaraSystemInstance:: FinalizeTick_GameThread(bool bEnqueueGPUTickIfNeeded) Line 2714 C++
UnrealEditor-Niagara.dll! FNiagaraSystemInstance:: WaitForConcurrentTickAndFinalize(bool) Line 2418 C++
UnrealEditor-Niagara.dll! FNiagaraSystemSimulation:: WaitForInstancesTickComplete(bool bEnsureComplete) Line 1670 C++
UnrealEditor-Niagara.dll! FNiagaraSystemSimulation:: UnpauseInstance(FNiagaraSystemInstance *) Line 2222 C++
UnrealEditor-Niagara.dll! FNiagaraSystemInstance:: SetPaused(bool bInPaused) Line 747 C++
UnrealEditor-Niagara.dll! FNiagaraSystemInstanceController:: SetPaused(bool) Line 138 C++
UnrealEditor-Niagara.dll! UNiagaraComponent:: SetPausedInternal(bool bInPaused, bool bIsScalabilityCull) Line 1090 C++
UnrealEditor-Niagara.dll! UNiagaraComponent:: SetPaused(bool bInPaused) Line 1064 C++
UnrealEditor-Niagara.dll! UNiagaraCullProxyComponent:: RegisterCulledComponent(UNiagaraComponent * Component, bool bForce) Line 80 C++
UnrealEditor-Niagara.dll! UNiagaraComponent:: CreateCullProxy(bool bForce) Line 2148 C++
UnrealEditor-Niagara.dll! FNiagaraScalabilityManager:: ProcessSignificance(FNiagaraWorldManager * WorldMan, UNiagaraSignificanceHandler * SignificanceHandler, FComponentIterationContext & Context) Line 335 C++
UnrealEditor-Niagara.dll! FNiagaraScalabilityManager:: UpdateInternal(FNiagaraWorldManager * WorldMan, FComponentIterationContext & Context) Line 459 C++
UnrealEditor-Niagara.dll! FNiagaraScalabilityManager:: Update(FNiagaraWorldManager * WorldMan, float DeltaSeconds, bool bNewOnly) Line 526 C++
UnrealEditor-Niagara.dll! TConstSetBitIterator<FDefaultBitArrayAllocator>::operator++() Line 1827 C++
UnrealEditor-Niagara.dll! TSparseArray<TSetElement<TTuple<TObjectPtr<UNiagaraEffectType>,FNiagaraScalabilityManager>>,TSparseArrayAllocator<TSizedDefaultAllocator<32>,FDefaultBitArrayAllocator>>::TBaseIterator<0>::operator++() Line 891 C++
UnrealEditor-Niagara.dll! TSet<TTuple<TObjectPtr<UNiagaraEffectType>,FNiagaraScalabilityManager>,TDefaultMapHashableKeyFuncs<TObjectPtr<UNiagaraEffectType>,FNiagaraScalabilityManager,0>,FDefaultSetAllocator>::TBaseIterator<0,1>::operator++() Line 1597 C++
UnrealEditor-Niagara.dll! TMapBase<TObjectPtr<UNiagaraEffectType>,FNiagaraScalabilityManager,FDefaultSetAllocator,TDefaultMapHashableKeyFuncs<TObjectPtr<UNiagaraEffectType>,FNiagaraScalabilityManager,0>>::TBaseIterator<0,1>::operator++() Line 798 C++
UnrealEditor-Niagara.dll! FNiagaraWorldManager:: UpdateScalabilityManagers(float DeltaSeconds, bool bNewSpawnsOnly) Line 1559 C++
UnrealEditor-Niagara.dll! FNiagaraWorldManager:: Tick(ETickingGroup TickGroup, float DeltaSeconds, ELevelTick TickType, ENamedThreads::Type CurrentThread, const TRefCountPtr<FGraphEvent> & MyCompletionGraphEvent) Line 1452 C++
UnrealEditor-Niagara.dll! FNiagaraWorldManagerTickFunction:: ExecuteTick(float DeltaTime, ELevelTick TickType, ENamedThreads::Type CurrentThread, const TRefCountPtr<FGraphEvent> & MyCompletionGraphEvent) Line 327 C++
UnrealEditor-Engine.dll! FTickFunctionTask:: DoTask(ENamedThreads::Type CurrentThread, const TRefCountPtr<FGraphEvent> & MyCompletionGraphEvent) Line 290 C++
UnrealEditor-Engine.dll! TGraphTask<FTickFunctionTask>:: ExecuteTask(TArray<FBaseGraphTask *,TSizedDefaultAllocator<32>> & NewTasks, ENamedThreads::Type CurrentThread, bool bDeleteOnCompletion) Line 1233 C++
UnrealEditor-Core.dll! FBaseGraphTask:: Execute(TArray<FBaseGraphTask *,TSizedDefaultAllocator<32>> & CurrentThread, ENamedThreads::Type) Line 838 C++
UnrealEditor-Core.dll! FNamedTaskThread:: ProcessTasksNamedThread(int QueueIndex, bool bAllowStall) Line 779 C++
UnrealEditor-Core.dll! FNamedTaskThread:: ProcessTasksUntilQuit(int QueueIndex) Line 670 C++
UnrealEditor-Core.dll! FTaskGraphCompatibilityImplementation:: ProcessThreadUntilRequestReturn(ENamedThreads::Type CurrentThread) Line 2062 C++
UnrealEditor-Core.dll! FTaskGraphCompatibilityImplementation:: WaitUntilTasksComplete(const TArray<TRefCountPtr<FGraphEvent>,TSizedInlineAllocator<4,32,TSizedDefaultAllocator<32>>> & Tasks, ENamedThreads::Type CurrentThreadIfKnown) Line 2135 C++
UnrealEditor-Engine.dll! FTickTaskSequencer:: ReleaseTickGroup(ETickingGroup WorldTickGroup, bool bBlockTillComplete) Line 572 C++
UnrealEditor-Engine.dll! FTickTaskManager:: RunTickGroup(ETickingGroup Group, bool bBlockTillComplete) Line 1649 C++
UnrealEditor-Engine.dll! UWorld:: RunTickGroup(ETickingGroup Group, bool bBlockTillComplete) Line 788 C++
UnrealEditor-Engine.dll! UWorld:: Tick(ELevelTick TickType, float DeltaSeconds) Line 1524 C++

-------------------------

-------------------------

sertion failed: (Index >= 0) & (Index < ArrayNum) [Link Removed] [Line: 758]
Array index out of bounds: 10 into an array of size 10

FNiagaraScalabilityManager::ProcessSignificance() [Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraScalabilityManager.cpp:287]
FNiagaraScalabilityManager::Update() [Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraScalabilityManager.cpp:516]
FNiagaraWorldManager::UpdateScalabilityManagers() [Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraWorldManager.cpp:1559]
FNiagaraWorldManager::Tick() [Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraWorldManager.cpp:1452]
FNiagaraWorldManagerTickFunction::ExecuteTick() [Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraWorldManager.cpp:327]
FTickFunctionTask::DoTask() [Engine\Source\Runtime\Engine\Private\TickTaskManager.cpp:283]
TGraphTask<FTickFunctionTask>::ExecuteTask() [Engine\Source\Runtime\Core\Public\Async\TaskGraphInterfaces.h:1236]
FNamedTaskThread::ProcessTasksNamedThread() [Engine\Source\Runtime\Core\Private\Async\TaskGraph.cpp:779]
FNamedTaskThread::ProcessTasksUntilQuit() [Engine\Source\Runtime\Core\Private\Async\TaskGraph.cpp:670]
FTaskGraphCompatibilityImplementation::WaitUntilTasksComplete() [Engine\Source\Runtime\Core\Private\Async\TaskGraph.cpp:2135]
FTickTaskSequencer::ReleaseTickGroup() [Engine\Source\Runtime\Engine\Private\TickTaskManager.cpp:572]
FTickTaskManager::RunTickGroup() [Engine\Source\Runtime\Engine\Private\TickTaskManager.cpp:1649]
UWorld::RunTickGroup() [Engine\Source\Runtime\Engine\Private\LevelTick.cpp:788]
UWorld::Tick() [Engine\Source\Runtime\Engine\Private\LevelTick.cpp:1525]
UGameEngine::Tick() [Engine\Source\Runtime\Engine\Private\GameEngine.cpp:1876]
FEngineLoop::Tick() [Engine\Source\Runtime\Launch\Private\LaunchEngineLoop.cpp:5975]
GuardedMain() [Engine\Source\Runtime\Launch\Private\Launch.cpp:180]
GuardedMainWrapper() [Engine\Source\Runtime\Launch\Private\Windows\LaunchWindows.cpp:134]
LaunchWindowsStartup() [Engine\Source\Runtime\Launch\Private\Windows\LaunchWindows.cpp:274]
WinMain() [Engine\Source\Runtime\Launch\Private\Windows\LaunchWindows.cpp:315]