Description

IO cost of directory scans is one of the costs present in the AssetRegistry when a large map is resaved. Maps can have external actors which all have to be rescanned and reported to the map for inclusion in its dependencies.

This cost shows up in the attached callstack. There is a parallel for, but it is only over the directories reported as modified, and does not parallelize the scans of subdirectories within each top level directory.

When discovering files normally during editor startup, we parallelize the directory scan as well, adding each discovered subdirectory to a queue shared between threads and each thread does a non-recursive scan of the next directory read from the queue. In some cases the number is so high that just scanning the directories takes up too much time for a single thread. This is probably due to the large directory structure created for a large map; ExternalActors are divided into many subdirectories to avoid IO performance problems when a single directory becomes too big.

Change FAssetRegistryImpl::OnDirectoryRescanRequired to call into the gatherer to do the scan rather than to keep its less optimized custom implementation.

Callstack
UAssetRegistryImpl::OnDirectoryChanged
UE::AssetRegistry::FAssetRegistryImpl::OnDirectoryChanged
UE::AssetRegistry::FAssetRegistryImpl::OnDirectoryRescanRequired
ParallelFor
ParallelForImpl::ParallelForInternal<TFunctionRef<void __cdecl(int)>,`ParallelFor'::`2'::<lambda_1>,std::nullptr_t>
static void `UE::AssetRegistry::FAssetRegistryImpl::OnDirectoryRescanRequired'::`2'::<lambda_2>::operator()
FPackageName::IteratePackagesInDirectory
IPackageResourceManager::IteratePackagesStatInPath
FPackageResourceManagerFile::IteratePackagesStatInPath
IPlatformFile::IterateDirectoryStatRecursively
FWindowsPlatformFile::IterateDirectoryStat
FWindowsPlatformFile::IterateDirectoryCommon
static bool `FWindowsPlatformFile::IterateDirectoryStat'::`2'::<lambda_1>::operator()
IPlatformFile::FDirectoryStatVisitor::CallShouldVisitAndVisit
static bool `IPlatformFile::IterateDirectoryStatRecursively'::`2'::FStatRecurse::Visit
FWindowsPlatformFile::IterateDirectoryStat
FWindowsPlatformFile::IterateDirectoryCommon
static bool `FWindowsPlatformFile::IterateDirectoryStat'::`2'::<lambda_1>::operator()
IPlatformFile::FDirectoryStatVisitor::CallShouldVisitAndVisit
static bool `IPlatformFile::IterateDirectoryStatRecursively'::`2'::FStatRecurse::Visit
FWindowsPlatformFile::IterateDirectoryStat

Have Comments or More Details?

There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-254606 in the post.

0
Login to Vote

Unresolved
CreatedMar 10, 2025
UpdatedMar 11, 2025
View Jira Issue