Description

During GC, if a file was manually deleted (but the storage system is still track), GC can erroneously fail (and never remove the reference). This causes a backup in GC, putting the whole algorithm in a bad state for the namespace. We should cleanup the dangling reference, and proceed to other items for GC.

1. System.IO.DirectoryNotFoundException: Could not find a part of the path '/app/Data/Storage/artifacts2/ugs-pcb/ftw-release-0.3.0.0/119528/editor/680a78a8fe720458f0c72627/8bf116012d8243fc823394c3dbcb184c_7.blob'.
2. at System.IO.FileSystem.DeleteFile(String fullPath)
3. at EpicGames.Core.MemoryMappedFileCache.Delete(FileReference file) in /app/Source/Programs/Shared/EpicGames.Core/MemoryMappedFileCache.cs:line 235
4. at EpicGames.Horde.Storage.ObjectStores.PrefixedObjectStore.DeleteAsync(ObjectKey locator, CancellationToken cancellationToken) in /app/Source/Programs/Shared/EpicGames.Horde/Storage/ObjectStores/PrefixedObjectStore.cs:line 38
5. at HordeServer.Storage.StorageService.CheckReachabilityAsync(NamespaceInfo namespaceInfo, SortedSetEntry`1 entry, ObjectId lastImportBlobInfoId, GcSweepState state, StorageConfig storageConfig, AsyncEvent queueChangeEvent, CancellationToken cancellationToken) in /app> at EpicGames.Core.AsyncPipelineExtensions.ProcessItemsAsync[T](ChannelReader`1 reader, Func`3 taskFunc, CancellationToken cancellationToken) in /app/Source/Programs/Shared/EpicGames.Core/AsyncPipeline.cs:line 202
6. at EpicGames.Core.AsyncPipeline.RunGuardedAsync(Func`2 taskFunc) in /app/Source/Programs/Shared/EpicGames.Core/AsyncPipeline.cs:line 97
7. at EpicGames.Core.AsyncPipeline.WaitForCompletionAsync() in /app/Source/Programs/Shared/EpicGames.Core/AsyncPipeline.cs:line 125
8. at HordeServer.Storage.StorageService.TickGcForNamespaceAsync(NamespaceInfo namespaceInfo, ObjectId lastImportBlobInfoId, DateTime utcNow, CancellationToken cancellationToken) in /app/Source/Programs/Horde/Plugins/Storage/HordeServer.Storage/Storage/StorageService.cs> at HordeServer.Storage.StorageService.TickGcForNamespaceAsync(NamespaceInfo namespaceInfo, ObjectId lastImportBlobInfoId, DateTime utcNow, CancellationToken cancellationToken) in /app/Source/Programs/Horde/Plugins/Storage/HordeServer.Storage/Storage/StorageService.cs> at HordeServer.Storage.StorageService.TickGcAsync(CancellationToken cancellationToken) in /app/Source/Programs/Horde/Plugins/Storage/HordeServer.Storage/Storage/StorageService.cs:line 1462

Have Comments or More Details?

There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-278395 in the post.

0
Login to Vote

Unresolved
ComponentUE - Foundation - Horde - Storage
Affects Versions5.5
CreatedMay 7, 2025
UpdatedMay 26, 2025
View Jira Issue