Unreal Engine Issues

UE-250551

Clean DocumentDB setup fails indexing

Clean DocumentDB setup fails indexing

Description

Source [Link Removed]

The user has encountered the error log:
MongoDB.Driver.MongoCommandException: Command createIndexes failed: Implicit collection creation not supported when creating indexes

It potentially has to do with a DocumentDB from clean edge case.

Steps to Reproduce

Extrapolated from UDN:

  • Setup new horde server (AWS ECR - for reference)
  • Setup a DocumentDB database to use; clean (no incremental)
  • Start the horde server
  • Server log shows multiple:
    • MongoDB.Driver.MongoCommandException: Command createIndexes failed: Implicit collection creation not supported when creating indexes
Callstack

1739973257294,[13:54:17 err] Exception while executing scheduled event,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"System.TimeoutException: A timeout occurred after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference =

{ Mode : Primary }

}, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 }, OperationsCountServerSelector }. Client view of cluster state is { ClusterId : ""1"", Type : ""Unknown"", State : ""Disconnected"", Servers : [{ ServerId: ""

{ ClusterId : 1, EndPoint : ""Unspecified/horde-paxdei-762131802939.eu-central-1.docdb-elastic.amazonaws.com:27017"" }

"", EndPoint: ""Unspecified/horde-paxdei-762131802939.eu-central-1.docdb-elastic.amazonaws.com:27017"", ReasonChanged: ""ServerInitialDescription"", State: ""Disconnected"", ServerVersion: , TopologyVersion: , Type: ""Unknown"", LastHeartbeatTimestamp: null, LastUpdateTimestamp: ""2025-02-19T13:53:32.1311850Z"" }] }.",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedHelper.HandleCompletedTask(Task completedTask),ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedAsync(IServerSelector selector, ClusterDescription description, Task descriptionChangedTask, TimeSpan timeout, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.Core.Clusters.Cluster.SelectServerAsync(IServerSelector selector, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.Core.Clusters.IClusterExtensions.SelectServerAndPinIfNeededAsync(ICluster cluster, ICoreSessionHandle session, IServerSelector selector, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,   at MongoDB.Driver.Core.Bindings.ReadPreferenceBinding.GetReadChannelSourceAsync(CancellationToken cancellationToken),ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,   at MongoDB.Driver.Core.Operations.RetryableReadContext.InitializeAsync(CancellationToken cancellationToken),ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.Core.Operations.RetryableReadContext.CreateAsync(IReadBinding binding, Boolean retryRequested, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(IReadBinding binding, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.OperationExecutor.ExecuteReadOperationAsync[TResult](IReadBinding binding, IReadOperation`1 operation, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.MongoCollectionImpl`1.ExecuteReadOperationAsync[TResult](IClientSessionHandle session, IReadOperation`1 operation, ReadPreference readPreference, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at HordeServer.Utilities.MongoTracingCollection`1.FindAsync[TProjection](FilterDefinition`1 filter, FindOptions`2 options, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Utilities/MongoTracingCollection.cs:line 348",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at MongoDB.Driver.IAsyncCursorSourceExtensions.FirstOrDefaultAsync[TDocument](IAsyncCursorSource`1 source, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at HordeServer.Server.MongoService.GetSingletonAsync[T](Func`1 constructor, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Server/MongoService.cs:line 821",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at HordeServer.Server.MongoService.SetSchemaVersionAsync(SemVer schemaVersion, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Server/MongoService.cs:line 755",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at HordeServer.Server.MongoService.UpdateIndexesAsync[T](String collectionName, IMongoCollection`1 collection, MongoIndex`1[] newIndexes, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Server/MongoService.cs:line 624",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at HordeServer.Utilities.MongoTracingCollection`1.FindAsync[TProjection](FilterDefinition`1 filter, FindOptions`2 options, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Utilities/MongoTracingCollection.cs:line 346",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at HordeServer.Utilities.MongoExtensions.FindWithHintAsync[TDoc,TResult](IMongoCollection`1 collection, FilterDefinition`1 filter, String indexHint, Func`2 processAsync) in /app/Source/Programs/Horde/HordeServer.Shared/Utilities/MongoExtensions.cs:line 75",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,   at HordeServer.Jobs.Bisect.BisectTaskCollection.FindActiveAsync(CancellationToken cancellationToken)+MoveNext() in /app/Source/Programs/Horde/Plugins/Build/HordeServer.Build/Jobs/Bisect/BisectTaskCollection.cs:line 181,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,   at HordeServer.Jobs.Bisect.BisectTaskCollection.FindActiveAsync(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult(),ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,   at HordeServer.Jobs.Bisect.BisectService.TickAsync(CancellationToken cancellationToken) in /app/Source/Programs/Horde/Plugins/Build/HordeServer.Build/Jobs/Bisect/BisectService.cs:line 59,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,   at HordeServer.Jobs.Bisect.BisectService.TickAsync(CancellationToken cancellationToken) in /app/Source/Programs/Horde/Plugins/Build/HordeServer.Build/Jobs/Bisect/BisectService.cs:line 59,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at HordeCommon.Clock.TriggerSharedAsync(RedisKey key, TimeSpan interval, Func`2 tickAsync, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Utilities/Clock.cs:line 156",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,   at EpicGames.Core.ClockExtensions.<>c_DisplayClass0_0.<<AddTicker>g_WrappedTrigger|0>d.MoveNext() in /app/Source/Programs/Shared/EpicGames.Core/Clock.cs:line 161,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,'"--- End of stack trace from previous location ---",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257294,"   at HordeCommon.Clock.TickerImpl.RunAsync(TimeSpan delay, Func`2 triggerAsync, ILogger logger) in /app/Source/Programs/Horde/HordeServer/Utilities/Clock.cs:line 71",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,[13:54:17 err] Exception while executing scheduled event,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"System.TimeoutException: A timeout occurred after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference =

{ Mode : Primary }

}, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 }, OperationsCountServerSelector }. Client view of cluster state is { ClusterId : ""1"", Type : ""Unknown"", State : ""Disconnected"", Servers : [{ ServerId: ""

{ ClusterId : 1, EndPoint : ""Unspecified/horde-paxdei-762131802939.eu-central-1.docdb-elastic.amazonaws.com:27017"" }

"", EndPoint: ""Unspecified/horde-paxdei-762131802939.eu-central-1.docdb-elastic.amazonaws.com:27017"", ReasonChanged: ""ServerInitialDescription"", State: ""Disconnected"", ServerVersion: , TopologyVersion: , Type: ""Unknown"", LastHeartbeatTimestamp: null, LastUpdateTimestamp: ""2025-02-19T13:53:32.1311850Z"" }] }.",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedHelper.HandleCompletedTask(Task completedTask),ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedAsync(IServerSelector selector, ClusterDescription description, Task descriptionChangedTask, TimeSpan timeout, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.Core.Clusters.Cluster.SelectServerAsync(IServerSelector selector, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.Core.Clusters.IClusterExtensions.SelectServerAndPinIfNeededAsync(ICluster cluster, ICoreSessionHandle session, IServerSelector selector, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,   at MongoDB.Driver.Core.Bindings.ReadPreferenceBinding.GetReadChannelSourceAsync(CancellationToken cancellationToken),ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,   at MongoDB.Driver.Core.Operations.RetryableReadContext.InitializeAsync(CancellationToken cancellationToken),ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.Core.Operations.RetryableReadContext.CreateAsync(IReadBinding binding, Boolean retryRequested, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(IReadBinding binding, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.OperationExecutor.ExecuteReadOperationAsync[TResult](IReadBinding binding, IReadOperation`1 operation, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.MongoCollectionImpl`1.ExecuteReadOperationAsync[TResult](IClientSessionHandle session, IReadOperation`1 operation, ReadPreference readPreference, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at HordeServer.Utilities.MongoTracingCollection`1.FindAsync[TProjection](FilterDefinition`1 filter, FindOptions`2 options, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Utilities/MongoTracingCollection.cs:line 348",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.IAsyncCursorSourceExtensions.FirstOrDefaultAsync[TDocument](IAsyncCursorSource`1 source, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at HordeServer.Server.MongoService.GetSingletonAsync[T](Func`1 constructor, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Server/MongoService.cs:line 821",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at HordeServer.Server.MongoService.SetSchemaVersionAsync(SemVer schemaVersion, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Server/MongoService.cs:line 755",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at HordeServer.Server.MongoService.UpdateIndexesAsync[T](String collectionName, IMongoCollection`1 collection, MongoIndex`1[] newIndexes, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Server/MongoService.cs:line 624",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at HordeServer.Utilities.MongoTracingCollection`1.FindAsync[TProjection](FilterDefinition`1 filter, FindOptions`2 options, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Utilities/MongoTracingCollection.cs:line 346",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at MongoDB.Driver.IAsyncCursorSourceExtensions.ToListAsync[TDocument](IAsyncCursorSource`1 source, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,   at HordeServer.Agents.AgentCollection.DeleteExpiredEphemeralAgentsAsync(CancellationToken cancellationToken) in /app/Source/Programs/Horde/Plugins/Compute/HordeServer.Compute/Agents/AgentCollection.cs:line 360,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,   at HordeServer.Agents.AgentCollection.TickSharedAsync(CancellationToken stoppingToken) in /app/Source/Programs/Horde/Plugins/Compute/HordeServer.Compute/Agents/AgentCollection.cs:line 324,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at HordeCommon.Clock.TriggerSharedAsync(RedisKey key, TimeSpan interval, Func`2 tickAsync, CancellationToken cancellationToken) in /app/Source/Programs/Horde/HordeServer/Utilities/Clock.cs:line 156",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,   at EpicGames.Core.ClockExtensions.<>c_DisplayClass0_0.<<AddTicker>g_WrappedTrigger|0>d.MoveNext() in /app/Source/Programs/Shared/EpicGames.Core/Clock.cs:line 161,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,'"--- End of stack trace from previous location ---",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973257303,"   at HordeCommon.Clock.TickerImpl.RunAsync(TimeSpan delay, Func`2 triggerAsync, ILogger logger) in /app/Source/Programs/Horde/HordeServer/Utilities/Clock.cs:line 71",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346,"[13:54:32 err] Exception while expiring reservations: A timeout occurred after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference =

{ Mode : Primary }

}, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 }, OperationsCountServerSelector }. Client view of cluster state is { ClusterId : ""1"", Type : ""Unknown"", State : ""Disconnected"", Servers : [{ ServerId: ""

{ ClusterId : 1, EndPoint : ""Unspecified/horde-paxdei-762131802939.eu-central-1.docdb-elastic.amazonaws.com:27017"" }

"", EndPoint: ""Unspecified/horde-paxdei-762131802939.eu-central-1.docdb-elastic.amazonaws.com:27017"", ReasonChanged: ""Heartbeat"", State: ""Disconnected"", ServerVersion: , TopologyVersion: , Type: ""Unknown"", HeartbeatException: ""MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server.",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346, ---> MongoDB.Driver.MongoConnectionException: An exception occurred while receiving a message from the server.,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346, ---> System.IO.IOException: Unable to read data from the transport connection: Connection timed out.,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346, ---> System.Net.Sockets.SocketException (110): Connection timed out,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346,"   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 count)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346,   — End of inner exception stack trace ---,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346,"   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 count)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346,"   at MongoDB.Driver.Core.Misc.StreamExtensionMethods.ReadBytes(Stream stream, Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)",ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346,   at MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveBuffer(CancellationToken cancellationToken),ecs/horde-server/6b96b4b76fd245e383ed268568dbd031
1739973272346,   — End of inner exception stack trace ---,ecs/horde-server/6b96b4b76fd245e383ed268568dbd031

Have Comments or More Details?

There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-250551 in the post.

0
Login to Vote
Unresolved
ComponentUE - Foundation - Horde - Server
Target Fix5.7
CreatedFeb 21, 2025
UpdatedApr 23, 2025
View Jira Issue