MongoDB configuration consideration

Hosting (OnPremise)
1

Hi Sebastian

We are currently setting up a mongodb cluster where we want to run multiple squidex instances on top of.

Do you see any limitation in that, as long as the evenstore and content database has different names, based on the squidex instances?

Also can see in the configuration of the store there is reference to the eventstore, what kind of access to the store require to eventstore ?

Last, but not least :slight_smile: as part of migration to the new cluster we have to rename some of the current databases, which has been running by them self for now, do you see any problems in that ?

2

You do not have to use different databases. This comes from a time when there was one content database per app.

You can also rename them, I don’t see any issues.

3

Thanks, we are trying to make it work with SSL, but getting:

—> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
2020-04-14T18:34:46.108515683Z at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)

Is there anything that needs to be configured from Squidex in order to support SSL, or is it primarily a MongoDB configuration?

4

SSL for MongoDB should be only a config thing.

5

Okay thanks, i’ll sent it back to the Database team :slight_smile:

6

@Sebastian do you have any experience/example of running Squidex using a certificate towards mongodb

7

No, sorry… I am hosting squidex inside the same kubernetes cluster and have only basic authentication in place. But others use it with Mongodb Atlas and it works fine as well.

8

Okay, do you know what the normal behaviour is to access to the certificate? is it to add it to image or to mount a datastorage with access to it?

9

I would mount it, because it is dependent on your deployment.

10

Okay thanks for the help

11

@Sebastian we have just tested out the new cluster, but i get a: MongoDB.Driver.MongoCommandException: Command createIndexes failed: Index with name: Timestamp_1 already exists with different options.

Have you any idea what that could be?

12

In which collection?

13 
2020-04-21T18:47:44.362605752Z Unhandled exception. Squidex.Infrastructure.ConfigurationException: MongoDb connection failed to connect to database SquidexEventstore
2020-04-21T18:47:44.362628352Z  ---> MongoDB.Driver.MongoCommandException: Command createIndexes failed: Index with name: Timestamp_1 already exists with different options.
2020-04-21T18:47:44.362633652Z    at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol`1.ProcessResponse(ConnectionId connectionId, CommandMessage responseMessage)
2020-04-21T18:47:44.362637152Z    at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol`1.ExecuteAsync(IConnection connection, CancellationToken cancellationToken)
2020-04-21T18:47:44.362647052Z    at MongoDB.Driver.Core.Servers.Server.ServerChannel.ExecuteProtocolAsync[TResult](IWireProtocol`1 protocol, ICoreSession session, CancellationToken cancellationToken)
2020-04-21T18:47:44.362650652Z    at MongoDB.Driver.Core.Operations.CommandOperationBase`1.ExecuteProtocolAsync(IChannelSource channelSource, ICoreSessionHandle session, ReadPreference readPreference, CancellationToken cancellationToken)
2020-04-21T18:47:44.362654052Z    at MongoDB.Driver.Core.Operations.WriteCommandOperation`1.ExecuteAsync(IWriteBinding binding, CancellationToken cancellationToken)
2020-04-21T18:47:44.362657352Z    at MongoDB.Driver.Core.Operations.CreateIndexesUsingCommandOperation.ExecuteAsync(IWriteBinding binding, CancellationToken cancellationToken)
2020-04-21T18:47:44.362660652Z    at MongoDB.Driver.Core.Operations.CreateIndexesOperation.ExecuteAsync(IWriteBinding binding, CancellationToken cancellationToken)
2020-04-21T18:47:44.362663952Z    at MongoDB.Driver.OperationExecutor.ExecuteWriteOperationAsync[TResult](IWriteBinding binding, IWriteOperation`1 operation, CancellationToken cancellationToken)
2020-04-21T18:47:44.362667252Z    at MongoDB.Driver.MongoCollectionImpl`1.ExecuteWriteOperationAsync[TResult](IClientSessionHandle session, IWriteOperation`1 operation, CancellationToken cancellationToken)
2020-04-21T18:47:44.362670552Z    at MongoDB.Driver.MongoCollectionImpl`1.MongoIndexManager.CreateManyAsync(IClientSessionHandle session, IEnumerable`1 models, CreateManyIndexesOptions options, CancellationToken cancellationToken)
2020-04-21T18:47:44.362673952Z    at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)
2020-04-21T18:47:44.362677952Z    at Squidex.Infrastructure.MongoDb.MongoRepositoryBase`1.InitializeAsync(CancellationToken ct) in /src/src/Squidex.Infrastructure.MongoDb/MongoDb/MongoRepositoryBase.cs:line 109
2020-04-21T18:47:44.362681452Z    --- End of inner exception stack trace ---
2020-04-21T18:47:44.362684652Z    at Squidex.Infrastructure.MongoDb.MongoRepositoryBase`1.InitializeAsync(CancellationToken ct) in /src/src/Squidex.Infrastructure.MongoDb/MongoDb/MongoRepositoryBase.cs:line 113
2020-04-21T18:47:44.362688052Z    at Squidex.Config.Startup.InitializerHost.StartAsync(ISemanticLog log, CancellationToken ct) in /src/src/Squidex/Config/Startup/InitializerHost.cs:line 33
2020-04-21T18:47:44.362691352Z    at Squidex.Config.Startup.SafeHostedService.StartAsync(CancellationToken cancellationToken) in /src/src/Squidex/Config/Startup/SafeHostedService.cs:line 29
2020-04-21T18:47:44.362694653Z    at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
2020-04-21T18:47:44.362697853Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
2020-04-21T18:47:44.362701253Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
2020-04-21T18:47:44.362704453Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
2020-04-21T18:47:44.367460899Z    at Squidex.Program.Main(String[] args) in /src/src/Squidex/Program.cs:line 23
14

We are running multiple databases on the same cluster, don’t know if that can be a problem

15

As long as you don’t run multiple instances on the same database.

Before this log should be something with “initializedSystem” … can you give me that?

16

2020-04-21T18:47:43.941640786Z “app”: {
2020-04-21T18:47:43.941685986Z “name”: “Squidex”,
2020-04-21T18:47:43.941701286Z “version”: “4.2.0.0”,
2020-04-21T18:47:43.941741387Z “sessionId”: “d64d099c-78e7-4661-b933-035dc0f04bbf”
2020-04-21T18:47:43.941756387Z },
2020-04-21T18:47:43.941785587Z “timestamp”: “2020-04-21T18:47:43Z”
2020-04-21T18:47:43.941799087Z }
2020-04-21T18:47:43.941843488Z
2020-04-21T18:47:44.137827381Z {
2020-04-21T18:47:44.137848181Z “logLevel”: “Information”,
2020-04-21T18:47:44.137853181Z “initializedSystem”: “AzureBlobAssetStore”,
2020-04-21T18:47:44.137856781Z “app”: {
2020-04-21T18:47:44.137859881Z “name”: “Squidex”,
2020-04-21T18:47:44.137863281Z “version”: “4.2.0.0”,
2020-04-21T18:47:44.137866481Z “sessionId”: “d64d099c-78e7-4661-b933-035dc0f04bbf”
2020-04-21T18:47:44.137878481Z },
2020-04-21T18:47:44.137882281Z “timestamp”: “2020-04-21T18:47:44Z”
2020-04-21T18:47:44.137885581Z }
2020-04-21T18:47:44.137888681Z
2020-04-21T18:47:44.302698573Z {
2020-04-21T18:47:44.302719873Z “logLevel”: “Information”,
2020-04-21T18:47:44.302725373Z “initializedSystem”: “MongoEventStore”,
2020-04-21T18:47:44.302729473Z “app”: {
2020-04-21T18:47:44.302733173Z “name”: “Squidex”,
2020-04-21T18:47:44.302736973Z “version”: “4.2.0.0”,
2020-04-21T18:47:44.302751174Z “sessionId”: “d64d099c-78e7-4661-b933-035dc0f04bbf”
2020-04-21T18:47:44.302756974Z },
2020-04-21T18:47:44.302760474Z “timestamp”: “2020-04-21T18:47:44Z”
2020-04-21T18:47:44.302764374Z }
2020-04-21T18:47:44.302767774Z
2020-04-21T18:47:44.306784013Z {
2020-04-21T18:47:44.306798113Z “logLevel”: “Information”,
2020-04-21T18:47:44.306803113Z “initializedSystem”: “SerializationInitializer”,
2020-04-21T18:47:44.306807113Z “app”: {
2020-04-21T18:47:44.306819313Z “name”: “Squidex”,
2020-04-21T18:47:44.306823213Z “version”: “4.2.0.0”,
2020-04-21T18:47:44.306826713Z “sessionId”: “d64d099c-78e7-4661-b933-035dc0f04bbf”
2020-04-21T18:47:44.306830313Z },
2020-04-21T18:47:44.306843913Z “timestamp”: “2020-04-21T18:47:44Z”
2020-04-21T18:47:44.306847613Z }
2020-04-21T18:47:44.306850913Z
2020-04-21T18:47:44.311139355Z {
2020-04-21T18:47:44.311155855Z “logLevel”: “Information”,
2020-04-21T18:47:44.311161755Z “initializedSystem”: “MongoMigrationStatus”,
2020-04-21T18:47:44.311166255Z “app”: {
2020-04-21T18:47:44.311170555Z “name”: “Squidex”,
2020-04-21T18:47:44.311175055Z “version”: “4.2.0.0”,
2020-04-21T18:47:44.311179355Z “sessionId”: “d64d099c-78e7-4661-b933-035dc0f04bbf”
2020-04-21T18:47:44.311183755Z },
2020-04-21T18:47:44.311235155Z “timestamp”: “2020-04-21T18:47:44Z”
2020-04-21T18:47:44.311239756Z }
2020-04-21T18:47:44.311243856Z
2020-04-21T18:47:44.338763621Z {
2020-04-21T18:47:44.338777022Z “logLevel”: “Error”,
2020-04-21T18:47:44.338781622Z “message”: “QueueWorkItem was called on a non-null context [SystemTarget: S127.0.0.1:11111:325190862*stg/13/0000000d@S0000000d] but there is no valid WorkItemGroup for it.”,
2020-04-21T18:47:44.338785122Z “eventId”: {
2020-04-21T18:47:44.338788322Z “id”: 101231
2020-04-21T18:47:44.338791422Z },
2020-04-21T18:47:44.338794422Z “app”: {
2020-04-21T18:47:44.340212035Z “name”: “Squidex”,
2020-04-21T18:47:44.340223735Z “version”: “4.2.0.0”,
2020-04-21T18:47:44.340227836Z “sessionId”: “d64d099c-78e7-4661-b933-035dc0f04bbf”
2020-04-21T18:47:44.340231536Z },
2020-04-21T18:47:44.340234636Z “timestamp”: “2020-04-21T18:47:44Z”,
2020-04-21T18:47:44.340238036Z “category”: “Orleans.Runtime.Scheduler.OrleansTaskScheduler”
2020-04-21T18:47:44.340241336Z }

17

Then it is the request log repository, you could have a look to this collection,

here are the indexes that are created:

protected override string CollectionName()
        {
            return "RequestLog";
        }

        protected override Task SetupCollectionAsync(IMongoCollection<MongoRequest> collection, CancellationToken ct = default)
        {
            return collection.Indexes.CreateManyAsync(new[]
            {
                new CreateIndexModel<MongoRequest>(
                    Index
                        .Ascending(x => x.Key)
                        .Ascending(x => x.Timestamp)),
                new CreateIndexModel<MongoRequest>(
                    Index
                        .Ascending(x => x.Timestamp),
                    new CreateIndexOptions
                    {
                        ExpireAfter = TimeSpan.FromDays(options.StoreRetentionInDays)
                    })
            }, ct);
        }
18

Okay, do you have any idea what it could be caused this? Basicly what has happend is that the databases are backed up and restored into the new cluster, with new names though.

And what could a fix be?

19

No, sorry. You can send me a backup or so, but I have not seen that yet. But you can just delete the indexes.

20

Okay, i will try to get the backups and share it with you.

If we delete the indexes then i guess it will try to rebuild it?