MongoDB rebuild content fails with collection key already exists

I have…

  • Read the following guideline: Troubleshooting and Support | Squidex. I understand that my support request might get deleted if I do not follow the guideline.
  • Used code blocks with ``` to format my code examples like JSON or logs properly.

I’m submitting a…

  • Regression (a behavior that stopped working in a new release)
  • Bug report
  • Performance issue
  • Documentation issue or request

Current behavior

I’ve tried migrating our setup to use optimized for self-hosting as noted here after seeing a lot of index-related alerts from Mongo, but when setting REBUILD__CONTENTS: true the worker goes into crashloop with apparently a Mongo collection duplicate key error:

Unhandled exception. MongoDB.Driver.MongoBulkWriteException`1[Squidex.Domain.Apps.Entities.Contents.MongoContentEntity]: A bulk write operation resulted in one or more errors. WriteErrors: [ { Category : "DuplicateKey", Code : 11000, Message : "E11000 duplicate key error collection: SquidexContent-staging_8e562f82-fdea-4f4e-8b1e-cdcddefd4dc3.States_Contents_All3_2e5393cb-acc0-4e50-b148-76f18ae59caf index: _id_ dup key: { _id: "8e562f82-fdea-4f4e-8b1e-cdcddefd4dc3--dde46f2b-a550-4427-ad3f-abe5706aff75" }" } ].

Expected behavior

The data migration is done without issues.

Minimal reproduction of the problem

Create some content when CONTENTS__OPTIMIZEFORSELFHOSTING: false.
Switch to CONTENTS__OPTIMIZEFORSELFHOSTING: true.

Run with REBUILD__CONTENTS: true.

Environment

Our setup is:

  • k8s stateless squidex 7 with multiple service pods and one worker pod (although during deployments there may be 2 worker pods, this technically can be an issue I guess.)
  • mongo DB atlas

App Name: gte

  • Self hosted with docker
  • Self hosted with IIS
  • Self hosted with other version
  • Cloud version

Version: 7.18.0

Browser:

  • Chrome (desktop)
  • Chrome (Android)
  • Chrome (iOS)
  • Firefox
  • Safari (desktop)
  • Safari (iOS)
  • IE
  • Edge

Others:

I’m unfortunately no expert in squidex yet, just starting the journey, so appreciate help and suggestions.

can you send me a dump of your database? Troubleshooting and Support | Squidex