Upgrade 5.4.0 to 6.3.0

jsajdak · December 28, 2021, 9:23pm

I have…

[ x] Read the following guideline: https://docs.squidex.io/01-getting-started/installation/troubleshooting-and-support. I understand that my support request might get deleted if I do not follow the guideline.

I’m submitting a…

[ ] Regression (a behavior that stopped working in a new release)
[ ] Bug report
[ ] Performance issue
[x ] Documentation issue or request

Current behavior

Performing an app restore on version 6.3.0 using a backup that was create from 6.3.0 results in errors.
First: If the system user exists (i.e. the admin user defined by IDENTITY__ADMINEMAIL), attempting to restore an app results in the following error stack:
2021-12-28 14:20:21"logLevel": “Warning”,
2021-12-28 14:20:21"message": “User validation failed: DuplicateUserName.”,
2021-12-28 14:20:21"eventId": {
2021-12-28 14:20:21"id": 13
2021-12-28 14:20:21},
2021-12-28 14:20:21"errors": “DuplicateUserName”,
2021-12-28 14:20:21"timestamp": “2021-12-28T20:20:21Z”,
2021-12-28 14:20:21"app": {
2021-12-28 14:20:21"name": “Squidex”,
2021-12-28 14:20:21"version": “6.3.0.0”,
2021-12-28 14:20:21"sessionId": “4999391a-6d97-4725-9db4-11111111111”
2021-12-28 14:20:21},
2021-12-28 14:20:21"category": “Microsoft.AspNetCore.Identity.UserManager”
2021-12-28 14:20:21}
2021-12-28 14:20:21{
2021-12-28 14:20:21"logLevel": “Warning”,
2021-12-28 14:20:21"message": “User validation failed: DuplicateUserName.”,
2021-12-28 14:20:21"eventId": {
2021-12-28 14:20:21"id": 13
2021-12-28 14:20:21},
2021-12-28 14:20:21"errors": “DuplicateUserName”,
2021-12-28 14:20:21"timestamp": “2021-12-28T20:20:21Z”,
2021-12-28 14:20:21"app": {
2021-12-28 14:20:21"name": “Squidex”,
2021-12-28 14:20:21"version": “6.3.0.0”,
2021-12-28 14:20:21"sessionId": “4999391a-6d97-4725-9db4-11111111111”
2021-12-28 14:20:21},
2021-12-28 14:20:21"category": “Microsoft.AspNetCore.Identity.UserManager”
2021-12-28 14:20:21}
2021-12-28 14:20:21{
2021-12-28 14:20:21{
2021-12-28 14:20:21"logLevel": “Error”,
2021-12-28 14:20:21"action": “IdentityOperation”,
2021-12-28 14:20:21"status": “Failed”,
2021-12-28 14:20:21"message": “DuplicateUserName: Username \u0027[the identity email]\u0027 is already taken.\n”,
2021-12-28 14:20:21"timestamp": “2021-12-28T20:20:21Z”,
2021-12-28 14:20:21"app": {
2021-12-28 14:20:21"name": “Squidex”,
2021-12-28 14:20:21"version": “6.3.0.0”,
2021-12-28 14:20:21"sessionId": “4999391a-6d97-4725-9db4-11111111111”
2021-12-28 14:20:21}
2021-12-28 14:20:21}
2021-12-28 14:20:21"timestamp": “2021-12-28T20:20:21Z”,
2021-12-28 14:20:21"app": {
2021-12-28 14:20:21"name": “Squidex”,
2021-12-28 14:20:21"version": “6.3.0.0”,
2021-12-28 14:20:21"sessionId": “4999391a-6d97-4725-9db4-11111111111”
2021-12-28 14:20:21},
2021-12-28 14:20:21"category": “Microsoft.AspNetCore.Identity.UserManager”
2021-12-28 14:20:21}
2021-12-28 14:20:21{
2021-12-28 14:20:21"logLevel": “Warning”,
2021-12-28 14:20:21"message": “User validation failed: DuplicateUserName.”,
2021-12-28 14:20:21"eventId": {
2021-12-28 14:20:21"id": 13
2021-12-28 14:20:21},
2021-12-28 14:20:21"errors": “DuplicateUserName”,
2021-12-28 14:20:21"timestamp": “2021-12-28T20:20:21Z”,
2021-12-28 14:20:21"app": {
2021-12-28 14:20:21"name": “Squidex”,
2021-12-28 14:20:21"version": “6.3.0.0”,
2021-12-28 14:20:21"sessionId": “4999391a-6d97-4725-9db4-11111111111”
2021-12-28 14:20:21},
2021-12-28 14:20:21"category": “Microsoft.AspNetCore.Identity.UserManager”
2021-12-28 14:20:21}
2021-12-28 14:20:21{
2021-12-28 14:20:21"logLevel": “Warning”,
2021-12-28 14:20:21"message": “User validation failed: DuplicateUserName.”,
2021-12-28 14:20:21"eventId": {
2021-12-28 14:20:21"id": 13
2021-12-28 14:20:21},
2021-12-28 14:20:21"errors": “DuplicateUserName”,
2021-12-28 12:46:09{
2021-12-28 12:46:09"logLevel": “Warning”,
2021-12-28 12:46:09"message": “CleanupDefunctSiloEntries operation is not supported by the current implementation of IMembershipTable. Disabling the timer now.”,
2021-12-28 12:46:09"eventId": {
2021-12-28 12:46:09"id": 100662
2021-12-28 12:46:09},
2021-12-28 12:46:09"timestamp": “2021-12-28T18:46:09Z”,
2021-12-28 12:46:09"app": {
2021-12-28 12:46:09"name": “Squidex”,
2021-12-28 12:46:09"version": “6.3.0.0”,
2021-12-28 12:46:09"sessionId": “4999391a-6d97-4725-9db4-11111111111”
2021-12-28 12:46:09},
2021-12-28 12:46:09"category": “Orleans.Runtime.MembershipService.MembershipTableCleanupAgent”
2021-12-28 12:46:09}
2021-12-28 12:46:09{
2021-12-28 12:46:09"logLevel": “Warning”,
2021-12-28 12:46:09"message": “CleanupDefunctSiloEntries operation is not supported by the current implementation of IMembershipTable. Disabling the timer now.”,
2021-12-28 12:46:09"eventId": {
2021-12-28 12:46:09"id": 100662
2021-12-28 12:46:09},
2021-12-28 12:46:09"timestamp": “2021-12-28T18:46:09Z”,
2021-12-28 12:46:09"app": {
2021-12-28 12:46:09"name": “Squidex”,
2021-12-28 12:46:09"version": “6.3.0.0”,
2021-12-28 12:46:09"sessionId": “4999391a-6d97-4725-9db4-11111111111”
2021-12-28 12:46:09},
2021-12-28 12:46:09"category": “Orleans.Runtime.MembershipService.MembershipTableCleanupAgent”
2021-12-28 12:46:09}

Second attempt: Let’s create a new container with a different admin email identity and try the restore again.

2021-12-28 14:48:11"sessionId": "7799ebe3-16db-4f2d-8d27-11111111111"
2021-12-28 14:48:11},
2021-12-28 14:48:11"exception": {
2021-12-28 14:48:11"type": "System.InvalidOperationException",
2021-12-28 14:48:11"message": "Error rate of 0 is above threshold 0.",
2021-12-28 14:48:11"stackTrace": " at Squidex.Infrastructure.Commands.Rebuilder.InsertManyAsync[T,TState](IAsyncEnumerable\u00601 source, Int32 batchSize, Double errorThreshold, 
	CancellationToken ct) in /src/src/Squidex.Infrastructure/Commands/Rebuilder.cs:line 177\n at Squidex.Infrastructure.Commands.Rebuilder.InsertManyAsync[T,TState](IEnumerable\u00601 source, 
	Int32 batchSize, Double errorThreshold, CancellationToken ct) in /src/src/Squidex.Infrastructure/Commands/Rebuilder.cs:line 87\n at 
	Squidex.Domain.Apps.Entities.Assets.BackupAssets.RestoreAsync(RestoreContext context, CancellationToken ct) in /src/src/Squidex.Domain.Apps.Entities/Assets/BackupAssets.cs:line 111\n at 
	Squidex.Domain.Apps.Entities.Backup.RestoreGrain.ProcessAsync() in /src/src/Squidex.Domain.Apps.Entities/Backup/RestoreGrain.cs:line 173"
2021-12-28 14:48:11}
2021-12-28 14:48:11}
2021-12-28 14:48:11{
2021-12-28 14:48:11"logLevel": "Error",
2021-12-28 14:48:11"action": "restore",
2021-12-28 14:48:11"status": "failed",
2021-12-28 14:48:11"operationId": "1ec63891-b425-4afb-b88d-35042dee4465",
2021-12-28 14:48:11"url": "https://[my s3 bucket URL]",
2021-12-28 14:48:11"timestamp": "2021-12-28T20:48:11Z",
2021-12-28 14:48:11"app": {
2021-12-28 14:48:11"name": "Squidex",
2021-12-28 14:48:11"version": "6.3.0.0",
2021-12-28 14:33:54{
2021-12-28 14:33:54"logLevel": "Warning",
2021-12-28 14:33:54"message": "CleanupDefunctSiloEntries operation is not supported by the current implementation of IMembershipTable. Disabling the timer now.",
2021-12-28 14:33:54"eventId": {
2021-12-28 14:33:54"id": 100662
2021-12-28 14:33:54},
2021-12-28 14:33:54"timestamp": "2021-12-28T20:33:54Z",
2021-12-28 14:33:54"app": {
2021-12-28 14:33:54"name": "Squidex",
2021-12-28 14:33:54"version": "6.3.0.0",
2021-12-28 14:33:54"sessionId": "7799ebe3-16db-4f2d-8d27-11111111111"
2021-12-28 14:33:54},
2021-12-28 14:33:54"category": "Orleans.Runtime.MembershipService.MembershipTableCleanupAgent"
2021-12-28 14:33:54}
2021-12-28 14:33:54{
2021-12-28 14:33:54"logLevel": "Warning",
2021-12-28 14:33:54"message": "CleanupDefunctSiloEntries operation is not supported by the current implementation of IMembershipTable. Disabling the timer now.",
2021-12-28 14:33:54"eventId": {
2021-12-28 14:33:54"id": 100662
2021-12-28 14:33:54},
2021-12-28 14:33:54"timestamp": "2021-12-28T20:33:54Z",
2021-12-28 14:33:54"app": {
2021-12-28 14:33:54"name": "Squidex",
2021-12-28 14:33:54"version": "6.3.0.0",
2021-12-28 14:33:54"sessionId": "7799ebe3-16db-4f2d-8d27-11111111111"
2021-12-28 14:33:54},
2021-12-28 14:33:54"category": "Orleans.Runtime.MembershipService.MembershipTableCleanupAgent"
2021-12-28 14:33:54}

Expected behavior

I can restore any app backup without having to worry about user identity conflicts.
Not sure even what the issue is with the second error.

Minimal reproduction of the problem

Environment

App Name:

[x ] Self hosted with docker
[ ] Self hosted with IIS
[ ] Self hosted with other version
[ ] Cloud version

Version: 5.4.0

Browser:

[x ] Chrome (desktop)
[ ] Chrome (Android)
[ ] Chrome (iOS)
[ ] Firefox
[ ] Safari (desktop)
[ ] Safari (iOS)
[ ] IE
[ ] Edge

Others:
Need to upgrade from 5.4.0 to 6.3.0 and migrate assets from Mongo to S3 and use ASSETS__FOLDERPERAPP = true
We cannot perform in-place upgrade of the environments because we need to be able to perform QA on the new environment before upgrading prod. So, we took the route of creating parallel Squidex docker deployments and Mongo instances.
Basic steps were:

Create parallel 5.4.0 docker deployment. Set the new docker instance to use S3 storage. (Can’t use folderperapp yet, so just set it to false)
Create backup (backup1) from original docker deployment.
Restore this backup (backup1) to new docker deployment (results in a working install with S3 storage)
Create another backup (backup2) just in case)
Update the new deployment to use 6.3.0 (leaving folderperapp as false still)
At this point the app is working, but the assets are in the root of the S3 bucket/squidex_assets/ folder. I want them in separate folders.
Create another backup (backup3) and download it.
Delete the app. (I noticed that the assets were not deleted)…so I manually deleted the /squidex_assets/ folder in S3.
Update the new deployment and set “ASSETS_FOLDERPERAPP” = true
Attempt to restore the app.
Fails with Duplicate user error.
Delete the app again.
Update the new deployment and create a new admin user.
Log in and delete all system users (except the new admin user).
Attempt to restore the app.
Am faced with the second error stack above.

Not sure how to proceed at this point.

Thank you.

Sebastian · December 28, 2021, 9:32pm

Hi,

only the last error with the threshold is the issue. Just go with dev-6578, it is already fixed. I am going to make a new release tomorrow or the day after.

jsajdak · December 28, 2021, 9:49pm

Thanks for the reply. I’ll stop banging my head and wait for the updated container image.

jsajdak · February 16, 2022, 8:10pm

Hello Sebastian,

Is this fix not merged in 6.5.0 ? I’m running into a similar issue after switching from dev-6578 to 6.5.0 stable.

Thanks,
Jason

Sebastian · February 16, 2022, 8:23pm

It should be merged in. What do you mean with “similar issue”?

jsajdak · February 16, 2022, 9:50pm

This error “message”: “User validation failed: DuplicateUserName.”.

But I should clarify, I was attempting to restore a backup of a single app into Squidex with an app that already existed with the same name. I’ve since deleted the existing app and I was able to restore without any error.

Given this nuance, I think that the original issue fix is, in fact, merged into 6.5.0.
Sorry for the confusion.

Sebastian · February 17, 2022, 7:11am

This error is normal, nothing to worry about.