[SOLVED] 500 error when permanently deleting content with custom IDs

slalFe · February 14, 2023, 2:33pm

I have…

[x] Checked the logs and have uploaded a log file and provided a link because I found something suspicious there. Please do not post the log file in the topic because very often something important is missing.

I’m submitting a…

[ ] Regression (a behavior that stopped working in a new release)
[x] Bug report
[ ] Performance issue
[ ] Documentation issue or request

Current behavior

After creating content with custom IDs using the Squidex.ClientLibrary (v13.0) we are unable to delete the records permanently using either the Squidex.ClientLibrary or the API directly. In case it’s relevant these custom IDs contain special characters, specifically _, $, [, and ].

We can weirdly workaround this by soft-deleting (permanent=false) the content and THEN permanently deleting it (permanent=true).

Content created using the UI and similar custom IDs does not have problems being directly permanently deleted. This is quite confusing as the documents in MongoDB looks almost exactly the same.

Expected behavior

Any content created using an ID accepted by the custom ID field (either by UI or API) should be able to be deleted.

Minimal reproduction of the problem

Have a schema
Create content for that schema via Squidex.ClientLibrary specifying a custom ID containing special characters $, [, and ]
Delete permanently using the API or Squidex.ClientLibrary
Content not deleted and you get a 500 error

Environment

[x] Self hosted with docker
[ ] Self hosted with IIS
[ ] Self hosted with other version
[ ] Cloud version

Version: 7.2.0

Browser:

[x] Chrome (desktop)
[ ] Chrome (Android)
[ ] Chrome (iOS)
[ ] Firefox
[ ] Safari (desktop)
[ ] Safari (iOS)
[ ] IE
[ ] Edge

Others:
This could well be indicative of a more fundamental problem with our site, but we are getting this repeatedly happen to us and haven’t used custom IDs before so feel there could be a problem with them. Deletions of other content that use auto-generated IDs is not giving us any issue at all.

Sebastian · February 14, 2023, 2:45pm

500 => Check the logs

slalFe · February 14, 2023, 3:00pm

Ah yes sorry, was blinded by our application logs so forgot to add the Squidex one!

category: Squidex.Web.ApiExceptionFilterAttribute
exception.type: Squidex.Infrastructure.States.InconsistentStateException
web.requestMethod    DELETE
web.requestPath: /api/content/[APPNAME]/[SCHEMANAME]/lnk_$$[redacted]$$[redacted]%5B%5D[redacted]
web.routeValues.action: DeleteContent
web.routeValues.area: api
web.routeValues.controller: Contents

Requested version -1, but found 0.

at Squidex.Infrastructure.States.Persistence`1.WriteEventsAsync(IReadOnlyList`1 events, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\States\Persistence.cs:line 234
   at Squidex.Infrastructure.Commands.DomainObject`1.DeleteCoreAsync[TCommand](TCommand command, Func`3 handler, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\DomainObject.cs:line 207
   at Squidex.Infrastructure.Commands.DomainObject`1.DeletePermanentAsync[TCommand](TCommand command, Func`3 handler, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\DomainObject.Execute.cs:line 156
   at Squidex.Infrastructure.Commands.CachingDomainObjectMiddleware`3.ExecuteCommandAsync(T executable, TCommand command, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\CachingDomainObjectMiddleware.cs:line 38
   at Squidex.Infrastructure.Commands.AggregateCommandMiddleware`2.ExecuteCommandAsync(CommandContext context, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\AggregateCommandMiddleware.cs:line 38
   at Squidex.Infrastructure.Commands.AggregateCommandMiddleware`2.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\AggregateCommandMiddleware.cs:line 26
   at Squidex.Domain.Apps.Entities.Contents.DomainObject.ContentsBulkUpdateCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Contents\DomainObject\ContentsBulkUpdateCommandMiddleware.cs:line 136
   at Squidex.Infrastructure.Commands.AggregateCommandMiddleware`2.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\AggregateCommandMiddleware.cs:line 26
   at Squidex.Domain.Apps.Entities.Comments.DomainObject.CommentsCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Comments\DomainObject\CommentsCommandMiddleware.cs:line 38
   at Squidex.Infrastructure.Commands.AggregateCommandMiddleware`2.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\AggregateCommandMiddleware.cs:line 26
   at Squidex.Domain.Apps.Entities.Assets.DomainObject.AssetCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Assets\DomainObject\AssetCommandMiddleware.cs:line 66
   at Squidex.Domain.Apps.Entities.Assets.DomainObject.AssetsBulkUpdateCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Assets\DomainObject\AssetsBulkUpdateCommandMiddleware.cs:line 130
   at Squidex.Infrastructure.Commands.AggregateCommandMiddleware`2.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\AggregateCommandMiddleware.cs:line 26
   at Squidex.Domain.Apps.Entities.Apps.DomainObject.AppCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Apps\DomainObject\AppCommandMiddleware.cs:line 40
   at Squidex.Domain.Apps.Entities.Schemas.Indexes.SchemasIndex.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Schemas\Indexes\SchemasIndex.cs:line 152
   at Squidex.Domain.Apps.Entities.Apps.Indexes.AppsIndex.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Apps\Indexes\AppsIndex.cs:line 189
   at Squidex.Domain.Apps.Entities.Invitation.InviteUserCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Invitation\InviteUserCommandMiddleware.cs:line 68
   at Squidex.Domain.Apps.Entities.Apps.Plans.RestrictAppsCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Apps\Plans\RestrictAppsCommandMiddleware.cs:line 63
   at Squidex.Domain.Apps.Entities.Apps.AlwaysCreateClientCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Apps\AlwaysCreateClientCommandMiddleware.cs:line 38
   at Squidex.Domain.Apps.Entities.Apps.Templates.TemplateCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Domain.Apps.Entities\Apps\Templates\TemplateCommandMiddleware.cs:line 55
   at Squidex.Infrastructure.Commands.CustomCommandMiddlewareRunner.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\CustomCommandMiddlewareRunner.cs:line 27
   at Squidex.Web.CommandMiddlewares.ETagCommandMiddleware.HandleAsync(CommandContext context, NextDelegate next, CancellationToken ct) in C:\src\src\Squidex.Web\CommandMiddlewares\ETagCommandMiddleware.cs:line 61
   at Squidex.Infrastructure.Commands.InMemoryCommandBus.PublishAsync(ICommand command, CancellationToken ct) in C:\src\src\Squidex.Infrastructure\Commands\InMemoryCommandBus.cs:line 43
   at Squidex.Areas.Api.Controllers.Contents.ContentsController.DeleteContent(String app, String schema, DomainId id, DeleteContentDto request) in C:\src\src\Squidex\Areas\Api\Controllers\Contents\ContentsController.cs:line 573
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextExceptionFilterAsync>g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)

slalFe · February 14, 2023, 3:31pm

Just to confirm I could recreate on v7.4.0 of Squidex.

Issue seems to be after you have already permanently deleted them once, subsequently recreate data and then try to permanently delete again. Are the custom IDs cached somewhere or something?

Sebastian · February 14, 2023, 3:35pm

Yes, but only for a short duration.

Sebastian · February 14, 2023, 3:37pm

Would you be able to create a pull request with a test case?

github.com

Squidex/squidex/blob/master/tools/TestSuite/TestSuite.ApiTests/ContentUpdateTests.cs

// ==========================================================================
//  Squidex Headless CMS
// ==========================================================================
//  Copyright (c) Squidex UG (haftungsbeschraenkt)
//  All rights reserved. Licensed under the MIT license.
// ==========================================================================

using Newtonsoft.Json.Linq;
using Squidex.ClientLibrary;
using Squidex.ClientLibrary.Management;
using TestSuite.Model;

#pragma warning disable CS0618 // Type or member is obsolete
#pragma warning disable SA1300 // Element should begin with upper-case letter
#pragma warning disable SA1507 // Code should not contain multiple blank lines in a row

namespace TestSuite.ApiTests;

[UsesVerify]
public class ContentUpdateTests : IClassFixture<ContentFixture>

This file has been truncated. show original

slalFe · February 14, 2023, 3:38pm

I’ll give it a bash!

Edit: Have opened a PR for it but have not run those tests before so haven’t been able to confirm whether or not it actually recreates the issue (if it is one with the code and not just our site!).

Edit again: Oh of course pushing up to GitHub has run it and I am getting a 500! So I think that’s good news?

slalFe · February 17, 2023, 4:31pm

In case anyone else hits this issue the workaround does work well for us.

We have a method that soft deletes everything before then permanently deleting them:

Sebastian · February 17, 2023, 9:36pm

I can still not reproduce it.

Sebastian · February 17, 2023, 10:37pm

Sorry, I haven not seen your PR. I am trying to reproduce it locally, but so far I can’t

Sebastian · February 17, 2023, 10:59pm

I have found another bug. When you delete a content permanently that was recreated before it does not work. Perhaps this was your problem.

slalFe · February 20, 2023, 9:55am

Ah yes that sounds like it! So did my test in the PR recreate this as it tries to permanently delete it twice (second one in the teardown) or was it failing for another reason irrelevant to the test?

Edit: Am building now with hopes to test your change later today or tomorrow.

Sebastian · February 20, 2023, 11:07am

Yes, exactly. I have recreated the test so that I can easily test all options how to delete something:

github.com

Squidex/squidex/blob/master/tools/TestSuite/TestSuite.ApiTests/ContentUpdateTests.cs#L601


    // STEP 3: Recreate the item with the same id.
    var content_2 = await _.Contents.UpsertAsync(content_1.Id, new TestEntityData
    {
        Number = 2
    }, ContentUpsertOptions.AsPublish);

    Assert.Equal(Status.Published, content_2.Status);
}

[Theory]
[InlineData(Strategies.Deletion.SingleSoft)]
[InlineData(Strategies.Deletion.SinglePermanent)]
[InlineData(Strategies.Deletion.BulkSoft)]
[InlineData(Strategies.Deletion.BulkPermanent)]
public async Task Should_delete_recreated_content(Strategies.Deletion strategy)
{
    var id = $"custom-{Guid.NewGuid()}";

    // STEP 1: Create a new item with a custom id.
    var options = new ContentCreateOptions { Id = id, Publish = true };

system · February 22, 2023, 8:16pm

This topic was automatically closed after 2 days. New replies are no longer allowed.