Total on contents view doesn't match the number of paged results

slalFe · December 15, 2022, 4:22pm

I have…

[x] Checked the logs and have uploaded a log file and provided a link because I found something suspicious there. Please do not post the log file in the topic because very often something important is missing.

I’m submitting a…

[ ] Regression (a behavior that stopped working in a new release)
[x] Bug report
[ ] Performance issue
[ ] Documentation issue or request

Current behavior

‘Total’ on Contents view (and querying with the API and Squidex.ClientLibrary) is showing vastly more results than there actually are.

Clicking on ‘Next page’ shows an empty page with only back button and page size showing:

Deleting permanently via the API is not failing, but it isn’t deleting anything. Deleting via the UI is working.

Expected behavior

‘Total’ matches the number of documents.

Minimal reproduction of the problem

No idea sorry!

Environment

[x] Self hosted with docker
[ ] Self hosted with IIS
[ ] Self hosted with other version
[ ] Cloud version

Version: 7.2.0

Browser:

[x] Chrome (desktop)
[ ] Chrome (Android)
[ ] Chrome (iOS)
[ ] Firefox
[ ] Safari (desktop)
[ ] Safari (iOS)
[ ] IE
[ ] Edge

Others:
This has happened a few times for us in shared development environments, so may not be a bug with Squidex (was happening when on v7.1.0 too so not a regression) but perhaps is caused by us performing concurrent actions like deleting and inserting at the same time, deploying halfway through a deletion etc. Just wondered if you had any ideas on how we could get into this state? Also if you had any thoughts on how we could fix it that doesn’t require deleting and recreating the schema?

Sebastian · December 14, 2022, 11:46am

I am actually do not believe that, because I have api tests for this use, case, which are using the API obviously.

Squidex has a cache for the total count, because it is actually a very slow operation. Is it only temporary the wrong value or constantly? Have you tried to delete this cache?

It is a mongodb collection ending with _Count.

slalFe · December 14, 2022, 1:11pm

Thank you very much! Simply deleting the record for the problematic schema in “[app name]Content.States_Contents_All3_Count” immediately fixed the issue.

Yes sorry the deletion issue was just a side effect on our code of the total being incorrect, and it wasn’t us directly hitting the API; just using the Squidex.ClientLibrary. Our code does the following:

Make initial lightweight request to get the ‘Total’ of all documents in a schema (this was returning the wrong cached value)
We then make a multiple request to get 500 documents at a time (just their IDs) until we hit that total (however this was failing as we would never hit that total)
Uses Bulk API to delete all the documents by their IDs

It seems to be constantly, at least it had been the wrong value for hours. Is there a built in cache expiration or action that is supposed to cause it to expire? Is there a way we can force cache expiration via the API or Squidex.ClientLibrary?

Edit: All I can find are options to not return a Total at all, nothing to get it to recalculate the total: https://github.com/Squidex/squidex-samples/blob/master/csharp/Squidex.ClientLibrary/Squidex.ClientLibrary/QueryContext.cs#L150

Sebastian · December 14, 2022, 1:36pm

No, but it is only 10 sec:

github.com

Squidex/squidex/blob/master/backend/src/Squidex.Domain.Apps.Entities.MongoDb/MongoCountCollection.cs

// ==========================================================================
//  Squidex Headless CMS
// ==========================================================================
//  Copyright (c) Squidex UG (haftungsbeschraenkt)
//  All rights reserved. Licensed under the MIT license.
// ==========================================================================

using MongoDB.Driver;
using NodaTime;
using Squidex.Infrastructure.MongoDb;
using Squidex.Infrastructure.Tasks;

namespace Squidex.Domain.Apps.Entities.MongoDb;

internal sealed class MongoCountCollection : MongoRepositoryBase<MongoCountEntity>
{
    private readonly string name;

    public MongoCountCollection(IMongoDatabase database, string name)
        : base(database)

This file has been truncated. show original

I would just query until nothing is returned anymore.

slalFe · December 14, 2022, 2:00pm

Yeah looking at the code again I have definitely over complicated it, and there is even already a bit of code basically doing exactly what you are saying that I must of added in the past, making that initial call to get the total completely redundant!

if (results.Count == 0)
{
  break;
}

Ah right, so possibly for some reason actualCount != cachedCount or isOutdated are false. Will let you know if this recurs for us and try to give more insight on how we got into this state. Interesting to know this will only happen when there are more than 5000 documents!

Sebastian · December 14, 2022, 4:32pm

I think I have found a bug:

See this line:

github.com

Squidex/squidex/blob/master/backend/src/Squidex.Domain.Apps.Entities.MongoDb/MongoCountCollection.cs#L45


    // This is our hardcoded limit at the moment. Usually schemas are much smaller anyway.
    if (cachedTotal < 5_000)
    {
        // We always want to have up to date collection sizes for smaller schemas.
        return await RefreshTotalAsync(key, cachedTotal, provider, ct);
    }

    if (isOutdated)
    {
        // If we have a loot of items, the query might be slow and therefore we execute it in the background.
        RefreshTotalAsync(key, cachedTotal, provider, ct).Forget();
    }

    return cachedTotal;
}

private async Task<long> RefreshTotalAsync(string key, long cachedCount, Func<CancellationToken, Task<long>> provider,
    CancellationToken ct)
{
    var actualCount = await provider(ct);

I am not sure how familiar you are with .NET, but the ct variable is CancellationToken. It is used to cancel an operation, e.g. database call. Such a cancellation token can be populated from multiple source and in this case it is…

The cancellation token from the request.
The hard timeout for queries.

But his line is supposed to run in the background so it does not make sense to cancel it at all.

slalFe · December 14, 2022, 4:42pm

That makes far more sense than what I was saying! So it would just continue to run in the background until it hits an actual error? Would it not still need to be able to timeout but with a lot longer until it does so?

Also while you’re in that area there’s a minor typo…

Sebastian · December 15, 2022, 4:22pm

I have pushed a fix…docker tag dev-7390

system · December 17, 2022, 4:22pm

This topic was automatically closed after 2 days. New replies are no longer allowed.