[SOLVED] Failed to update consumer TextIndexer6

Bugs Solved Bugs
1

I have…

  • Checked the logs and have uploaded a log file and provided a link because I found something suspicious there. Please do not post the log file in the topic because very often something important is missing.

I’m submitting a…

  • Regression (a behavior that stopped working in a new release)
  • Bug report
  • Performance issue
  • Documentation issue or request

Current behavior

After upgrade from 7.9 to 7.12 a got this error.

{
  
message: 'The output byte buffer is too small to contain the encoded data, encoding codepage ''65001'' and fallback ''System.Text.EncoderReplacementFallback''. (Parameter ''bytes'')',
  
stackTrace: '   at System.Text.Encoding.ThrowBytesOverflow()
   at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder, Boolean throwForDestinationOverflow)
   at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar, Boolean throwForDestinationOverflow)
   at Squidex.Domain.Apps.Entities.MongoDb.BsonUniqueContentIdSerializer.Serialize(BsonSerializationContext context, BsonSerializationArgs args, UniqueContentId value) in /src/src/Squidex.Domain.Apps.Entities.MongoDb/BsonUniqueContentIdSerializer.cs:line 123
   at MongoDB.Driver.SingleItemAsArrayOperatorFilterDefinition`2.Render(IBsonSerializer`1 documentSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.MongoCollectionImpl`1.CreateFindOperation[TProjection](FilterDefinition`1 filter, FindOptions`2 options)
   at MongoDB.Driver.MongoCollectionImpl`1.FindAsync[TProjection](IClientSessionHandle session, FilterDefinition`1 filter, FindOptions`2 options, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)
   at MongoDB.Driver.IAsyncCursorSourceExtensions.ToListAsync[TDocument](IAsyncCursorSource`1 source, CancellationToken cancellationToken)
   at Squidex.Domain.Apps.Entities.MongoDb.Text.MongoTextIndexerState.GetAsync(HashSet`1 ids, CancellationToken ct) in /src/src/Squidex.Domain.Apps.Entities.MongoDb/Text/MongoTextIndexerState.cs:line 57
   at Squidex.Domain.Apps.Entities.Contents.Text.TextIndexingProcess.On(IEnumerable`1 events) in /src/src/Squidex.Domain.Apps.Entities/Contents/Text/TextIndexingProcess.cs:line 352
   at Squidex.Infrastructure.EventSourcing.Consume.EventConsumerProcessor.DispatchAsync(IReadOnlyList`1 events) in /src/src/Squidex.Infrastructure/EventSourcing/Consume/EventConsumerProcessor.cs:line 175
   at Squidex.Infrastructure.EventSourcing.Consume.EventConsumerProcessor.<>c__DisplayClass13_0.<<OnNextAsync>b__0>d.MoveNext() in /src/src/Squidex.Infrastructure/EventSourcing/Consume/EventConsumerProcessor.cs:line 84
--- End of stack trace from previous location ---
   at Squidex.Infrastructure.EventSourcing.Consume.EventConsumerProcessor.UpdateAsync(Func`1 action, String position, String caller) in /src/src/Squidex.Infrastructure/EventSourcing/Consume/EventConsumerProcessor.cs:line 198',
  
type: 'System.ArgumentException'
}

I try manually reset consmer in ../app/administration/event-consumers but it seems it not works (there is bug icon next to it)

Environment

  • Self hosted with docker
  • Self hosted with IIS
  • Self hosted with other version
  • Cloud version

Version: 7.12.0

2

Do you know if you have some special Ids that are very long or something like that? I cannot find the root cause.

3

I do not think. We use only c# Guid as IDs.

In mongo log there are still “slow query” could be somehow related?

4

No, it is related to a custom MongoDB serializer that I wrote to make the text index smaller. But I am not sure how exactly it happens. I have tests for that and could not find the reason yet.

5

If it helps there are also this in log. Sorry for not including it in original post.

image_name: squidex/squidex:7.12.0
logLevel: Fatal
message: Failed to update consumer TextIndexer6 at position 1642780684-1-1-2 from OnNextAsync.
position: 1642780684-1-1-2
tag: ebd6e16377cb
6

Thats just the same, more or less.

7

The following part refers to an event: “Timestamp(1642780684, 1)”.

If you could just query for the next event and sent this to me?

Collection: Events2
Filter: { "Timestamp" : { $gt : Timestamp(1642780684, 1) } }
Sort: { "Timestamp":  1 }
8

Problem is solved. The root cause are very large IDs, that have been created by accident in this case.

1 Like