Squidex has stopped sending events

Support
1

I have…

  • Read the following guideline: Troubleshooting and Support | Squidex. I understand that my support request might get deleted if I do not follow the guideline.
  • Used code blocks with ``` to format my code examples like JSON or logs properly.

I’m submitting a…

  • Regression (a behavior that stopped working in a new release)
  • Bug report
  • Performance issue
  • Documentation issue or request

Current behavior

I have a webhook that listens for content changes of a schema and sends updates to my backend. Whenever I make changes to the contents of that schema in Squidex, the changes are saved, but Squidex is not sending an event to my endpoint. Upon checking the rule logs, I noticed that the last time I received any event from Squidex was on March 8th.

Same issue happens with manual webhooks. When I trigger a manual rule, I see that it is added in the rule log with “pending” status, but it is never actually executed and stays in that state forever.

Expected behavior

I should receive an event when changes are made to schemas I have set up with a content change webhook as well as whan executing a manual webhook rule.

Minimal reproduction of the problem

  1. Create a schema.
  2. Add a content change webhook.
  3. Add content to the schema.
  4. Check the rule log; you will see that there is no activity recorded.

Environment

App Name: Tried with different apps, for example: sgp

  • Self hosted with docker
  • Self hosted with IIS
  • Self hosted with other version
  • Cloud version

Version: [VERSION]

Browser:

  • Chrome (desktop)
  • Chrome (Android)
  • Chrome (iOS)
  • Firefox
  • Safari (desktop)
  • Safari (iOS)
  • IE
  • Edge

Others:

2

I am already investigating that. I wanted to deploy a new worker version and this version could not connect to the database, so I rolled back to the previous version, which has the same problem unfortunately. I still don’t understand it, because nothing has been changed.

3

Alright, let’s hope this issue gets resolved soon.

4

Yes, it is super weird. When the application starts a bunch of startup tasks run. Many of them involve the database. For example creating indexes and so on.

Then there is this one task where a find operation on a primary key fails and times our after 30 seconds. The collection has like 20 entries.

5

Confirming that we’re experiencing the same issue as well.

6

Should be solved now.

7

The manual webhook is now working, but the Content Change webhook remains non-functional. Despite changing the content less than a minute ago, it is not reflected in the event history or the rule log.

Screenshot 2024-03-11 at 15.58.30
Screenshot 2024-03-11 at 15.58.30636×652 28.2 KB

8

I will have a look to that.

9

I had a closer look. At the moment the text indexer has to rebuild the full text indexes and it seems it takes all the available resources.

10

Do you have an estimated timeline for when the text indexer will complete the rebuilding process and when we can expect webhooks to work properly again?

11

No, sorry … Something is strange at the moment.

I am trying to get both running at the same time. Because the indexer could take a day or so. It has to handle millions of records.