[INACTIVE] Asynch agent Runtime.Messaging.IncomingMessageAgent/System encountered unexpected exception The Stage will be restarted

I’m submitting a…

[ ] Regression (a behavior that stopped working in a new release)
[ x] Bug report
[ ] Performance issue
[ ] Documentation issue or request

Current behavior

Expected behavior

Minimal reproduction of the problem

  1. Create and run containers with docker-compose-noproxy.yml (docker-compose -f docker-compose-noproxy.yml up -d)
  2. Create an app
  3. Create a schema
  4. Create a content page of that schema
  5. Shutdown your PC
  6. Start your PC
  7. Go to Squidex (localhost) => not working

If I check my squidex container it is stopped and can’t be started / restarted. And the error logs tells me the following:

Environment

  • [ ] Self hosted with docker
  • [ ] Self hosted with IIS
  • [ ] Self hosted with other version
  • [ ] Cloud version
  • [X] Local dev environment Windows 10

Browser:

  • [ x] Chrome (desktop)
  • [ ] Chrome (Android)
  • [ ] Chrome (iOS)
  • [ ] Firefox
  • [ ] Safari (desktop)
  • [ ] Safari (iOS)
  • [ ] IE
  • [ ] Edge

Others:

2019-03-27T19:08:07.805985500Z 
{
  "logLevel": "Error",
  "message": "Asynch agent Runtime.Messaging.IncomingMessageAgent/System encountered unexpected exception The Stage will be restarted.",
  "eventId": {
    "id": 100026
  },
  "exception": {
    "type": "System.ArgumentNullException",
    "message": "Value cannot be null.",
    "stackTrace": "   at System.Threading.Monitor.Exit(Object obj)\n   at System.Threading.SemaphoreSlim.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)\n   at System.Collections.Concurrent.BlockingCollection`1.TryTakeWithNoTimeValidation(T& item, Int32 millisecondsTimeout, CancellationToken cancellationToken, CancellationTokenSource combinedTokenSource)\n   at System.Collections.Concurrent.BlockingCollection`1.TryTake(T& item, Int32 millisecondsTimeout, CancellationToken cancellationToken)\n   at System.Collections.Concurrent.BlockingCollection`1.Take(CancellationToken cancellationToken)\n   at Orleans.Runtime.Messaging.InboundMessageQueue.WaitMessage(Categories type, CancellationToken cancellationToken)\n   at Orleans.Runtime.Messaging.MessageCenter.WaitMessage(Categories type, CancellationToken ct)\n   at Orleans.Runtime.Messaging.IncomingMessageAgent.Run()\n   at Orleans.Threading.FiltersApplicant`1.Apply(T action)"
  },
  "app": {
    "name": "Squidex",
    "version": "1.0.0.0",
    "sessionId": "ab41e701-74d8-4fad-96fd-303af57e5a36"
  },
  "timestamp": "2019-03-27T19:08:07Z",
  "category": "Runtime.Messaging.IncomingMessageAgent/System"
}
2019-03-27T19:08:07.806725600Z 
Application is shutting down...

I pruned all my docker images and containers before this so I’m using the latest Squidex image available on hub.docker.com.

More logs:

2019-03-27T19:08:07.717952200Z 
Hosting environment: Production
Content root path: /app
Now listening on: http://[::]:80
Application started. Press Ctrl+C to shut down.
{
  "logLevel": "Error",
  "message": "RunClientMessagePump has thrown exception",
  "eventId": {
    "id": 100326
  },
  "exception": {
    "type": "System.OperationCanceledException",
    "message": "The operation was canceled.",
    "stackTrace": "   at System.Collections.Concurrent.BlockingCollection`1.TryTakeWithNoTimeValidation(T& item, Int32 millisecondsTimeout, CancellationToken cancellationToken, CancellationTokenSource combinedTokenSource)\n   at System.Collections.Concurrent.BlockingCollection`1.TryTake(T& item, Int32 millisecondsTimeout, CancellationToken cancellationToken)\n   at System.Collections.Concurrent.BlockingCollection`1.Take(CancellationToken cancellationToken)\n   at Orleans.Runtime.HostedClient.RunClientMessagePump()"
  },
  "app": {
    "name": "Squidex",
    "version": "1.0.0.0",
    "sessionId": "ab41e701-74d8-4fad-96fd-303af57e5a36"
  },
  "timestamp": "2019-03-27T19:08:07Z",
  "category": "Orleans.Runtime.HostedClient"
}
2019-03-27T19:08:07.789692300Z 
{
  "logLevel": "Error",
  "message": "Asynch agent Runtime.Messaging.IncomingMessageAgent/Application encountered unexpected exception The Stage will be restarted.",
  "eventId": {
    "id": 100026
  },
  "exception": {
    "type": "System.ArgumentNullException",
    "message": "Value cannot be null.",
    "stackTrace": "   at System.Threading.Monitor.Exit(Object obj)\n   at System.Threading.SemaphoreSlim.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)\n   at System.Collections.Concurrent.BlockingCollection`1.TryTakeWithNoTimeValidation(T& item, Int32 millisecondsTimeout, CancellationToken cancellationToken, CancellationTokenSource combinedTokenSource)\n   at System.Collections.Concurrent.BlockingCollection`1.TryTake(T& item, Int32 millisecondsTimeout, CancellationToken cancellationToken)\n   at System.Collections.Concurrent.BlockingCollection`1.Take(CancellationToken cancellationToken)\n   at Orleans.Runtime.Messaging.InboundMessageQueue.WaitMessage(Categories type, CancellationToken cancellationToken)\n   at Orleans.Runtime.Messaging.MessageCenter.WaitMessage(Categories type, CancellationToken ct)\n   at Orleans.Runtime.Messaging.IncomingMessageAgent.Run()\n   at Orleans.Threading.FiltersApplicant`1.Apply(T action)"
  },
  "app": {
    "name": "Squidex",
    "version": "1.0.0.0",
    "sessionId": "ab41e701-74d8-4fad-96fd-303af57e5a36"
  },
  "timestamp": "2019-03-27T19:08:07Z",
  "category": "Runtime.Messaging.IncomingMessageAgent/Application"
}

So it is deadlocked? It would be great if you could upload the full logs somewhere.

I have the same problem with MongoDB by the way, I have to restart my docker instance after every start, don’t know why.

I have no idea what is happening but I would be more than happy to share the logs. Exactly which logs do you need?

Exactly what are you referring to when you say “docker instance”? The container? Or the docker daemon?

The daemon. … ! !

The full logs of the squidex process would be great.