Calling webhook with an error returned

Support
1

I have…

  • [X] Checked the logs and have provided the logs if I found something suspicious there

I’m submitting a…

  • [ ] Regression (a behavior that stopped working in a new release)
  • [X] Bug report
  • [ ] Performance issue
  • [ ] Documentation issue or request

Current behavior

We have a google cloud function deployed as HTTP endpoint. The squidex webhook should send the data to the HTTP endpoint and success.
Currently I can see the data has been post to Google Function and processed. However the webhook function still returned an error and will resend it again.

  1. this is the log from the webhook,

System.Threading.Tasks.TaskCanceledException: The operation was canceled.
—> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
—> System.Net.Sockets.SocketException (125): Operation canceled
— End of inner exception stack trace —
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Security.SslStream.g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask`1 task, Int32 min, Int32 initial)

  1. this is the javascript code we deployed as cloud function.
exports.publishToQueue = async (req, res) => {
  const pubsub = new PubSub();
  const topic = pubsub.topic(process.env.TOPIC);

  const callback = (err, messageId) => {
    if (err) {
      res.status(400).send();
      console.error(err);
    } else {
      console.info(`${logPrefix} ${messageId} created`);
      res.status(200).send('Result: OK');
    }
  };
  topic.publishJSON(req.body, callback);
};

It seems this not work with nodejs for callback or async.
if I moved the res.send() after the topic.publishJSON(), then it worked. but I need to know if the operation worked or not to let Squidex know and resend it if it is not working

Expected behavior

Minimal reproduction of the problem

Create a node js async function and deploy as a service for squidex webhook to call.
I wonder if this can be the same issue for AWS lambda, we are using GCP cloud function.

Environment

  • [ ] Self hosted with docker
  • [ ] Cloud version

Others:

2

Hi, what is the result in the rules log? Looks like a timeout for me.

3

This is the full log from the webhook

Request:
POST: https://asia-northeast1-sca-abnormally-pro-stingray.cloudfunctions.net/mercury-squidex-webhook HTTP/1.1
X-Signature: G0r+/e4uiIx6hsLg0LdW6EcR5KvDoWmqRT/XSOio9Is=
X-Application: Squidex Webhook
User-Agent: Squidex; Webhook
Content-Type: application/json; charset=utf-8
Content-Length: 1026

{“type”:“PodcastUpdated”,“payload”:{"$type":“EnrichedContentEvent”,“type”:“Updated”,“id”:“50db2bad-36ab-4b5c-a9c1-19815a89d6fc”,“created”:“2020-01-21T00:19:26Z”,“lastModified”:“2020-02-04T05:23:51Z”,“createdBy”:“subject:5e1ffa9e158a9a0001a9610a”,“lastModifiedBy”:“subject:5e1ffa9e158a9a0001a9610a”,“data”:{“name”:{“iv”:“concussion”},“slug”:{“iv”:“concussion”},“playlistId”:{“iv”:“b966b0b1-3580-4b0e-9bf6-ab200031ba16”},“teaser”:{“iv”:“112233”},“heroImage”:{“iv”:[“bf2105ba-40bc-457d-85ac-cd19feb8dc19”]},“heroImageMobile”:{“iv”:[“bf2105ba-40bc-457d-85ac-cd19feb8dc19”]},“categories”:{“iv”:[“240b98ab-9c1a-4088-a32e-886fb5772c9e”]},“creators”:{“iv”:[“4cd80e1e-3e04-47d7-8345-1f858f9a5f08”]},“relatedPodcasts”:{“iv”:[]}},“status”:“Published”,“partition”:1716016629,“schemaId”:“5f355cb0-1ce0-4fcd-91ac-3e0dc44251b6,podcast”,“actor”:“subject:5e1ffa9e158a9a0001a9610a”,“appId”:“0ddffcbb-c151-43b3-afba-6bbd80d34015,mercury”,“timestamp”:“2020-02-04T05:23:51Z”,“name”:“PodcastUpdated”,“version”:9},“timestamp”:“2020-02-04T05:23:51Z”}

Response:

System.Threading.Tasks.TaskCanceledException: The operation was canceled.
—> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
—> System.Net.Sockets.SocketException (125): Operation canceled
— End of inner exception stack trace —
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Security.SslStream.g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial) at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer)
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
— End of inner exception stack trace —
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Squidex.Extensions.Actions.HttpHelper.OneWayRequestAsync(HttpClient client, HttpRequestMessage request, String requestBody, CancellationToken ct) in /src/extensions/Squidex.Extensions/Actions/HttpHelper.cs:line 24

Action timed out.

Elapsed 00:00:02.0090000.

4

I think the issue is that I have response.send inside the callback function.
But this should be supported.

And it is only 2 seconds then it is time out?
what is the default time out time and can we set it up longer?

5

You can just use fire and forget in your webhook. Return immediately and do not wait for the result.

6

Yes, we can do it that way. But then if there is issue, from squidex perspective, it is successful then it won’t be able to resubmit if there is really error. so I wonder if we can have some code in node js callback to make it work that way?

const callback = (err, messageId) => {
  if (err) {
    res.status(400).send();
  } else { 
    res.status(200).send(‘Result: OK’);
  }
};

myTestService.submitToQueue(req.body, callback);
7

Sure, you can also push it to a queue like azure or redis or so.

Btw: Please format your code, just a ``` at the beginning and end of your code block.

8

Thanks and I have formatted the code now.
You can see the code is in async call, so I wonder if the current webhook support it or not?
If it is, then it should work or do I have to increase the timeout in squdiex webhook and how?
Regards,

9

For the webhook it does not really matter, as long as it is a normal HTTP endpoint you can do whatever you want.

There is this setting: https://github.com/Squidex/squidex/blob/master/backend/src/Squidex/appsettings.json#L56

But I am confused if it is respected, because your timeout seems to be 2 seconds. Either you are running an old version or I have to check it.

10

the version I am running is 4.0.3.
and I tried to add the environment variable CONTENTS__DEFAULTPAGESIZEGRAPHQL to update it to 20, but it still the same error.

11

CONTENTS__DEFAULTPAGESIZEGRAPHQL has nothing to do with the timeout.

12

sorry it is my copy paste error, the pod is set up like this

        - name: RULES__EXECUTIONTIMEOUTINSECONDS
          value: "20"
13

Thx, I will have a look.

14

This should work, I have tested it. The default timeout is 3sec by the way. Perhaps you are running an older version?