SquidexClient: Creating content - Fields with null values are not part of content's data

Support
#1

I have…

I’m submitting a…

  • [ ] Regression (a behavior that stopped working in a new release)
  • [x] Bug report
  • [ ] Performance issue
  • [ ] Documentation issue or request

Current behavior

When using the C# SquidexClient to create content for schemas where some of the values for fields are null, those fields with null values are not in the created content’s data.

E.g.
After creating content for schema with two fields Title and Name, but setting Name as null, this is the data we see in Squidex and the API:

"data": {
        "Title": {
            "iv": "Test"
        }
}

Expected behavior

The content’s data should show the fields with a null value.

E.g.

"data": {
        "Title": {
            "iv": "Test"
        },
        "Name": {
            "iv": null
        }
}

Minimal reproduction of the problem

  1. Create a schema with nullable fields
  2. Use the SquidexClient to create some contents for that schema with one or more of the fields null
  3. Check the content’s data in Squidex on the ‘Inspect’ tab or by querying the API - the fields with null values are not in the data

Environment

App Name: n/a

  • [x] Self hosted with docker
  • [ ] Self hosted with IIS
  • [ ] Self hosted with other version
  • [ ] Cloud version

Version: More stories (#873) (Revision: 377fe120d74085bac86607674b244855cab0215b)

Browser:

  • [x] Chrome (desktop)
  • [ ] Chrome (Android)
  • [ ] Chrome (iOS)
  • [ ] Firefox
  • [ ] Safari (desktop)
  • [ ] Safari (iOS)
  • [ ] IE
  • [ ] Edge

Others:
Noticed this because we have some scripts that fire on the ‘change’ event (specifically when publishing) to populate some fields that are set to ‘Required on publish’, but because those fields were migrated to Squidex with null values they are not in the data so I cannot set the values. Or maybe it is possible but I just don’t know how!

Note that as soon as we have saved the problem content using the Squidex UI, even without making any changes, those fields do exist as expected with null values.

#2

I think the issue is that the json serializer just omits null values.

#3

I have more infos.

The problem is that JSON.NET does not call converters for null values. There is no workaround for this right now. You could change your field definition to something like

class MyData
{
   public JToken Foo { get; set; }
}

or I could introduce a wrapper, something like

NullableValue<T>

But besides that, I do not see an option.

1 Like
#4

Thanks for the explanation, I’ll look into getting a workaround in place.

#5

I have pushed a new version of the client library with a JsonNull<T> wrapper value.

1 Like
#6

Oh ace thanks, will try it out asap!

1 Like
#7

Good morning, can confirm the serialisation is working or us now with null values coming through to Squidex.

Just have issues with the deserialization now (our app ingests payloads from Squidex too) but that’s very likely due to our code so will have another look today with fresh eyes and come back to you with a bit more detail for guidance if I cannot resolve it. For posterity here are some example error messages I am seeing:

System.AggregateException : One or more errors occurred. (Error converting value {null} to type 'Squidex.ClientLibrary.JsonNull`1[System.Nullable`1[System.DateTime]]'. Path 'FieldName.iv', line 12, position 22.)
---- Newtonsoft.Json.JsonSerializationException : Error converting value {null} to type 'Squidex.ClientLibrary.JsonNull`1[System.Nullable`1[System.DateTime]]'. Path 'FieldName.iv', line 12, position 22.
-------- System.ArgumentException : Could not cast or convert from {null} to Squidex.ClientLibrary.JsonNull`1[System.Nullable`1[System.DateTime]].

Newtonsoft.Json.JsonSerializationException: Error converting value 1 to type 'Squidex.ClientLibrary.JsonNull`1[System.Nullable`1[System.Int32]]'. Path 'AnotherFieldName.iv', line 1, position 27742.
 ---> System.ArgumentException: Could not cast or convert from System.Int64 to Squidex.ClientLibrary.JsonNull`1[System.Nullable`1[System.Int32]].
   at Newtonsoft.Json.Utilities.ConvertUtils.EnsureTypeAssignable(Object value, Type initialType, Type targetType)
   at Newtonsoft.Json.Utilities.ConvertUtils.ConvertOrCast(Object initialValue, CultureInfo culture, Type targetType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType(JsonReader reader, Object value, CultureInfo culture, JsonContract contract, Type targetType)
#8

You can register the converter: https://github.com/Squidex/squidex-samples/blob/master/csharp/Squidex.ClientLibrary/Squidex.ClientLibrary/Utils/JsonNullConverter.cs

But you also need the contract resolver, I can make it public:

#9

I was trying to use Squidex.ClientLibrary.Utils.FromJson to resolve the issue as thought it would setup that contract resolver in it’s static constructor but perhaps that is causing me different issues.

Edit: Had a feeling I was just doing something wrong, turns out using Squidex.ClientLibrary.Utils.FromJson does work for us as expected as long as you use it in the right location! I was fixing tests but only applying it to the deserialization of the Payload, not the actual Data object for the entity. :man_facepalming: Not sure if I needed JsonNullContractResolver to be public as made this realisation when testing against 8.20.0 but shouldn’t hurt to be public.

Out of interest do you advise against me using Squidex.ClientLibrary.Utils.FromJson instead of setting up my own JsonSerializerSettings that makes use of JsonNullContractResolver?

#10

No, it is fine, but if you can read from streams directly, it is usally faster.

1 Like
#11

Edit: Sorry realised this morning that of course it works for lists, I had just forgotten to update some of my code to handle it. Thanks for getting this change made so quickly!

Would you expect JsonNull to work for lists or should we be using empty lists when we want the field to exist in Squidex without a value? Atm I think we explicitly state the list is null so that it matches what Squidex sets after creating content with empty lists in the UI.
E.g.

"ListFieldName": {
      "iv": null
 },
#12

Squidex does not really differentiate here. So it is up to you I would say.