`client.contents.getReferences()` throws error (Node.js, TypeScript)

[daball]

I have…

• Read the following guideline: Troubleshooting and Support | Squidex. I understand that my support request might get deleted if I do not follow the guideline.
• Used code blocks with ``` to format my code examples like JSON or logs properly.

I’m submitting a…

• Regression (a behavior that stopped working in a new release)
• Bug report
• Performance issue
• Documentation issue or request

Current behavior

Either this is a bug or I don’t know what I’m doing. (Hint: The current docs leave more docs to be desired.) The interface seems fairly obvious to me though. And I’m using it the way it seems like the API docs say to use it, and how it appears to be used by the admin UI in the Network tab of my Edge browser Dev Tools.

Which is: give it the schema and the content ID within the schema and it’ll give you the references back from your fields which contain references. So it’s probably not the docs or me or that I’m using wrong. So, this is what’s going on.

Node.js TypeScript await client.contents.getReferences('schema-name', 'valid-uuid') throws:

Error: response: Missing required key "statuses"
    at ...\node_modules\@squidex\squidex\core\schemas\builders\schema-utils\getSchemaUtils.js:25:19
    at Generator.next (<anonymous>)
    at fulfilled (...\node_modules\@squidex\squidex\core\schemas\builders\schema-utils\getSchemaUtils.js:5:58)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Expected behavior

await getReferences() should return ContentsDto instance.

Minimal reproduction of the problem

1. Create a schema that includes a reference field. Assign a valid schema to reference in the admin UI for that reference field.
2. Create and publish a content item for the schema including a valid reference to a content item.
3. Take note of the schema name and content item UUID/GUID. Call the await client.content.getReferences() function using these two as actual parameters for getReferences(schema: string, id: string) formal parameters, schema and id, respectively.
4. Observe the thrown error.

Environment

App Name: dasher-supply

• Self hosted with docker
• Self hosted with IIS
• Self hosted with other version
• Cloud version

Version: 7.14.0.0

Browser:

• Chrome (desktop)
• Chrome (Android)
• Chrome (iOS)
• Firefox
• Safari (desktop)
• Safari (iOS)
• IE
• Edge

Others:

You can possibly observe it crashing here: https://dev.dashersupply.com/brand/coast

The code causing the crash, with console traces along the way:

console.log("routeLookup:", routeLookup!);
const redirectDto = await getRedirectsByPreviousSlug(routeLookup!);
console.log("redirectDto:", redirectDto);
if (redirectDto && redirectDto.items && redirectDto.items.length > 0) {
    console.log("getting redirectReferencesDto for", SCHEMAS.REDIRECTS, redirectDto.items[0].id);
    let redirectReferencesDto = await client.contents.getReferences(SCHEMAS.REDIRECTS, redirectDto.items[0].id);
    console.log("redirectReferencesDto:", redirectReferencesDto);
//...

The console output on the Node.js frontend:

routeLookup: brand/coast
redirectDto: {
  total: 1,
  items: [
    {
      id: 'b4d983d9-94a3-4b6c-8b0e-15a949fa0f19',
      createdBy: 'subject:66b10d503c72d885491f262f',
      lastModifiedBy: 'subject:66b10d503c72d885491f262f',
      data: [Object],
      created: 2024-08-22T04:40:37.000Z,
      lastModified: 2024-08-22T04:40:37.000Z,
      status: 'Published',
      statusColor: '#4bb958',
      editToken: 'eyJhIjoiZGFzaGVyLXN1cHBseSIsInMiOiJyZWRpcmVjdHMiLCJpIjoiYjRkOTgzZDktOTRhMy00YjZjLThiMGUtMTVhOTQ5ZmEwZjE5IiwidSI6Imh0dHBzOi8vc3F1aWRleC5kYXNoZXJzdXBwbHkuY29tLyJ9',
      schemaId: 'f0976cf8-e184-4b06-99c3-0a3e6bd56ec7',
      schemaName: 'redirects',
      schemaDisplayName: 'Redirects',
      isDeleted: false,
      version: 1,
      links: [Object]
    }
  ],
  statuses: [
    { status: 'Archived', color: '#eb3142' },
    { status: 'Draft', color: '#8091a5' },
    { status: 'Published', color: '#4bb958' }
  ],
  links: {
    self: { href: '/api/content/dasher-supply/redirects', method: 'GET' },
    create: { href: '/api/content/dasher-supply/redirects', method: 'POST' }
  }
}
getting redirectReferencesDto for redirects b4d983d9-94a3-4b6c-8b0e-15a949fa0f19
02:06:32 [ERROR] Missing required key "statuses"

I temporarily assigned a permission of * to the API role just to make sure it wasn’t permissions. It’s not permissions. There’s something broken. I just don’t know what it is. Either that or I’m using it wrong, and that’s perfectly reasonable considering I’m guessing most of the time.

Using docker logs <container_name> | tail -n 100 I was able to locate the Squidex log for the particular server response which appears to have executed properly.

...
{
  "logLevel": "Information",
  "message": "HTTP request executed.",
  "elapsedRequestMs": 2,
  "filters": {
    "appId": "8d415745-cc4b-41fb-b078-2614504d5497",
    "appName": "dasher-supply",
    "userId": "dasher-supply:dasher-supply-astro",
    "clientId": "dasher-supply:dasher-supply-astro",
    "costs": 1
  },
  "timestamp": "2024-08-22T06:38:17Z",
  "app": {
    "name": "Squidex",
    "version": "7.14.0.0",
    "sessionId": "e9804ee2-84e9-4c3a-8bc7-f0155c521678"
  },
  "web": {
    "requestId": "00-d0c627c783ad2ffec661b7f0949560f2-3ff9dc8aa51be2b9-01",
    "requestPath": "/api/content/dasher-supply/redirects",
    "requestMethod": "GET"
  }
}

{
  "logLevel": "Information",
  "message": "HTTP request executed.",
  "elapsedRequestMs": 13,
  "filters": {
    "appId": "8d415745-cc4b-41fb-b078-2614504d5497",
    "appName": "dasher-supply",
    "userId": "dasher-supply:dasher-supply-astro",
    "clientId": "dasher-supply:dasher-supply-astro",
    "costs": 1
  },
  "timestamp": "2024-08-22T06:38:17Z",
  "app": {
    "name": "Squidex",
    "version": "7.14.0.0",
    "sessionId": "e9804ee2-84e9-4c3a-8bc7-f0155c521678"
  },
  "web": {
    "requestId": "00-93f52d49681e10a211dd212ad5e26122-f9128313c186f0b9-01",
    "requestPath": "/api/content/dasher-supply/redirects/b4d983d9-94a3-4b6c-8b0e-15a949fa0f19/references",
    "requestMethod": "GET"
  }
}

A potential workaround is to create a non-localized string field on the schema adjacent to the reference field, e.g., for a field named referenceField create an adjacent field referenceFieldSchema; then using a Create and Update Script pair in Squidex, automatically track the referenceField’s schema name (which I presume should never change) using the server’s implementation of getReference() which works similarly to the TypeScript SDK’s implementation. Then using the TypeScript client, perform a content item lookup using content.getContents() with the id as referenceField[locale|'iv'] against the schema in referenceFieldSchema.iv.

I don’t have an exact recipe for this Squidex Script yet because I would have to research how to use the server-side implementation of getReference() which probably uses callbacks instead of Promises and async / await. But I believe I could figure out the script faster than I can figure out why the client SDK isn’t working as expected.

[Sebastian]

Thanks a lot for your detailed explanation. I will point to that when I get a bad support request the next time.

I think you are doing nothing wrong. Perhaps the OpenAPI spec does not match to the result in some way and therefore the client throws an error.

[daball]

I’ve been working on that script. This was as far as I could get. It appears that neither getReferences() nor getReference() are working for me, in this case. To facilitate debugging, I added a textarea string field named consoleOut and wrote a console.log() analog called output.log() so I could capture the output on the record itself. Here’s the script I have so far (not working):

/** Script Header **/

//if it went async we need to handle our updates after the callbacks are returned
let wentAsync = false;
//otherwise we can call replace() at the end of the sync script
let didChange = false;
let output = {
    logged: "",
    log: function (/*...args*/) {
        let args = Array.from(arguments);
        //if there is a real console.log() somewhere, call it
        if (console && console.log && typeof console.log === 'function') {
            console.log.apply(console, args);
        }
        let line = '';
        for (let a = 0; a < args.length; a++) {
            let arg = args[a];
            if (typeof arg === 'string') {
                line += arg;
            }
            else {
                line += JSON.stringify(arg);
            }
            if (a < args.length - 1) {
                line += ' ';
            }
            else if (a == args.length - 1) {
                line += '\n';
            }
        }
        output.logged += line;
        ctx.data.consoleOut = { iv: output.logged };
        didChange = true;
    }
}
//if it went async use these three in tandem to decide when to finally
//call replace()
let onReplaceCountExpected = 0;
let onReplaceCount = 0;
const onReplace = () => {
    output.log('onReplace() hit', onReplaceCount+1, "times, expected ", onReplaceCountExpected, "times to call replace()");
    didChange = true;
    if (++onReplaceCount >= onReplaceCountExpected) {
        output.log('onReplace() finalized, calling into replace()');
        replace();
        output.log('onReplace() finalized, calling into complete()');
        complete();
    }
}


/**
 * Workaround: Automatically obtain getReference() on the server because it's
 * broken on the TypeScript client SDK.
 **/

/* In my case the newContent field refers to a new content item to redirect
   the slug in prevSlug field on the Astro frontend. newContent is localized,
   so I have created an array field called newContentSchema which contains
   objects with a schemaName property. Since newContent is localized, so is
   newContentSchema. If newContent were non-localized, be sure that
   newContentSchema is also non-localized. This script should work for both
   localized and non-localized fields, but the fields localization setting
   must match.
   */

/**
 * Gets the referenced schema name from referenceProp and puts it into
 * referenceSchemaProp array of { referenceSchemaPropSubPropName: string }.
 */
function getReferenceSchemaName(referenceProp, referenceSchemaProp, subPropName) {
    output.log("Getting referenced schema name for", referenceProp, 'into', referenceSchemaProp, 'with sub-prop', subPropName);
    // Check if referenceProp is present on ctx.data
    if (ctx.data[referenceProp]) {
        output.log("Referenced property", referenceProp, "does exist.");
        // Get localization/non-localized propery names, these must match on both referenceProp and referenceSchemaProp
        let localeProps = Object.keys(ctx.data[referenceProp]); // e.g. ['en-US', ...] or ['iv']
        output.log("Located localeProps on", referenceProp, localeProps);
        // Provide empty object for referenceSchemaProp
        ctx.data[referenceSchemaProp] = {};
        // Try to find the referenced schema here:
        for (let l = 0; l < localeProps.length; l++) {
            let locale = localeProps[l];
            // Provide empty array for newContentSchema[key].
            ctx.data[referenceSchemaProp][locale] = [];
            // Override default script behavior.
            didChange = true;
            onReplaceCountExpected += localeProps.length;
            output.log("Setting onReplaceCountExpected to", onReplaceCountExpected);
            output.log("Searching for references on", "ctx.data[\"" + referenceProp + "\"][\"" + locale + "\"], which evaluates to", ctx.data[referenceProp][locale]);
            if (ctx.data[referenceProp][locale].length) {
                for (let i = 0; i < ctx.data[referenceProp][locale].length; i++) {
                    output.log("Searching for references on", "ctx.data[\"" + referenceProp + "\"][\"" + locale + "\"][" + i + "], which evaluates to", JSON.stringify(ctx.data[referenceProp][locale][i]));
                    //should set wentAsync here to true, but then the script never gets the callback
                    //the correct behavior would be to set wentAsync to true before the script reaches the script footer
                    //wentAsync = true;
                    getReference(ctx.data[referenceProp][locale][i], function (refs) {
                        // Override default script behavior.
                        output.log("For ctx.data[\"" + referenceProp + "\"][\"" + locale + "\"][" + i + "]  located references ", refs);
                        //For testing purposes, I should be able to stop the callback and report immediate output to the output log using these two calls:
                        //But it seems that the code inside this callback isn't reachable for some reason.
                        //replace();
                        //complete();
                        if (refs && refs.length) {
                            // Provide empty array for ctx.data[referenceSchemaProp][locale][i]
                            ctx.data[referenceSchemaProp][locale][i] = [];
                            for (let r = 0; r < refs.length; r++) {
                                let ref = refs[r];
                                // Provide empty object for contentSchemaProp[key][r][i]
                                ctx.data[referenceSchemaProp][locale][i][r] = {};
                                // Inform newContentSchema of the referenced schema for client.
                                ctx.data[referenceSchemaProp][locale][i][r][subPropName] = ref.schemaName;
                            }
                        }
                        // Inform onReplace we changed some of the data, finally calling
                        // replace after all the keys have been assigned.
                        onReplace();
                    });
                }
            }
        }
    }
}

// Put referenced schema name for newContent into newContentSchema array with
// sub-prop schemaName
getReferenceSchemaName('newContent', 'newContentSchema', 'schemaName');

/** Script Footer **/

//inform Squidex we made changes to the ctx.data
if (!wentAsync) {
    output.log("Exiting script main code block; no callbacks expected to fire");
    if (didChange) {
        output.log("Updated data, calling replace()");
        replace();
    }
    //else {
        output.log("Calling complete()");
        complete();
    //}
}
else {
    output.log("Exiting script main code block; callbacks may still fire")
    output.log("Calling complete()");
    complete();
}

When I set wentAsync to true before the call to getReference() I don’t get any trace data suggesting that the callback never gets called. To further hammer this point home, I put a trace inside the callback for getReference() which would log the output and immediately exit.

When I don’t set wentAsync to true before the call to getReference() or getReferences(), I’m getting these output.log() traces, with no idea why I’m not getting anything in the callback for getReference().

Getting referenced schema name for newContent into newContentSchema with sub-prop schemaName
Referenced property newContent does exist.
Located localeProps on newContent ["en-US","es-US","fr-CA"]
Setting onReplaceCountExpected to 3
Searching for references on ctx.data["newContent"]["en-US"], which evaluates to ["25a7bba7-4614-4956-83e7-f6cc3087737b"]
Searching for references on ctx.data["newContent"]["en-US"][0], which evaluates to "25a7bba7-4614-4956-83e7-f6cc3087737b"
Setting onReplaceCountExpected to 6
Searching for references on ctx.data["newContent"]["es-US"], which evaluates to ["25a7bba7-4614-4956-83e7-f6cc3087737b"]
Searching for references on ctx.data["newContent"]["es-US"][0], which evaluates to "25a7bba7-4614-4956-83e7-f6cc3087737b"
Setting onReplaceCountExpected to 9
Searching for references on ctx.data["newContent"]["fr-CA"], which evaluates to ["25a7bba7-4614-4956-83e7-f6cc3087737b"]
Searching for references on ctx.data["newContent"]["fr-CA"][0], which evaluates to "25a7bba7-4614-4956-83e7-f6cc3087737b"
Exiting script main code block; no callbacks expected to fire
Updated data, calling replace()

So I can’t seem to get either the client SDK to resolve the reference, nor can I seem to get the Script on Squidex’s Update event to resolve the reference.

I’m not sure what to do. I previously had manually typed in the schema name on another schema, which worked for me. But I should be able to evaluate this stuff based on my understanding of both the SDK and the Squidex docs about Scripting.

What do you think I’m doing wrong, here?

[Sebastian]

To be honest I have not read the full thing, because it is super easy:

This is the class in the backend:

public sealed class ContentsDto : Resource
{
    /// <summary>
    /// The total number of content items.
    /// </summary>
    public long Total { get; set; }

    /// <summary>
    /// The content items.
    /// </summary>
    public ContentDto[] Items { get; set; }

    /// <summary>
    /// The possible statuses.
    /// </summary>
    public StatusInfoDto[] Statuses { get; set; }

    public static async Task<ContentsDto> FromContentsAsync(IResultList<EnrichedContent> contents, Resources resources,
        Schema? schema, IContentWorkflow workflow)
    {
        var result = new ContentsDto
        {
            Total = contents.Total,
            Items = contents.Select(x => ContentDto.FromDomain(x, resources)).ToArray()
        };

        if (schema != null)
        {
            await result.AssignStatusesAsync(workflow, schema);
            await result.CreateLinksAsync(resources, workflow, schema);
        }

        return result;
    }
}

As you can see, the schema is nullable, so it is passed over with a null in some cases. Therefore statuses is null and the serializer complains.

I just have to initialize it with an empty list.

[daball]

I’m glad the backend fix is easier than the headache of working around it. Scripting inside of Squidex turns out to be quite difficult to do. I do sort of wonder if both the client SDK’s async client.content.getReferences() and the Squidex internal JavaScript getReference(id, callback) and getReferences(ids, callback) are failing for different reasons, but if at some point I don’t need the workaround, that’s probably best because this script has turned out to be longer than I originally anticipated.

When the fix is incorporated, can you let me know which version number I will need to inform Docker Compose about to do the upgrade?

Also is there an internal console.log() implementation somewhere that I can see the Squidex Script outputs?

[Sebastian]

I have not read the stuff for getReference yet, tbh. It is also better to discuss this in another thread.