What difference of Schema and Component

[sgalcheung]

I’m confused with the component.

I read the docs, but I could not find it, I only found a filed component of filed type.

When should I use it?
Does the component seem like a frontend component?

I already have done some experiments.
I added a component, that contains an HTML field, an input field, a dropdown field,

and I attached the component on Posts, so I can edit it after adding some content.

I also noticed the component doesn’t appear in the Content menu.
So I guess the component seems like a web component, it is just like a schema, contains some basic fields that become a component, and could be reused in schema.

[Sebastian]

Technically they are almost the same.

When you have a component field you can reference a schema that is used for the component field.

A component field is just like a nested document(s), e.g.

{
   "pageElements": {
      "iv": [{
          "type": "article",
          "text": "Hello"
      }, {
          "type": "image",
          "url": "http://"
      }]
   }
}

or for component field:

{
   "pageElement": {
      "iv": {
          "type": "article",
          "text": "Hello"
      }
   }
}

As you can see component fields can have multiple types inside. The shape of the components inside a component field are defined by other schemas. Component schemas are just like normal schemas, but cannot have any content.

[daball]

Here’s one of the ways I have used Components. We’ll use web pages as an analogy for the use case.

I made a simple component on my frontend which I just called a Callout. On the web page it looks like this:

My definition of a Callout component is that it contains a text property.

The interface I described in my Astro Callout Component (TypeScript) looks like:

interface Callout {
    text: string,
}

Later on in the same Astro page component file I render it about like so:

<div class="callout" set:html={text}></div>

This is a fairly simple Component, which is why I chose it to demonstrate it. I then modeled the data for a Callout component.

That Component can be used on a Page. When modeling a Page I chose Multiple contents.

I then defined a field named content (“Content”) as a Components field, which lets you embed other schemas (Components).

After doing that, I then chose the particular Components which I would want to add to the page.content field.

In my case, there are several components which I might add to the page.

Then when creating a Page entity, I can add several of these Components to my Page.

Later on in Astro, I described a Dynamic Component and sifted through the Schema names to get the right Astro Component to handle the particular data returned. The front-end code looks sort of like this:

//...
{componentRouter.map(async (dynComponent) => {
    switch (dynComponent.schemaName) {
        case 'page-callout':
            let callout = dynComponent as PageCallout;
            return (
                <Callout text={renderMarkdown(callout.text, renderContext)} />
            );
//...

renderMarkdown() here is a custom function for my app which uses a particular Markdown processor and renderContext refers to template data that may (or may not) be used during the rendering. However, a cautious reader may notice that PageCallout refers to a different structure. There’s some middleware glue I have as well.

export interface Component {
    schemaId?: string,
    schemaName?: string,
}

export interface PageCallout extends Component {
    text: string,
}

export interface Localized<T> {
    [key: string]: T,
    //in my case I supported these languages for the content
    'en-US': T,
    'es-US': T,
    'fr-CA': T,
};

export interface Page {
//...
    content: Localized<Component[]>,
//...
}

[sgalcheung]

Thank you reply quickly.

[sgalcheung]

That’s interesting, I want to sum up, let me know if I understand you.

1. the frontend has a custom Component, called ‘Callout’.
2. add a corresponding component in Squidex.
3. after modeling a Page(Create Schema), then define a field as a Components field, and also add the component(Callout) link(choose to add Schemas) to the Page.
4. then create a Page entity(Content).

But I only see the component scheme, unlike you can add component entities.

The question is whether the component should be Instantiation. Should Create the component entity?
Hey, I got it, it collapsed.

by the way, I also used Astro and Squidex to build up a website, which is both too new for me, thank you very much, and I also found your repo, I’m reading your Astro code for my study.

[daball]

As to your understanding of my use case for Components, for this use case, you are correct in your understanding. It is not a requirement that you use it in this way, that sort of depends on how you want to use Squidex Components in your own designs, if at all.

While I’m no stranger to web design and database architecture in general, I’m also very new to both Astro and Squidex as well. I’ve been learning as I go for sure. I’m using Astro in SSR mode.

If you want more clues as to how I designed the Squidex schemas:

• I created a Site schema which has for instance, siteName. There’s only one site, so I defined it as a single.
• I created a SEO component and included it on all pages and the site, and anywhere else I need a SEO component.
• I created schemas for my business data models, and components for parts of the business data.
• I created a Page as a multi. I created page components. Pages are for everything that isn’t a business data model but just a page on the site. In my case I have 3 pages: Home page, About page, and 404 page. Everything else is generated from business models.
• On the frontend I routed my URLs to particular Astro handlers for whatever base Schema would be rendering, based in part on a slug cache with references back to the underlying schema data.

To better understand why Components are what they are, perhaps we can peel back a layer from Squidex. Squidex is built as a .NET client to a MongoDB service. The underlying data we observe from Squidex is a projection of the MongoDB data. MongoDB is a noSQL database which, instead of storing relational data (like MySQL, Postgres, Oracle, MS SQL Server, SQLite, etc.) or key-value data (like Redis, etc.), it stores BSON documents. BSON is like JSON but with support for binary blobs of data in MongoDB. If we examine the MongoDB manual, we can see what these BSON documents look like.

So what a Component is, it’s like a defined reusable excerpt of the BSON data, as a complex type, for a portion of a whole MongoDB document, and for the purposes of creating an editor UI for the reusable excerpt of the BSON document. Embedding a Component is like creating an object structure within the object structure, like in this JSON:

{
   "field": "value",
   "component": {
      "componentField": "value"
   }
}

You can see how the introduction of the Components represent a portion of the overall document by inspecting the JSON document that Squidex projects for us. Notice how the data in the Component becomes an embedded part of the JSON output for the Page schema:

I don’t necessarily expect you will use Components in the exact same way I do. This was actually an approach I discovered in a YouTube video which was quite useful for me in understanding how people are using headless CMS in the wild. The video was about Strapi and I originally started out using Strapi and decided later I wanted to avoid the complexities of relational data.

Content Modeling - how to structure your data in Strapi | YouTube @ 12:48.

While this is for an entirely different headless CMS ecosystem, it is what I used as a design to replicate. The practical difference here between the system described in the video is that Components in Squidex are embedded portions of BSON into your overall MongoDB BSON document whereas Components in Strapi are different tables of the SQL database relational model, which introduces its own complexities. Also I liked that Squidex supports versioning for all single and multi schemas and that it supports custom workflows, which I can use as part of my automation processes.

I was able to replicate the use case from the Strapi video in Squidex. I will comment that building the frontend to adapt to the complexities of dynamic components is a lot of extra work on the frontend, with the effect being that you’ll have a UI to edit your site by adding and removing Components of a Page and the frontend can pick and choose which front-end components match up at that pivot point. There are perhaps other, maybe even better, ways to model your data depending upon the shape of things and how much data you’re modeling and who’s editing it and how frequently your page designs may need to adapt, etc. As developers we tend to create our own complexities, and this was one of the complex things I deliberately added to my design.

I also use a lot of Reference fields, which work similar to (but definitely entirely different from) relationships in a traditional RDBMS. Unlike a Components field, a Reference field is an ID reference to another Schema’s particular data whereas a Component represents an embedded portion of the Component Schema’s document. When you use references in Squidex or MongoDB, you’ll want to pay particular attention to which side you are putting your references on; as there are no two-way references as is the case in traditional relational data models.

The takeaway about Squidex Components is they represent an embeddable portion of the MongoDB BSON. There is some documentation about them on the Squidex docs as well, as you originally mentioned. Looking back at these docs, it should start to all make sense where Squidex Components fit into designing your schema models.

Schemas | Squidex

I have also contrived an example where within a component I have created a relation field. On the part of my Page where I can add components, I have added a Selected Brands List component.

Then I select the relations on the business schema.

You can see how this affects the JSON output from Squidex. You can see the embedded object within the object and the unique ID for the Brand reference.

Then when this renders on the front-end app, the component router knows what to do with it on the front-end.

Actually just kidding. It doesn’t know what to do until I build it.

Done!

        case 'page-brands-list':
            let brandsList = dynComponent as PageBrandList;
            let brandsListDto = (await getBrandsByIds(brandsList.brands.join(',')));
            let listedBrands = brandsDto!.items;
            let listedBrandsHeading: { [key: string]: string } = {
                'en-US': "Brands",
                'es-US': "Marcas",
                'fr-CA': "Marques",
            };
            return (
            listedBrands.length > 0 && <h3 class="section">{listedBrandsHeading[locale]}</h3>
                <ul class="link-card-grid brands" data-squidex-token={pageEditToken}>
                    {listedBrands.sort((a, b) => a.data!.brandName[locale].localeCompare(b.data!.brandName[locale])).map(brandDto => {
                        return (
                            <BrandCard
                                locale={locale}
                                editToken={brandDto.editToken}
                                brand={brandDto.data!}
                            />
                        );
                    })}
                </ul>
            );

This demonstrates a Page Schema with a Page Brands List Component containing References to Brand, a business object.

So that’s kind of cool if you ask me. This demonstrates the way you can use both Components and Relations.

It’s important to think ahead and figure out which problem you are trying to solve.

Let me contrive another possibility. Say my business needs change and I can now sell brand slots on certain pages. I want to store each list of brand slots and then reference the list instead of create the list on each page. Then we refactor. I can leave everything in place while I do. I will create two new Schemas. One for the business model, brands-slots, and one for the page component, page-brands-slots so that I can pick which list we’re looking at on certain pages.

Save us a content for this campaign.

Then make the page component.

Add page-brand-slots to the page.content field.

Replace the Selected Brands List with the new page-brand-slots component.

Now we can see the reference to the brands slots, which has a list of brands.

Add all the glue and the feature to the front-end:

        case 'page-brands-slots':
            let brandsSlots = dynComponent as PageBrandSlots;
            let brandsSlotsDto = (await getBrandSlotsByIds(brandsSlots.brandSlots.join(',')));
            let brandsSlotsListDto = (await getBrandsByIds(brandsSlotsDto.items[0].data!.brands.iv.join(',')));
            let listedBrandSlots = brandsSlotsListDto!.items;
            let listedBrandSlotsHeading: { [key: string]: string } = {
                'en-US': "Brands",
                'es-US': "Marcas",
                'fr-CA': "Marques",
            };
            return (
            listedBrandSlots.length > 0 && <h3 class="section">{listedBrandSlotsHeading[locale]}</h3>
                <ul class="link-card-grid brands" data-squidex-token={pageEditToken}>
                    {listedBrandSlots.sort((a, b) => a.data!.brandName[locale].localeCompare(b.data!.brandName[locale])).map(brandDto => {
                        return (
                            <BrandCard
                                locale={locale}
                                editToken={brandDto.editToken}
                                brand={brandDto.data!}
                            />
                        );
                    })}
                </ul>
            );

Now we have the Page with a Component to a Referenced Brands List with a list of References to a Brand.

Think ahead about how you might want to use the data. But know there are many ways to combine the techniques.

[sgalcheung]

Thanks again for demonstrating both cases in detail, this is enough to make people understand and sort out their thoughts.

Pages have two methods to link Components:

1. Pages(Schema) reference other partial pages(Schema).
2. Pages(Schema) add Components(Components)/Component(Component).

I’m considering using some dynamic page generation by your design, but I think it should be split business data and component data. Also, I recently used GraphQL to fetch business data. I’m still using GraphQL to communicate business data. I notice you use Content API to fetch the component data, I also worry that many requests seem to impact performance, but GraphQL can’t get some component schema fields, but I have to use it. I will try to implement it and test its performance.

[Sebastian]

Do you want to create an article for the docs from that. It is is very comprehensive

[daball]

Yeah, it’s very crude right now and I have been refactoring. Removing domain logic from the Astro components is on my todo list. I was just trying to get it to work that day.

I found these generic interfaces indispensable for modeling schema interfaces with the Squidex Node.js Typescript Client SDK interface:

import { Squidex } from "@squidex/squidex";

export interface Component {
    schemaId?: string,
    schemaName?: string,
}

export interface ContentDto<T> extends Squidex.ContentDto {
    data?: T;
}

export interface ContentsDto<T> extends Squidex.ContentsDto {
    /** The generic content items. */
    items: ContentDto<T>[];
}

export interface Localized<T> {
    [key: string]: T,
    'en-US': T,
    'es-US': T,
    'fr-CA': T,
};

export interface NonLocalized<T> {
    [key: string]: T,
    iv: T,
}

They are useful for modeling schema models such as a single:

export interface Site { //single schema: site
    siteName: Localized<string>,
    homePage: NonLocalized<string[]>, //reference to pages
}

Or multis:

export interface Page {
    title: Localized<string>,
    slug: Localized<string>,
    content: Localized<Component[]>, //embedded components (of any type extending Component)
    seo: Localized<PageSeo>, //embedded component of a specific type
    parentPage: NonLocalized<string[]>, //for breadcrumbs
}

export interface BrandSlots { //multi schema: brand-slots
    slotName: NonLocalized<string>;
    brands: NonLocalized<string[]>, //references to brands
}

Or a component:

export interface PageBrandSlots extends Component { //component schema: page-brand-slots
    brandSlots: string[], //reference to brand-slots
}

And generic functions such as these have been instrumental in adapting the data coming out of the SDK:

export const getContentsUsingJsonQuery = async <T>(schema: SCHEMAS|string, jsonQuery: string|undefined = undefined) => (
    await client.contents.getContents(schema, { q: jsonQuery }, { timeoutInSeconds: TIMEOUT_IN_SECONDS })
) as ContentsDto<T>;

A more generic call might look like:

export const getContents = async <T>(schema: string, request: Squidex.contents.ContentsGetContentsRequest | undefined, options: Contents.RequestOptions | undefined) => (
    await client.contents.getContents(schema, request, options)
) as ContentsDto<T>;

Some of these would possibly make great improvements to the Node.js TypeScript client SDK, which my understanding is that it is generated.

Take note that you cannot call a template function within the Astro TSX code, though I’m not certain you should; if you do, you’ll need to make helper functions that call it for you, such as:

export const enum SCHEMAS {
    BRAND_SLOTS = 'brand-slots',
    PAGES = 'pages',
    SITE = 'site',
    //...
};

//because I chose localized page slugs,
//e.g. `en-US/about` and `es-US/sobre`,
//both refer to the same Page object
//therefore my URLs are constructed using both locale and slug
//set forLang to 'iv' for nonlocalized lookup
export const getContentsByLangSlug = async <T>(schema: SCHEMAS|string, forLang: SupportedLocales|string, slug: string) => (
    await getContentsUsingJsonQuery<T>(schema, JSON.stringify({ filter: { path: `data.slug.${forLang}`, op: 'eq', value: slug }}))
);

export const enum SupportedLocales {
    'en-US' = 'en-US',
    'es-US' = 'es-US',
    'fr-CA' = 'fr-CA',
};

// this one removes the template, if needed inside of Astro TSX
export const getPagesByLangSlug = async (forLang: SupportedLocales|string, slug: string) =>
    await getContentsByLangSlug<Page>(SCHEMAS.PAGES, forLang, slug);

[Sebastian]

@daball

Some of these would possibly make great improvements to the Node.js TypeScript client SDK, which my understanding is that it is generated.

Yes, it is generated, but it does not mean that we cannot extend it with custom code. We cannot touch the generated code though or if we do it, we need some kind of postprocessor, that can be rerun after the code generator.

Usually I would model my component like this:

type Component = Article | BlogPost;

interface Article {
    schemaName: 'article';
    title: string;
}

interface Blog {
    schemaName: 'post';
    text: string;
}

Because then typescript will tell you which properties are available, e.g. something like this:

if (component.schemaName === 'article') {
   let x = component.title;
} else {
   let x = component.text;
}

[daball]

Is the generator an upstream module or in the code base for Squidex?

I think I can agree that putting the schemaName on each type would be a good move. It was however, my intention to follow the document model that the SDK provides us.

When I request data from the SDK, it provides a ContentsDto type. Each item in the data set will then be within the context of ContentsDto.items[i], a ContentDto type. Then inside of each of those I can access the actual schema data as ContentDto.data. This is how the SDK returns the data.

The problem with ContentDto.data is that it is type any. Which then I lose the TypeScript language server Intellisense.

That’s why I extended both ContentsDto and ContentDto to provide the generic interface for my schema objects in the data field.

The way you are putting schemaName and title together on your Schema models isn’t the way the SDK provides data, at least, unless you are using the flatten: true request parameter; then perhaps. I haven’t used the flatten request parameter. I just figured I’d like it better not flat. Perhaps flat would be easier to wrangle though. And all of this sort of ignores the fact that all the fields on the singles and the multis are contained within locale objects, data.field.iv for non-localized, and data.field[locale] for localized. Which is why I wrapped all my fields with Localized<> or NonLocalized<> interfaces in my schema models.

Like for instance this is more like how I might annotate the schema using the extra types, adding in schemaName.

export interface Page {
    title: Localized<string>,
    slug: Localized<string>,
    content: Localized<Component[]>,
    seo: Localized<PageSeo>,
    parentPage: NonLocalized<string[]>,
}

export interface PageDto extends ContentDto<Page> {
    schemaName: SCHEMAS.PAGES,
}

export type PagesDto = ContentsDto<Page>;

[Sebastian]

I think you misunderstood me. I just said I would define the component in your app code differently. As shown above. The schema name in the page does not provide that much value (except for getReferences), but for components it is super helpful if you content can hold more than one component type.

Perhaps we can change the contents client to accept a type argument as well (the code is here btw: )GitHub - Squidex/sdk-node: Node.js SDK for the Squidex API).
But in the meantime I would define your model as you did and use a simple as you have shown above with the getContents function.

I am not sure how we can bring this into the SDK. Your function is very helpful, but it breaks the class based architecture a little bit and therefore I see it more like a good recipe for the documentation and not a part of the SDK. But I will have a look into that.