[DECLINED] Data ingestion form external data sources

Feature Requests Declined Features
1

I guess what as a concept in Squidex is missing are placeholder for data from external data sources.

Therefor 3 things are necessary:

  1. Define a schema for placeholders manually or provide a api definition (maybe swagger). So you have ot provide which methods are in scope and read the definition to generate auto schema for placeholders.

  2. In the schema definition of Squidex to provide a possibility to select data schemas which are applicable for components.

  3. During entering the data (editing the content) a pick list should be available which placeholders are available. If placeholders are used.

So how this works if you consume now Squidex data feeds:
In the data feed you will have then additional data what the schema of place holders is supported and which place holders are in use. So the consumer can decide manually or in an automated way how to get the placeholder portion of data via his repo (which can be from a DB or the real external api service or what ever it is) and search and replace reliably.

Example:

  1. Manually:
{
 "currency": "EUR",
 "value": "1.99"
}

Data definition would be:

{
 "dataFeed1" :
 {
  "currency": "string",
  "value": "number"
 }
}
  1. Schem definition selectors
    Lets say you are adding a rich text field you need kind of dropdown to select from one of the data feeds. Lets call the rich text field “MainContent” and for “MainContent” you select the data feed “dataFeed1”.

3.Content editing
Now if you provide content for “MainContent”

The current price is: {dataFeed1.value} in the currency: {dataFeed1.currency}.

So the placeholder should be proposed either via intelli-sense or just as list of available placeholders which can be used.

Consuming json could look like

{
    "data": {
        "queryHomepageContents": [
            {
                "data": {
                    "mainContent": {
                        "en": {
                            "text": "The current price is: {dataFeed1.value} in the currency: {dataFeed1.currency}.",
                            "dataFeed" : {
                                "source": "dataFeed1",
                                "placeholders": [
                                    { "currency": "string" },
                                    { "value": "number" }
                                ]
                            }
                        }
                    }
                },
                "metaData": {
                    "en": [
                        {
                            "id": "9b24b2a9-43b5-46d3-9c44-3c81582f81a4"
                        }
                    ]
                }
            }
        ]
    }
}

So you have all infos in the squidex retuning data how to deal with those placeholders.

P.S.
Of course to have in Squidex also the possibilities to simulate external data feeds would be great to see in query editor already some useful example data.

Regards
Saqib

2

Somehow you can already implement that with scripting. You can write a schema script that is executed on each query and then you can enrich your data with the result from an external source, e.g. prices.

3

Interesting, will try it out, in what language are the scripts written or must be written?

But apart from this possibility, would it not make sense to have such function more integrated? With your proposed approach I guess (will try out) you can achieve some parts, but if you seriously want better customer experience, then it must be available during content authoring?!

4

Reading in the docs about scripting. Already facing some limits:

There exists some restrictions:

  1. You cannot include external libraries.

  2. You cannot make calls to external services.

  3. Scripts will timeout after 200ms.

5

I didn’t want to say that scripting solves everything, but it can be a workaround.

The docs are not correct,I will update them today. Since a few days you can also make external calls. You can have a look to the tests:

There are also 2 timeouts now: A total timeout of 4seconds for the whole script and 200ms for the synchronous part, e.g. for calculations or regexes.

6