Implementing a path to live in the cloud

Hi we have a quick question we have 3 instances of Squidex

  1. Dev
  2. Pre production / QA
  3. Production

We have migrated our Pre Prod and Production to the cloud (SaaS) platform. We used to migrate code and content between environments by backing up and restoring but this option is not available in the SaaS platform. Please can you advise how we should do this? We typically do 2 major releases a month.

Many thanks
Charlie

In general I cannot recommend this approach: If you restore your installation twice a week you have two issues:

  1. You get a lot of zombie apps, archived app that still exist in the system.
  2. Your app is not available during this time.

Most users either write a script or use the CLI to sync settings from one app to another: https://docs.squidex.io/02-documentation/developer-guides/automation-tools#synchronize-all-app-settings-beta

It also depends on your changes. Most people only add schemas and add a few fields here and there, so you cannot destroy your system.

Many thanks for the feedback.

For settings and schemes we will write a script

Can we sync content changes as well using the export and import function?

The idea is that content changes can only be made on Prep Prod rather than production ensuring that Prod Website is secure and free from any “tampering” / mistakes made by Business Users / Developers.

Many thanks
Charlie

Hello,

Further to my previous post we are solving a problem with migrating content between apps in cloud, in order to have a stable development, qa and production environment.

Our aim is to “release” development structure and content into production app. We believe this should be possible to achieve, either through synchronisation or simple backup/restore. Unfortunately, backup and restore is available only on-premise squidex and synchronisation does not support content syncing.

Following your feedback we tried to develop our solution of content syncing, but we have encountered several issues:

  1. Exported content from on-premise squidex cannot be correctly imported to cloud
  2. Assets synchronisation does not include metadata and therefore assets are invalid
  3. Referential integrity between content is not preserved due to dynamically assigned IDs
  4. Some content is not uploaded at all and does not throw any error

Is there a way to do this?

Or is there a plan to enable backing up and restoring apps on cloud, or adding possibility to current synchronization tools to sync assets and content?

It would really come in handy in production apps.

Many thanks
Charlie

In general I think the best opportunity would be to extend the CLI, perhaps together.

About your questions:

Exported content from on-premise squidex cannot be correctly imported to cloud.

Why is that? It is a very general statement.

Assets synchronisation does not include metadata and therefore assets are invalid.

What kind of metadata do you mean? All metadata can be changed later. So part of your sync process should be upload of the asset as well as the change of the metadata in a later process. Some metadata.

Referential integrity between content is not preserved due to dynamically assigned IDs

Thats not true anymore. Most API endpoints do have an optional ID parameter now that can be used to assign a custom ID. But I have to admin that the auto-generated OpenAPI endpoint is probably not up to date at the moment. I will have a look into that.

Some content is not uploaded at all and does not throw any error.

It is a very general statement. If this would be the case, it would be a very big and crucial app and I have my doubts that this is the case, right now. If you are correct it needs to be solved, but perhaps it is just a bug in your synchronizer?

Or is there a plan to enable backing up and restoring apps on cloud, or adding possibility to current synchronization tools to sync assets and content?

There are 2 problems with the restore tool:

  1. It is not safe. Content is not validated and I think there is a change that you can break other apps if your backup contains invalid events. I am not sure about that, but to make it secure and safe to use for everybody is a big deal. The only option would be to use signing, so that a backup gets signed but a platform and you can only restore it if the key is correct, but this makes other scenarios difficult.

  2. The restore tool creates a new app and after a few syncs the system would look like earth after a zombie apocalypse.

I am open for content and asset synchronization in the CLI, but we have to talk about the details and how to deal with conflicts.

Hi Sebastian I am working with Lundegaard and they have been doing the work on this. Jan has developed the CLI code to implement the solution.

Jan has just stated that they are willing to share the CLI code with you so you can integrate it into your toolset. I believe the main issue is around the assets being given new IDs - as this breaks our whole site.

If we can ensure that these are given a custom ID that would resolve our main issue I believe.

Many thanks
Charlie

As I said: You can assign custom IDs now. Have a look to the API or the C# SDK for that.

1 Like

thanks for your prompt reply

Jan has tired assigning customID, but it only works the first time …. then it gives an error about ID validation … he is going to try to replicate it

When Jan runs his script he is getting an error " object can not be uploaded, ID is in use”. Do you know what could be causing this?

If you upload a data and then it check if parameters and meta-datas are correct … if not, it will PUT(update) the data - This when the error occurs

Many thanks
Charlie