[SOLVED] CLI assets sync error! Manually deleted assets from the disk

[Vitali_Nikolenko]

I have…

• [ x ] Read the following guideline: https://docs.squidex.io/01-getting-started/installation/troubleshooting-and-support. I understand that my support request might get deleted if I do not follow the guideline.

I’m submitting a…

• [ ] Regression (a behavior that stopped working in a new release)
• [ ] Bug report
• [ ] Performance issue
• [ x ] Documentation issue or request

Current behavior

I have a problem with assets. The files are stored in the Assets folder. For some reason, some files were manually deleted from this folder.
Now we need to copy the app to app2 via the CLI.
I try sq.exe sync out testOut and get 404 for the files. The rest of the data exports successfully.

I try sq.exe sync in testOut in another application and the process hangs because it cannot load the missing file.

How can I skip manually deleted assets?
Broken files should be deleted or skipped during export.
There is another option - to restore dummy asset files and delete them manually. But there are a lot of them and I don’t see an automatic tool.

Tried settings
rebuild:assets: true
rebuild:assetFiles: true

In this case I get another error

Expected behavior

Resolve error with manually deleted asset files and data in db when copy to another app via CLI.

Minimal reproduction of the problem

Delete asset file from Assets folder. Then do
.\sq.exe sync out <folder>
.\sq.exe config use <another_appprofile>
.\sq.exe sync in <folder>

Environment

App Name:

• [ x ] Self hosted with docker
• [ ] Self hosted with IIS
• [ ] Self hosted with other version
• [ ] Cloud version

Version: 6.4.0

Others:
Partially touches on this topic

Bulk remove assets from App

We have multiple apps running on an instance. In many cases there is a primary one which has all the assets required for replicating the main app. The problem is that the replicated app will only require a small set of the assets. Currently we remove the assets by clicking the small bin next to each asset in the assets folder. Question/Request: Is there there the option or can there be the option to either empty the whole assets folder on an app or bulk select most of them that was replicate…

[Sebastian]

Why do you do that?

[Vitali_Nikolenko]

I study the possibilities of CLI.
At first, when the files were deleted, the team did not yet know that they should not be manually deleted from the Assets folder)

[Sebastian]

Do you want to ignore broken assets during export (sync out) or import (sync in)?

[Vitali_Nikolenko]

Due to broken assets, it is not possible to make an import (sync in). Ideally, it would be nice to have this option in both cases.

[Sebastian]

Makes sense, but how would you repair them then? I could write the broken assets to a file so that you get a full list.

[Vitali_Nikolenko]

You can add a flag to ignore broken assets so that the user can choose whether to use it or not. Writing to a file is a good idea.
If we lost the blob file, then I think we can’t return the asset?)

[Sebastian]

I have tested it and it was never hanging. I have improved the error message for this but it was just going forward. Perhaps you tried an old version?

It seems that I still do not understand your requirements.

[Vitali_Nikolenko]

I use cli-v8.18.0
Hangup was with sync in. I see the error Could not find file and nothing happens

[Vitali_Nikolenko]

to ignore broken assets during import

[Sebastian]

I could not reproduce it, with the version you mentioned.

EDIT: Now I can.

[Vitali_Nikolenko]

Sorry, result was not hanging when do sync in. After a while I get this error
Do in new version sync in

[Sebastian]

I have deployed a new version that increases the timeout.

[Vitali_Nikolenko]

There is a bug in the last commit. The logger was not passed as a parameter.
I used the version from the master branch, passed the logger and the import worked.